List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Hyuna Cho, Guorong Wu, Won Hwa Kim
Abstract
Analyses of longitudinal brain networks, i.e., graphs, are of significant interest to understand the dynamics of brain changes with respect to aging and neurodegenerative diseases. However, each subject has a graph of heterogeneous structure and time-points as the data are obtained over several years. Moreover, most existing datasets suffer from lack of samples as the images are expensive to acquire, which leads to overfitting in complex deep neural networks. To address these issues for characterizing progressively altered changes of brain connectome and region-wise measures as early as possible, we develop Spatio-Temporal Graph Multi-Layer Perceptron (STGMLP) that mixes features over both graph and time spaces to classify sets of longitudinal human brain connectomes. The proposed model is made efficient and interpretable such that it can be easily adopted to medical imaging datasets and identify personalized features responsible for a specific diagnostic label. Extensive experiments show that our method achieves successful results in both performance and computational efficiency on Alzheimer’s Disease Neuroimaging Initiative (ADNI) and Adolescence Brain Cognitive Development (ABCD) datasets independently.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43895-0_73
SharedIt: https://rdcu.be/dnwzF
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
The authors propose longitudinal graph mixer to investigate longitudinal variations of spatio-temporal graphs. Specifically, they designed three modules (including GSM/GTM/STM) to capture the spatial and temporal context information of longitudinal graphs. Comparison with other methods demonstrate significant performance gains in terms of efficiency and accuracy.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is overall well-written and easy to follow, and the performance gain is significant with better efficiency.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1.From the paper, it is not clear how the node features are generated and how is the edge strength calculated.
- The authors claim it unnecessary to directly look at the whole changes, and temporally pooling the neighboring graph pairs are enough to capture the entire dynamics. And they verify this by calculating the weights (W^2). I am wondering why the signs of W^2 can validate the above claims?
- In Eq.(4), the softmax operation needs to be explicitly expressed for clarity
- M in Eq.(5) is not defined
- there is no ablation study on the effectiveness of each module
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
how the node features and edge weights are extracted are not clearly presented
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
please refer to the weakness part
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
the explanation of rationality in designing the GTM module with neighboring graph pairs is not straightforward, thus not convincing
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
based on the authors’ feedbacks and comments from other reviewers, I recommend acceptance of this paper, congrats
Review #3
- Please describe the contribution of the paper
- This paper proposed a new Spatio-Temporal Graph Multi-Layer Perceptron based on Graph Neural Network to predict gradually altered changes of brain connectome.
- The results on both performance and computational efficiency on ADNI and ABCD datasets were impressive.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Besides two above poitns in (3), the paper is well-written. The methodology is novelty and be easy for the readers to follow. The experiments are evaluated on diverse settings.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I have a few questions/suggestions:
-
In the section “Longitudial Graph Classifier”, authors said that “Although the S and T are jointly trained, they exact spatial ….independently”. So does it mean that S and T has another supervised loss function besides the fusing in Eq.(5)? If yes, what are the loss functions for those?
-
The author mentioned about the “interpretable” in the Abstract. So is there any experiment or visualization for this claim?
-
Though results in Table 2 and Table 3 are really good; however it is not clear about the contribution of GSM and GTM independently. Thus adding ablation studies for these networks is important.
-
It would be better if authors can change the “layernorm” in Eq (1), (2) by a notation to make the equations be more clear.
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of the paper is fine to me.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In general, I believe this is a good work and be well-organized. The problem is also interesting and the proposed method is novelty with GNN. Though, authors should clarify the mentioned points in (6) to have a better version.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See (9)
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This paper discusses about Longitudinal Brain Connectome Analysis and proposes the Spatio-Temporal Graph Multi-Layer Perceptron (STGMLP) that mixes features over both graph and time spaces. The proposed methods is proved efficient and interpretable through extensive evaluation on two datasets and comparison with several existing works.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is nicely organized and written, and easy to follow.
- The proposed method has been evaluated with several relevant approaches on two datasets, with 5-fold cross validation. The results are convincing and comprehensive.
- The proposed method is both efficient and effective based on the reported results. The presented analysis about the model interpretability is also interesting.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
No glaring weaknesses. Some important and interesting ablation studies are only available in the supplementary material, which is much better if moved to main text.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The author has guaranteed to release code for reproduce.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
As given in the weakness section, some ablation studies, i.e., the modular decomposition analysis, could be moved to the main text. In that sense, due to page limit, some part of the main text could be condensed, for example the last two paragraphs of the Introduction section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper overall proposes an efficient and effective method, as indicated by the extensive results. Besides, the paper is well-written with comprehensive results analysis, which makes the work solid.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposed a new GNN framework to predict longitudinal changes of brain connectome. The proposed method is both efficient and effective based on the reported results。 All reviewers show great enthusiasm on the paper. Although there are some concerns raised by each reviewer, I believe they are very minor and easy to address. Therefore, I would like to recommend rebuttal so that the authors can provide some discussions on those issues (especially raised by Reviewer #2).
Author Feedback
We thank all reviewers for their constructive reviews. We clarify all concerns raised by the reviewers below.
Q) Rev #2, #3, and #4: Ablation study on each module. A) We provided ablation studies on each module (i.e., GSM, GTM, and STM) in Table 1 of supplementary material due to the page limit. As suggested by Rev. #4, we will shorten the main text and include ablation results in the main paper.
Q) Rev #2: How are node features and edge strength calculated? A) For the ADNI, we used Destrieux atlas to parcellate the cortical surface into 148 regions using T1-weighted MR images by (1) skull stripping, (2) tissue segmentation, (3) image registration. Based on the tissue segmentation result, we measured the average cortical thickness in each region by Freesurfer. Next, we constructed structural connectomes from diffusion-weighted images by a probabilistic tractography algorithm in Freesurfer, resulting in a 148x148 connectivity matrix where each element measures the fiber count between two brain regions. We further normalized the counts of fibers into a probability of region-to-region connectivity for each node. Lastly, we calculated the region-wise average concentration level from amyloid and FDG-PET scans. Cerebellum was used as the reference region to further calculate the SUVR for each pathology modality. For the ABCD, detailed descriptions of the image processing are described in [1] and we used the fully preprocessed node features on Destrieux atlas released at the official ABCD study archive. As in [2], morphometric similarity networks were calculated as 148x148 connectivity matrices, whose element was Pearson correlation of node feature vectors between two regions.
[1] Hagler, et al. Image processing and analysis methods for the Adolescent Brain Cognitive Development Study. NeuroImage 2019. [2] Seidlitz, et al. Morphometric similarity networks detect microscale cortical organization and predict inter-individual cognitive variation. Neuron 2018.
Q) Rev #2: Is temporally pooling the neighboring graph pairs enough to capture the entire dynamics? A) The linear layer within the GTMs captures the variations between two adjacent timepoints, and the following pooling layer extracts the key features across all time-point pairs. To verify its effectiveness, we investigated W^2, which is the weight of a linear layer taking 2F node features from two adjacent timepoints as inputs. As each half of the weights in W^2 is associated with one timepoint, we compared the averaged trained weights of two weight sets corresponding to different timepoints. As the average signs of trained weights are opposite (-7.7 vs. +4.4), it is clear that the linear layer is able to capture the differences between two adjacent time points. Subsequently, taking all activations of all adjacent timepoints as inputs, the pooling layer encompasses the entire time series and captures the key features within the activations.
Q) Rev #2: Explicitly express softmax in Eq. 4. A) Notice that Eq. 4 is the exact formulation of softmax, which is a probability for label c among C number of classes. We will explicitly explain this operation in the revision.
Q) Rev #2: M is not defined. A) The M is the number of samples, which is described in the second line in Sec. 2: Problem Definition.
Q) Rev #3: Clarifications on loss functions for S and T. A) We used only one loss (i.e., Eq. 5), and the S, T, and F are jointly optimized features using the loss. We will improve the clarity in the revision.
Q) Rev #3: Any experiment for interpretable results? A) In Table 3 and Fig. 3, we provided interpretable nodal class activations which show the key ROIs that affect the model to classify a specific clinical label. Also, in Sec. 3 of the supplementary material, we provided analyses on the modality-wise class activation via Grad-CAM as well.
Q) Rev #3: Notation for layernorm? A) We will use a specific notation, e.g., LN(), for layernorm operation.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors addressed all concerns and all reviewers have agreed to accept this paper. Therefore, I recommend accept.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Overall this is a well-written paper with minor concerns from the reviewers. The rebuttal has adequately addressed the reviewers’ comments. Thus I would like to suggest acceptance of the paper.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Based on the reviews and the rubuttal, the papers seems to have value to be presented in the conference. I recommend acceptance.