List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yueting Li, Qingyue Wei, Ehsan Adeli, Kilian M. Pohl, Qingyu Zhao
Abstract
The white-matter (micro-)structural architecture of the brain promotes synchrony among neuronal populations, giving rise to richly patterned functional connections. A fundamental question for systems neuroscience is determining the best way to relate structural and functional networks quantified by diffusion tensor imaging and resting-state functional MRI. As one of the state-of-the-art approaches for network analysis, graph convolutional networks (GCN) has been separately used to analyze functional and structural networks, but separate analysis is not able to explore inter-network relationships. In this work, we propose to couple the two networks of an individual by adding inter-network edges between corresponding brain regions, so that the joint structure-function graph can be directly analyzed by a single GCN. The weights of inter-network edges are learnable, reflecting the possibility that the structure-function coupling strength is not uniform across the brain. We applied our approach to predict the age and sex of participants on the public dataset from the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA) based on their functional and structural networks. Our results support that the proposed Joint-GCN outperforms existing multi-modal graph learning approaches for analyzing structural and functional networks.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_22
SharedIt: https://rdcu.be/cVD44
Link to the code repository
https://github.com/YuetingLi666/brain_gcn.git
Link to the dataset(s)
https://cnslab.stanford.edu/data
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a joint graph convolution for both structural and functional brain connectomes.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The jointly analysis of functional and structural barin connectomes might provide some insight for medical use.
The method is simple and effective. The performance shows some improvement.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The proposed method seems to be related to weighted matching. Related methods might be considered in the baselines or at least be discussed.
There are large scale dataser for age predictions. 662 samples are considerably limited in size.
The statistical significance between the proposed method and the baselines is not clearly identified.
The ablation study seems be missing, e.g., fixing the matching using one-on-one matrix?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Code not submitted.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please refer to sec.5
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is interesting but the experimental settings need some improvement.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
5
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The authors proposed a graph convolutional networks (GCN) based model to represent the nodal coupling strength between brain structural and functional network by adding learnable inter-network edges between corresponding brain regions. By employing individual MRI data of 662 participants with 5-fold cross-validation strategy, they showed that this model performs better in age and sex prediction task than previous SVM or GCN methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors designed a novel learnable inter-network edge model to capture coupling strength between SC and FC in different tasks such as age and sex prediction.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The coupling strength is changing according to prediction tasks. It is a results-oriented weighting rather than the intrinsic properties of brain connections. Lots of studies have shown the coupling between SC and FC follows a typical spatial pattern from primary cortex to association cortex. The authors only found a little similarity between their findings and the classical brain function gradients. This is hard to be interpreted.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility is basically good. The method is present clearly.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1) The coupling strength seems show a similar pattern with a typical spatial pattern of structural strength. The authors should test the correlation between coupling strength and raw SC/FC strength across nodes. 2) Whether coupling strength is stable in adult brain network is still a question. The authors should use HCP data to reproduce the analysis. 3) The coupling strength should be corrected by a random label task. 4) The influence of network thresholding should be taken into account.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The author should prove that the coupling model still works in adult data and make a clear interpretation on the meaning of these task-driven coupling values.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This paper proposes a joint graph convolutional neural network to combine brain structural and functional connectome for further application, such as age prediction or gender classification.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of the paper is coupling the structural and functional connectome with inter-network edges between corresponding brain regions.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) lt looks like only the edges between the corresponding ROIs are included as the inter-network edges. The latent hypothesis that the functional and structural connection only limited to the corresponding ROIs is used without theorical support and may affect the performance of the model. 2) The experimental results were obtained from only one time of 5-fold cross-validation. The standard deviation of the results is not reported. 3) The correlation of the predicted and real age is kind of low (~0.38) comparing with the results in other cohorts (Infant: ~0.85, Adult:~0.89).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
1) It is better to clearly describe how the baseline methods were implemented. For example, for the multi-view GCN in comparison, how many views were uses in the experiments is not described. 2) The range of hyper-parameters considered, method to select the best hyper-parameter configuration are not described. 3) 831 individuals (ages 12-21) were recruited in NCANDA dataset. Since there are only 662 were used in this study, it is better to explain the exclusion criteria in the paper. 4) The age distribution for the 1976 pairs of scans can be added.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1) Please analyze why only the edges between the corresponding ROIs are included as the inter-network edges. 2) Since different partition of the data may lead to significant different average performance of the n-fold cross-validation, at least 5~10 times of n-fold cross-validation is recommend to get a relatively objective assessment of the method. It may be better to include the standard deviation of the cross-validation results in the comparison, not only the mean. 3) Please explain why the correlation of the predicted and real age is pretty low (~0.38) comparing with the results in other cohorts (Infant: ~0.85, Adult:~0.89). 4) Why the coupling strength between SC-DC learned in the gender prediction and age prediction is similar should be simply analyzed. It is interesting to know why the gender related-feature is highly correlated with the age related-feature (time-variant).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The rationality is not well explained. The experimental result is not good enough to support the advantage of the proposed model.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
The authors propose to couple SC and FC of an individual by adding inter-network edges between corresponding brain regions, so that the joint structure-function graph can be directly analyzed by a single GCN.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors present an interesting way to combine SC and FC together, so that the joint graph can be processed by a single GCN.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There is less novelty in the methods, and the applications are not attractive.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The results should be reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
It could be more interesting and meaningful to apply the way of combining SC and FC to disease classification, like the applications in references [5] and [6]. How coupling strength varies with aging is also a recommendation.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Less novelty on methods and applications.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
I agree with the reviewers that the method is interesting, but the experiment settings can be improved. The results need to be clearly presented, including significance/effectiveness compared to other methods and cohorts. Also, using DL models, e.g., GCN and others to simultaneously explore brain structural and functional connectivity is not new, some highly related work including MICCAI 20/21 should be mentioned.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Author Feedback
We thank all reviewers and AC for the insightful comments. Reviewers acknowledged our method is effective (R1), novel (R3), and has strength (R2). R4 questioned the novelty but also suggested our method is interesting. We hope the concerns are addressed in the following. Code will be made publicly available.
Missing related work (R1, AC) and limited novelty (R4): Weighted-matching is used for bipartite graph matching, but in our case nodes are already matched by construction. Nevertheless, we will cite prior work from MICCAI 20/21 on joint SC/FC analysis, but note that those prior works are either not GCN-based or not designed for classification/regression (e.g., Lu Zhang et al. MICCAI20). This also highlights our novelty in using GCN to perform prediction using multi-graph input. We tested those non-GCN methods (D’Souza et al. MICCAI21, Jing Yang et al. MICCAI20) that resulted in 73% and 79.6% in sex prediction, 0.22 and 0.31 PCC in age prediction, all lower than ours.
662 is a small sample size for age prediction (R1): There are very few large datasets for age analysis with both rs-fMRI and DTI available. E.g., HCP-Aging and HCP-Development released around the same number of samples (725 and 625). Also note that our data have 1976 longitudinal data points from 662 subjects.
Statistical significance (R1) and more runs of CV (R2) are needed: Joint-GCN was significantly more accurate than all baselines (McNemar and paired t-test for sex and age, all p<0.015). Running 10 times of CV resulted in 84.9+-0.6% and 0.4+-0.01 PCC for sex and age prediction, significantly higher than the best baseline MV-GCN (83.0+-0.6, 0.37+-0.01).
Inter-network edges only connect corresponding ROIs (R2) so ablation setups are needed (R1): The biological motivation that anatomical fiber tracts connected to a region give rise to its high-level cognitive function naturally leads to the design of ROI-specific edges. ROI-wise coupling is also frequently examined in clinical studies (see references [4,13]). We also tested an ablation setting that learns coupling strength between every pair of ROIs. The resulting sex and age prediction was signficantly worse (77% and 0.34PCC).
Age prediction is inaccurate compared to adult and infant studies (R2): Accuracy depends on the age range under investigation. Compared to adolescence, adult aging (especially after 50) and early brain development during infancy have more pronounced brain changes making the prediction an easier task. Also note that studies with higher accuracy typically use anatomical MRIs which we did not.
Why the learned coupling strength is similar (R2) or varies (R3) across tasks: We clarify that despite the training was task-oriented, the learned strength highly resembled between the two tasks. This indicates our data-driven strategy has the potential to reveal the intrinsic brain structural-functional organization. This is supported by our result that the strength correlates with brain function gradients.
Correlation with function gradients is only trend level (R3): We used a coarse parcellation of 80 ROIs so the sample size for the correlation N=80 is small. Increasing the ROI number to 400 or above (as in [4,13]) will increase the statistical power.
Why not correlate with raw SC/FC strength (R3): “The raw coupling strength” is a new concept in the field without standard definition/computation. That is why we prioritized the correlation analysis wrt function gradients, a more standardized metric.
Missing replication on adult data (R3): We agree that HCP-Aging (age 36-100) would be another good dataset for validation. Note that, as also mentioned above, our prediction in the short adolescent period is a harder task.
Unattractive application (R4): Clinical impact is not limited to diseases. Understanding normal brain development and sex differences has been an impactful topic for several decades. Searching for keywords “age prediction MRI’’ in PubMed returned 15,966 articles.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I think the authors have carefuly addressed all the major concerns from reviews. Though the score of this paper is not among the topest in my batch, the quality is deserved to be published in MICCAI conference.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
There is now some work on joint analysis of structural and functional connectomes. However, this paper makes a good contribution to the field. While the results can be improved further, after going through the reviews and the authors response, I recommend the paper be accepted.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The proposed idea of introducing inter-modal regional connections for integrating structural and functional connectivity seems novel. Although it still sounds like a relatively simple concatenation of inter-regional features than a new definition of connection, the rebuttal generally addressed its novelty and effectiveness. Further discussion is needed on the experimental results if they are aligned with recent findings.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
11