List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Dongdong Chen, Lichi Zhang
Abstract
Brain connectivity patterns such as functional connectivity (FC) and effective connectivity (EC), describing complex spatio-temporal dynamic interactions in the brain network, are highly desirable for mild cognitive impairment (MCI) diagnosis. Major FC methods are based on statistical dependence, usually evaluated in terms of correlations, while EC generally focuses on directional causal influences between brain regions. Therefore, comprehensive integration of FC and EC with complementary information can further extract essential biomarkers for characterizing brain abnormality. This paper proposes Spatio-Temporal Graph Neural Network with Dynamic Functional and Effective Connectivity Fusion (FE-STGNN) for MCI diagnosis using resting-state fMRI (rs-fMRI). First, dynamic FC and EC networks are constructed to encode the functional brain networks into multiple graphs. Then, spatial graph convolution is employed to process spatial structural features and temporal dynamic characteristics. Finally, we design the position encoding-based cross-attention mechanism, which utilizes the causal linkage of EC during time evolution to guide the fusion of FC networks for MCI classification. Qualitative and quantitative experimental results demonstrate the significance of the proposed FE-STGNN method and the benefit of fusing FC and EC, which achieves $82\%$ of MCI classification accuracy and outperforms state-of-the-art methods.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_7
SharedIt: https://rdcu.be/dnwM8
Link to the code repository
https://github.com/haijunkenan/FE-STGNN
Link to the dataset(s)
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposed spatio-temporal graph neural network with dynamic functional and effective connectivity fusion (FE-STGNN) for MCI diagnosis using resting-state fMRI. In experimental results, the proposed method showed the best performance with 82% MCI classification accuracy.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper proposed a new graph neural network that could combine the information of dynamic functional and effective connectivity, and the proposed method outperformed state-of-the-art methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There were many parameters that use pre-controlled before applying the proposed method to the data.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper was very clearly well written, but the proposed method was complex and had many parameters to be controlled.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
It would be nice if the results included how the parameters mentioned in Sec. 3.1 affected the performance of the proposed method.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method was novel, clearly written, and proven superior with appropriate experiments. However, since there were many parameters to be adjusted in the proposed method, it was not clear that the proposed method would be reproducible.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper introduces an approach for diagnosing mild cognitive impairment (MCI) using rs-fMRI time series data. This is a classical problem in neuroscience that has witnessed a competition between traditional machine learning models and more recent deep learning models. The authors advocate the fusion of functional connectivity (FC) and effective connectivity (EC) to aid in MCI diagnosis and present experimental results on real data, which are compared to a selection of other methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The paper addresses a challenging and highly competitive topic in neuroscience, provides an appropriate rationale for the approach, and compares its results to both traditional machine learning and deep learning approaches.
-
The paper is well-organized and provides a detailed illustration of the proposed framework, making it easy to follow.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
This paper seems to focus on dynamic brain connectivity (including FC and EC), while the title suggests it is relevant to static brain connectivity, which are two different research questions.
-
Although significant work has been done on this topic, including various deep learning approaches in recent years, the authors have not mentioned some relevant related works in deep learning for brain disease diagnosis (the most recent relevant work is from 2017).
-
It is recommended that the authors explain in detail why spatio-temporal features of the brain network are obtained under the guidance of EC networks instead of FC networks. Currently, mainstream brain disease diagnostic methods use FC networks.
-
The comparison methods used in this paper are all from before 2020, and a lot of related work has emerged recently. It is recommended that the authors compare their results to state-of-the-art deep learning approaches.
-
The authors state that they tracked the model in the MCI diagnosis task and evaluated the importance of different brain regions in the disease, providing explanatory evidence based on biomedical knowledge. However, it is recommended that experimental validation be added to confirm whether similar results can be achieved using only the features of the top 5 nodes.
-
The authors should provide the code and processed ANDI data to enhance the credibility of the algorithm results.
-
Overall, the main issue with this paper is the flaws in the experimental design. Firstly, the paper used the ADNI dataset to validate the algorithm’s effectiveness. However, the authors only selected 60 NC subjects and 54 MCI patients, and such a small dataset faces the randomness of the results due to the complexity of deep learning models, which is not generalizable and highly dependent on the selected subjects. In fact, the ADNI database contains many subjects, and for MCI, it is divided into LMCI, EMCI, and so on. Secondly, the compared algorithms in the paper are too outdated. The authors should refer to new methods on IEEE TMI, MICCI, and Medical Image Analysis from 2020 to 2023 for comparison. Finally, the selected indicators for the method are also one-sided, and many important indicators have not been compared.
-
In Fig. 2, the label “a” appears twice. Additionally, it is recommended that the authors make the color corresponding to their method more distinct.
-
This paper proposes a promising solution for MCI diagnosis. However, it can still be improved by addressing the aforementioned weak points.
-
It is suggested that the authors select a larger number of participants, incorporate new comparative algorithms, and enrich the comparison metrics.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
To enhance the credibility of the algorithm results, it is recommended that the author provide the code and processed ANDI data. This would allow other researchers to reproduce and verify the findings, ensuring transparency and reliability in the research.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please modify the article based on the identified weaknesses. Additionally, the following suggestions are recommended: 1.The author predominantly describes a functional connectivity network in both the title and abstract, while repeatedly emphasizing a dynamic functional connectivity network in later sections. The author should clarify the research focus of the paper, as this inconsistency can affect readers’ understanding of the paper. Dynamic and non-dynamic functional connectivity are two different research questions in this field. 2.The author should strengthen the experimental section. For deep learning methods, the randomness of a few dozen participants is not enough to demonstrate the effectiveness of the method. Additionally, the author should compare the proposed method with the latest methods, not just traditional methods. 3.Although this paper aims to identify MCI, the method does not address the characteristics of MCI specifically. It appears that the proposed method could be applied to AD or other diseases as well, and does not have any particular specificity.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The author should strengthen the experimental section. For deep learning methods, the randomness of a few dozen participants is not enough to demonstrate the effectiveness of the method. Additionally, the author should compare the proposed method with the latest methods, not just traditional methods. Although this paper aims to identify MCI, the method does not address the characteristics of MCI specifically. It appears that the proposed method could be applied to AD or other diseases as well, and does not have any particular specificity.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
The authors selected to answer three questions, but unfortunately failed to address the main problem at hand. Specifically, the experimental results cannot be fully trusted due to two factors: firstly, the dataset utilized was too small to draw meaningful conclusions; and secondly, the comparison algorithm employed was outdated, which limits the ability to assess the efficacy and competitiveness of the proposed method.
Review #3
- Please describe the contribution of the paper
This paper proposed a spatio-temporal graph neural network that combines functional and effective connectivity derived from resting state fMRI for mild cognitive impairment (MCI) classification. It first extracts the dynamic functional connectivity (FC) and dynamic effective connectivity (EC) matrices from the resting state fMRI BOLD signals by applying a sliding window over the BOLD signals. Then, it generates the FC and EC embeddings by applying spatial graph convolutions over the dynamic FC and EC matrices respectively. It then fuses these embeddings by using the cross-attention mechanism of transformers and generates the final prediction (MCI classification).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-This paper provides a very good motivation for using the proposed spatio-temporal graph neural network for MCI classification. It provides a nice explanation to the questions: why two different brain connectivity patterns (FC & EC) should be used together for accurate prediction of MCI, why the current GNN based methods fail to model the dynamic brain activity and why is a spatial GCN used instead of a spectral GCN to model the FC and EC matrices. -The fusion of FC and EC matrices to model the dynamic brain connectivity patterns is novel. -The results of FE-STGNN look promising and beats the SOTA methods for similar tasks. -The salient nodes of the brain network is shown by the class activation value, which provides an explanation about the important nodes in the brain that are responsible for accurate MCI diagnosis. This shows that FE-STGNN has some clinical significance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-For the spatial GCNs, it is shown that GCN is performing better than GAT. Now, since the basic GCN (Kipf, T, et. al. “Semi-supervised classification with graph convolutional networks.”) is spectral based, it cannot be used for the directed graphs for generating EC embeddings. Hence, the message passing and updating of spatial GCN is being used as introduced in Message Passing Neural Networks (MPNN). However, there are many recent more powerful spatial GCNs for inductive learning (FE-STGNN is designed for inductive learning) such as GraphSAGE (Hamilton, W, et. al. “Inductive representation learning on large graphs.” NeurIPS, 2017), GIN (Xu, K, et. al. “How powerful are graph neural networks?.”, arXiv, 2018), GraphSAINT (Zeng, H, et. al. “Graphsaint: Graph sampling based inductive learning method.”, arXiv, 2019). These algorithms work for both directed and undirected graphs and have proven to be better than simple MPNN for inductive learning. A comparison should be made with some of these methods applied as spatial GCNs in FE-STGNN such that we know about the maximum expressive power of these graph neural networks (refer to the GIN paper). These advanced GCNs might result in a better FC and EC embedding by capturing more information from the dynamic FC and EC matrices. These advanced GCN methods are easy to implement as there are several libraries available for this purpose such as PyTorch Geometric (PyG) – https://pytorch-geometric.readthedocs.io/en/latest/ and Deep Graph Library (DGL) - https://www.dgl.ai/.
-Why is a positional encoding not used for the EC embeddings in the Fusion Positional Transformer? How do you ensure that that the fusion transformer would keep track of the position of EC embeddings?
-The information about the division of the dataset into train, validation and test is not provided. Are there any test sets or the results of Table 1 and 2 are just reported on the validation sets repeated 10 times for cross-validation. Please explain. Also, in Figure 1 of supplemental materials, are the test subjects for whom the class activation score is shown, taken from one of the folds of cross-validation?
-The proposed method is evaluated on only one dataset (ADNI). Evaluating the proposed method on at least one more similar dataset should be considered to validate this method and make it clinically feasible.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper can be easily replicated. Additionally, the authors have agreed to provide the code once the paper is accepted.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please refer to the weaknesses section for additional comments. In addition to those, I have the following questions and comments:
-In the Fusion Positional Transformer, will it matter if the query-key is interchanged as the goal is to find the similarity between them? Logically it seems that EC embeddings are guiding the FC, so EC embeddings should be the query. Please provide a brief explanation regarding this.
-The shape of the weight matrices for the query, key and value matrices should be mentioned in section 2.2 to have a more formal description. Similarly, the shape of the weights and biases in all the fully connected layers should be mentioned in section 2.2.
-In the loss function, generally Y is denoted as real result and Y_hat is denoted as predicted result. Please consider revising if possible.
-What is NC in section 3.1? Please explain.
-In section 3.1, it is epochs and not epoches in the last sentence.
-For the cross-validation, how do you find those hyperparameters? Do you perform a grid-search to find the optimal hyperparameters or decide the parameters empirically by the performance on the validation set? Please explain briefly.
-In Table 1, the specificity (SPE) for FC+EC with LSTM and GCN should be marked in bold.
-Was there any study done on the temporal resolution in this paper, i.e., if we increase the number of time segments (K) by decreasing the sliding step size (s) for a given window-size (w), would it lead to a better classification performance? A brief explanation and study on the effect of these hyperparameters and temporal resolution should be provided should be provided.
-Why the correlation between same nodes is 0 for generating the FC matrix in the t-th time segment? Shouldn’t this be 1? This should be justified.
-How is the transfer of entropy between two directed nodes measured with the function? And why is it 1 if the nodes are similar? Please provide a brief explanation.
-Why the structural connectivity (SC) matrix is not used in this paper? Is it because it requires a diffusion-weighted MRI scan which is not available here or is it due to the fact that FC matrices already capture the structural characteristics? If the SC matrix would have been available, would it result in a better MCI classification accuracy or is SC matrix not needed for MCI classification? A small justification would suffice here.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The strengths in this paper outweigh the weaknesses. This paper has novelty in terms of the fusion method used and the application for MCI diagnosis. The paper is written clearly and easy to follow.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
The authors addressed some of my concerns in the rebuttal and hence I plan to stay with my rating of accept.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposed a spatio-temporal graph neural network to integrate both functional and effective connectivity via resting state fMRI for mild cognitive impairment diagnosis. The key strength of this paper is to provide an effective fusion framework for resting state fMRI data analysis and application in clinical studies. The whole paper is relatively clearly written. Although there are some merits in this paper, the meta-reviewer as well as most of the reviewers have some concerns and confusions, and invite the authors to provide the rebuttal to clarify these major concerns: 1. The rationale of introducing effective connectivity into MCI diagnosis. 2. Insufficient comparisons with latest SOTA models. 3. Unclearness of certain technical details, parameter settings, and validations. Please also refer to the detailed comments from each reviewer.
Author Feedback
We thank all reviewers and AC for the insightful comments. Our responses to the major concerns are itemized as follows. We will also make further revisions based on our responses in the final version. 1) The rationale of introducing effective connectivity (AC): FC describes brain activity from the perspective of statistical dependencies, while EC refers to the directional causal effects that one brain region exerts over another. EC encodes dynamic information flow that is often neglected in this field, yet it is significant in revealing the dynamic characteristics of the brain network. Therefore, the fusion of FC and EC with complementary information can further extract more comprehensive information for characterizing brain abnormality. 2) Insufficient comparisons with latest SOTA models (AC, R2): We compared seven well-known methods, including three static methods and four dynamic methods. We have further compared the BrainGB (TMI 2023), which resulted in 78.6±1.9% ACC, 75.9±1.7% SEN, 69.4±2.4% SPE, and 74.2±1.8% F-score, which are lower than the proposed method (82.5±2.2%, 82.3±2.9%, 86.7±2.8%, 82.4±2.1%). 3) Unclearness of certain technical details, parameter settings, and validations (AC, R1): We performed ten times 10-fold cross-validation for all the experiments, using a sliding window of w=37 and stride s=10 to construct dynamic brain networks. We trained our model with parameters: \lamda=0.01 weight of the regularization term, 1e−3 learning rate, and a maximum number of 800 epochs. We also listed the detailed settings in Supplementary Material S1. 4) Explain why under the guidance of EC instead of FC networks (R2), will it matter if the query-key is interchanged (R3): As FC focuses on describing the strength of brain connections in each time slice, while EC for characterizing the dynamic information flow. Thus, our motivation is to utilize the causal linkage of EC during time evolution to guide the fusion of FC networks at discrete time slices. Also, it will vary if the query key is interchanged since cross-attention is not a symmetry operation.
Our responses to the reviewers’ specific comments are itemized as follows. For Reviewer #2: 1) Unclear research area due to inconsistency between title and content: The paper focused on dynamic brain networks, and the “spatio-temporal” in the title indicated that graphs are dynamic graphs since static graph has no temporal signature. 2) The author should strengthen the experimental section for larger datasets: Considering that medical imaging data in the real world is often insufficient for deep learning, we randomly selected balanced data with small samples for experiments. We augmented data by integrating FC and EC information and constructing dynamic graphs, thus achieving better performance compared to other methods under the same small dataset. 3) The method does not address the characteristics of MCI specifically: We have conducted specific research on the diagnosis of MCI and explored important brain regions related to MCI diseases. The detailed contents are in Supplementary Material S2 due to page limitations. For Reviewer #3: 1) Suggest trying advanced GCNs such as GraphSAGE or GIN as the graph embedding backbone: It will be interesting to try more GNN backbones in our framework in the future. 2) Are results of Figure 1 in supplemental materials taken from one of the folds of cross-validation: The results are taken from the sum of all validation folds of cross-validation. 3) Explain NC: NC is the abbreviation of Normal Controls. 4) How to find hyperparameters: We perform a grid search. 5) Why not use SC: It requires a diffusion-weighted MRI scan which is not available.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposed a spatio-temporal graph neural network to integrate both functional and effective connectivity via resting state fMRI for mild cognitive impairment diagnosis. The whole paper is relatively clearly written. The authors have provided clear rebuttal to address the major concerns from AC and reviewers. A majority of reviewers retain positive comments on this paper. I also recommend acceptance of this paper.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors addressed most of the concerns from the reviewers. As for the small sample size issue raised by reviewer 2, the size of the data used in this study is reasonable in this area. Also, the authors compared with some method (BrainDB published in 2023), which should answered another major concern raised by reviewer 2. Considering reviewer 1 and 3’s opinions, I would recommend accept this paper.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Based on the reviews and the rebuttal, it looks like the paper need a little bit more work.