List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Ruiyan Fang, Yu Li, Xin Zhang, Shengxian Chen, Jiale Cheng, Xiangmin Xu, Jieling Wu, Weili Lin, Li Wang, Zhengwang Wu, Gang Li
Abstract
Brain functional connectivity analysis is important for understanding brain development, aging, sexual distinction and brain disorders. Existing methods typically adopt the resting-state functional connectivity (rs-FC) measured by functional MRI as an effective tool, while they either neglect the importance of information exchange between different brain regions or the heterogeneity of brain activities. To address these issues, we propose a Path-based Heterogeneous Brain Transformer Network (PH-BTN) for analyzing rs-FC. Specifically, to integrate the path importance and heterogeneity of rs-FC for a comprehensive description of the brain, we first construct the brain functional network as a path-based heterogeneous graph using prior knowledge and gain initial edge features from rs-FC. Then, considering the constraints of graph convolution in aggregating long-distance and global information, we design a Heterogeneous Path Graph Transformer Convolution (HP-GTC) module to extract edge features by aggregating different paths’ information. Furthermore, we adopt Squeeze-and-Excitation (SE) with HP-GTC modules, which can alleviate the over-smoothing problem and enhance influential features. Finally, we apply a readout layer to generate the final graph embedding to estimate brain age and gender, and thoroughly evaluate the PH-BTN on the Baby Connectome Project (BCP) dataset. Experimental results demonstrate the superiority of PH-BTN over other state-of-the-art methods. The proposed PH-BTN offers a powerful tool to investigate and explore brain functional connectivity.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_32
SharedIt: https://rdcu.be/dnwNx
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper offers a new perspective on modelling the brain network as a heterogeneous graph with multiple types of path-based features under the prior knowledge of brain partitions
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Using heterogeneous graph to investigating brain network is very interesting. However, brain functional network is not heterogeneous.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Regarding brain functional network as a heterogeneous graph is not appropriate. The contents are confused.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This manuscript seems to stack path convolution and transform which are from the existing methods. The details of the method are not clear. It is hard to perform the reproducibility of the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
How do authors define “heterogeneity of brain activities”? Why are brain activities heterogeneous?
-
Brain network defined on anatomical connectivity of brain structure is usually called brain structural network. Authors say “….reflect anatomical connectivity of brain structure” in introduction.
-
Are brain diseases related to age and sex? What is baby brain? Are age and gender prediction based on brain functional network related to baby brains?
-
Brain functional network is usually homogeneous. The brain region is regarded as a node and functional connectivity is regarded as edge. Why do authors say brain functional network is heterogeneous? In Dataset and Implementation Details, authors achieved rs_FC matrices by Pearson correlation. The rs_FC matrix is a brain functional network and is homogeneous. Hence, it is wrong to say “brain functional network is heterogeneous”! Authors provide the evidences in [7, 22, 30]. However, the contents in [7] are not related to “heterogeneous”. The Paper in [22] and [30] proposed a heterogeneous graph neural network or multimodal brain neuroimaging. Meantime, the work in [22] has not been published and the viewpoints may not be convincing. There are not enough evidences for“brain functional network is heterogeneous”!
-
Authors say “the assumptions of most existing graph-based methods are still far from the reality of the human brain with the following limitations: Ignoring path importance, Neglecting heterogeneity, and Overlooking global structures.” What are the logical relationships among “Ignoring path importance, Neglecting heterogeneity, and Overlooking global structures”? Which methods are related to these three contents? The introduction of describing “Ignoring path importance, Neglecting heterogeneity, and Overlooking global structures” are confused.
-
In visualization of important connections of different age and sex learned by PH-BTN in Fig. 3, why the important connections among insular, occipital, parietal, temporal, and frontal between boy and girl are same?
-
The description of graph path is confused. For example, “…where n denotes that this path is an n-hop path pn”, what does this sentence mean?
-
In 2.2 HP-GTC, authors say “…After brain network construction”, authors do not introduce the processes of constructing brain network!
-
In Fig.2, how do authors define special types? Graph path has simplified brain network. Why do authors use graph path convolution? What are the differences between Eq.(1) and graph path convolution in [19]? Base type and special types are constructed on different brain networks? From Fig.2, we can find that special types and base type are different graph paths. Is it appropriate to use convolution on different graphs?
-
In table 1, what is the full name of MAE and PCC? Why do the results of age prediction have mean and variance?
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The idea of regarding brain functional network as a heterogeneous graph is not appropriate. The contents of this manuscript are confused.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Using ‘heterogeneous’ to indicate functional differences among different types of brain regions is very interesting. The main concerns have been addressed in the rebuttal. Hence, the score can be raised to 5.
Review #2
- Please describe the contribution of the paper
In this paper, authors proposed an interesting brain functional connectivity analysis mode了: PH-BTN. The proposed model considers the path significance and heterogeneity by heterogeneous graph convolution, and incorporate global brain structure and key connections by Transformer mechanism. Overall,the proposed PH-BTN offers a powerful tool to investigate and explore brain functional connectivity.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1 This is a graph model whichis with more interpretability. 2 Extensive experiments. The proposed model has been evaluated related competing models (i.e., four brain backbone models - BrainNetCNN, BrainGNN, PR-GNN and BC-GCN, and three classical deep learning models - MLP, CNN and GCN). 3 the proposed model takes into account the path significance and heterogeneity of the brain simutaneously.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1 There are still a few grammar errors. 2 limited by the pages, there lack of more comparasion with other evaluation index. 3 Limited by the pages, there lack of mode validation on adult data.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The model detail is enough.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In this paper, authors proposed an interesting brain functional connectivity analysis mode了: PH-BTN. There are a several advantages. 1) It offer a new perspective on modelling the brain network, which takes into account the path significance and heterogeneity of the brain, and better simulates the brain network. 2) It proposed a novel Graph Transformer Network structure,which can aggregate rich and crucial path information to generate compact brain representation.
In general, it is well-written and easy to follow. I only have a few suggestions.
1) There are still a few grammar errrors. For instance, Step 2: feature is misspelled as faetures in Fig.1 Concate should probably be concontate in Fig.2.
2) Current validation is conducted on BCP data. If there are validation results on adult data, it would be much better. It may be added in future journal version.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The model novelty and detaild validation experiments.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
In this work, the authors proposed a Path-based Heterogeneous Brain Transformer Network (PH-BTN) for analyzing rs-FC and a Heterogeneous Path Graph Transformer Convolution (HP-GTC) module to extract edge features by aggregating different paths’ information.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The significant contributions are as follows: 1). Develop an innovative perspective on modeling the brain network as a heterogeneous graph with multiple types of path-based features under the prior knowledge of brain partitions 2). Design a novel Graph Transformer Network. 3). Employ a series of extensive experiments on the Baby Connectome Project (BCP) dataset to verify the superiority of the proposed method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1). It would be fascinating to validate the performance of the proposed PH-BTN based on multiple brain atlases and prior partitions. Furthermore, in Figure 1, the authors employed the resting-state (rs) Functional Connectivity (FC) to initialize the path-based heterogeneous brain network. The authors are curious whether rs-FCs are calculated. Is a single rs-FC calculated using the whole brain voxels or based on some atlas? The authors need to provide more details about the rs-FC matrix, such as the matrix size. If the size of rs-FC is extremely large, the reviewers are afraid of the efficiency of the proposed computational pipelines. 2). The reviewers are very curious about the architecture of the proposed LSGNN. In detail, the author needs to describe the hyperparameters, e.g., the number of layers, number of nodes, kernel size (if applied), and the optimization algorithms. For example, do authors employ the prevalent optimizer such as Adam to optimize the proposed HP-GTC? 3). In Table 1, the authors validate the proposed PH-BTN with other peer algorithms. The authors mentioned that the data in bold represents the best results. In this table, it is obvious that MLP can achieve the best std as 0.46. However, the std of the proposed PH-BTN is 1.64, even larger than other std in BrainNetCNN and BC-GCN. In addition, the proposed Ph-BTN is much more complicated than canonical GNNs. Therefore, the authors are suggested to validate PH-BTN with other canonical DNNs using time consumption. Furthermore, the validation results in Table 1 demonstrate that the standard deviation of PH-GTN is larger than other canonical learning methods, e.g., GNN and SVM. 4). The reviewers are very interested in Fig 3 (c). This sub-figure demonstrates the significance of functional connectivity graphs between boys and girls. Meanwhile, reviewers are concerned about how samples may differ, as shown in Fig 3 (b) and (c). Is the difference can be influenced by age and gender simultaneously? That said, the significant difference between boys and girls shown in subfigure (c) can change or not during 1-2 years old.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors do not provide the details on reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
1). Add more details about rs-FC matrices 2). Provide a detailed description of the architecture in the proposed PH-BTN. 3). Please correct the results in Table 1. 4). Clarify the results in Figure 1 (b) and (c).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
More details need to be clarified in this work, such as the size of rs-FC matrix.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The author proposed a brain functional connectivity analysis model which considers the path significance and heterogeneity by heterogeneous graph convolution, and incorporate global brain structure and key connections by the Transformer mechanism. In general, this paper is clearly organized and well-written. The key strength of this paper is to introduce an effective brain functional connectivity analysis framework and to apply it on the BCP dataset. Although it is a very interesting paper, there are some concerns and confusions raised by the reviewers (especially Reviewer 1). The meta-reviewer invites the authors to provide the rebuttal to clarify these major concerns: 1. The heterogeneity feature of functional brain networks; 2. Effectiveness of age and gender prediction on baby dataset. 3. Unclearness of some technical details and descriptions. Please also refer to the detailed comments from each reviewer.
Author Feedback
We thank all the reviewers for their valuable comments and affirmation of our contributions: 1) “a new perspective to model brain” and “consider both path significance and heterogeneity” (all reviewers); 2) “extensive experiments to verify the superiority of the proposed method” (R2, R3); 3) “more interpretability” (R2); and 4) “a novel Graph Transformer network” (R3).
R1 and meta-reviewer questioned why brain functional network is heterogeneous and how to define it. 1) We apologize for the unclear explanation of “heterogeneous” and also recognize the within-region homogeneity of brain functional connectivity; 2) Different brain regions are mainly responsible for different functions. For example, the brain is often divided into five lobes, which are anatomically and functionally distinct (e.g., the temporal lobe for hearing and the occipital lobe for vision). Accordingly, focusing on inter-region analysis, we use “heterogeneous” to indicate functional differences among different types of brain regions; 3) In a brain graph, nodes represent brain regions and paths represent connections among different regions. Mathematically, we model the brain functional network as a heterogeneous graph with multiple types of paths, in which nodes cross lobes are heterogeneous while nodes within lobes are homogeneous; 4) The heterogeneous graph can help the model learn differences in inter-region connectivity more effective and efficient, avoiding the impact of feature assimilation in the homogeneous graph. We will further clarify this concept in the final version.
Reviewers concerned the relevance and effectiveness of age and gender prediction based on baby brain functional networks. 1) Several studies have shown that brain functional networks are closely related to age and gender, even as early as infancy (e.g., Gozdas et al., Human brain mapping, 2019; Lenroot et al., Neuroimage, 2007 and Li et al., IEEE Trans. Medical Imaging, 2022); 2) From our results (Fig. 3) and Supplementary Materials, brain functional connectivity shows strong relation with age and gender. Thus, our work is effective in facilitating a better understanding and exploration of infant brains.
R1 queried why the important connections among different brain lobes between boys and girls are the same. 1) The relative importance of connections among different brain lobes is similar. Specifically, the frontal-parietal connection is found to be most strongly associated with gender [7]; 2) However, the exact coefficients of connections among these lobes vary between boys and girls, like the temporal-occipital connection of boys is stronger than that of girls. We will add these in the final version.
R1 expressed that the processes of constructing brain networks and the full names of MAE and PCC were not given. Please refer to Sec. 3 for full names. The network construction is mainly described in Sec. 2.1 with extra details in Sec. 3 and Supplementary Materials. We provide mean and variance based on 10-fold cross-validation, to better show expectation and stability.
As suggested by R2, we will correct grammar errors in the final version.
R3 requested more details about rs-FC and implementation. 1) The signal rs-FC is calculated by Harvard-Oxford atlas with 112 ROIs, in Sec. 3; 2) Due to the page limit, we have clarified details of rs-FC, hyperparameters and optimizer in Supplementary Materials. More details will be added in the final version.
R3 attentively noted that the std of our model is relatively high. We have found two main reasons accounting for this: 1) the impact of the wide range of age distribution and rapid development of baby brains; 2) the stability of PH-BTN still needs enhancement, which is our focus in future work.
R3 was interested in whether age and gender affect functional connectivity simultaneously. Our answer is yes, and we will further explore it in future work. In this work, we only visualize the mean results of different groups.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The author proposed a brain functional connectivity analysis model which considers the path significance and heterogeneity by heterogeneous graph convolution, and incorporate global brain structure and key connections by the Transformer mechanism. In general, this paper is clearly organized and well-written. The key strength of this paper is to introduce an effective brain functional connectivity analysis framework and to apply it on the BCP dataset.
The authors have satisfyingly addressed the reviewers’ concerns in the rebuttal and all reviewers and AC recommend acceptance of this paper.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal addressed the reviewer comments. The idea of modeling the brain network as a heterogeneous graph that accounts for path-based features is innovative. Similarly, the graph transformer network was deemed a novel contribution. All reviewers recognized that the extensive experimentation in the paper (baby connectome project) was perceived as a strength.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Based on the reviews and the rebuttal the paper has enough merits to be presented in the conference.