Authors

Pengshuai Zhang, Guangqi Wen, Peng Cao, Jinzhu Yang, Jinyu Zhang, Xizhe Zhang, Xinrong Zhu, Osmar R. Zaiane, Fei Wang

Abstract

The functional connectivity (FC) between brain regions is usually estimated through a statistical dependency method with functional magnetic resonance imaging (fMRI) data. It inevitably yields redundant and noise connections, limiting the performance of deep supervised models in brain disease diagnosis. Besides, the supervised signals of fMRI data are insufficient due to the shortage of labeled data. To address these issues, we propose an end-to-end unsupervised graph structure learning method for sufficiently capturing the structure or characteristics of the functional brain network itself without relying on manual labels. More specifically, the proposed method incorporates a graph generation module for automatically learning the discriminative graph structures of functional brain networks and a topology-aware encoding module for sufficiently capturing the structure information in the functional brain networks. Furthermore, we also design the view consistency and correlation-guided contrastive regularizations such that the network parameters can be trained jointly in an end-to-end manner. We evaluated our model on two real medical clinical applications: the diagnosis of Bipolar Disorder (BD) and Major Depressive Disorder (MDD). The results suggest that the proposed method outperforms state-of-the-art methods. In addition, our model is capable of identifying associated biomarkers and providing evidence of disease association. To the best of our knowledge, our work attempts to construct learnable functional brain networks with unsupervised graph structure learning. Our code is available at https://github.com/IntelliDAL/Graph/tree/main/BrainUSL.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43993-3_20

SharedIt: https://rdcu.be/dnwNl

Link to the code repository

https://github.com/IntelliDAL/Graph/tree/main/BrainUSL

Link to the dataset(s)

N/A

Reviews

Review #3

Please describe the contribution of the paper

Authors propose an end-to-end unsupervised graph structure learning method for functional brain networks analysis. They introduce a new contrastive-based loss and they validate their model on two discrimination tasks of psychiatric disorders using resting-state fMRI.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Interesting approach to generate a sparse interpretable graph representing relevant functional connectivities between brain regions
- The proposed formulation is both interpretable and efficient
- Validation on two psychiatric disorders (BD and MDD)
- Transfer learning experiments performed from one pathology to the other
- Plausible discriminative functional connections found for BD and MDD
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Unclear evaluation procedure: the authors seem to perform pre-training/fine-tuning to evaluate their model (from the code) but it is very unclear in the paper. Other strategies exist in the literature (e.g. linear probing) so I strongly recommend to detail this procedure in the manuscript.
- Robustness and statistical tests: the authors should indicate the standard deviation obtained with their 5-fold CV (which should be easy to compute). Statistical tests should be performed to reliably compare the models.
- Missing experimental details: the task used during fMRI is unclear (although I assume it is resting-state). Also, what hyper-parameters did the authors use and which ones did the authors tune ? Based on what criterion (accuracy on BD vs HC, MDD vs HC, both ?). All the DNN are custom and their architecture should be detailed at least in Supplementary. This point is crucial I think given the small sample size and the high risk of over-fitting.
- Thresholding the correlation matrix may be unnecessary, other solutions exist to deal specifically with continuous meta-data, e.g. [1]
[1] Contrastive Learning with Continuous Proxy Meta-Data for 3D MRI Classification, Dufumier et al., MICCAI 2021
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code is available to the reviewer however important experimental details are missing in the manuscript. Data are private for privacy reasons.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

The method is well-sounded and allows for interpretable results through a generative graph model. Empirical results confirm the main claim although lots of experimental details are missing (even in the Supplementary). Given the small sample size (n<500) and the current trend in the field, I would expect some clarifications.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The idea of generating a sparse graph for interpretable graph learning model is elegant and sounded. Experiments support the claims.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

This paper proposed a method for exploring functional brain network with unsupervised graph structure learning method. A correlation-guided contrastive loss is proposed to enforce the learning.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) Graph structure construction is essential to graph based deep learning and functional connectome exploration. This paper proposed an interesting way of unsupervised graph structure learning for fMRI data.
2) Contractive learning is introduced to optimize the learning process of the functional network. 3) The experimental design is reasonable
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1) It would be helpful to provide a more detailed introduction of the proposed model. 2) Since functional connectivity obtained by Pearson’s correlation coefficient (PCC) plays a crucial role in the design of loss functions, a more careful comparison with PCC would be beneficial.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

It is good. The code for the model is provided.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

1) Since 5 times of the 5-fold cross-validation have been implemented during experiments, it would be beneficial to include the standard deviation in the results for comparison. 2) To better highlight the advantage of the proposed BrainUSL, it is necessary to carefully compare the differences between connectivity patterns constructed by PCC and BrainUSL. The functional connectivity obtained by PCC can easily become sparser through thresholding, and the difference shown in Fig. 4 may not fully demonstrate the superiority of BrainUSL. Moreover, it would be interesting to investigate whether BrainUSL can capture neurologically meaningful connections that cannot be learned by a simple thresholding FC matrix. 3) If BrainUSL is used as an end-to-end disease diagnosis model, it may be better to include the ‘prediction loss’ term in Equation (5). 4) Please provide a more detailed description of how to estimate the discriminative ability of the edges.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The idea is innovative, and the experiments are sufficient.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #1

Please describe the contribution of the paper

A novel unsupervised graph structure learning method is proposed for brain functional connectivity analysis.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Contrastive learning is adopted to learn brain connectivity structure in an end-to-end, unsupervised way.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

It is not clear how the learned brain connectivity structure is used for the final diagnosis/classification.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

It is not clear how the learned brain connectivity structure is used for the final diagnosis/classification.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
1. It is not clear how the learned brain connectivity structure is used for the final diagnosis/classification.
2. For the contrastive learning, how was the threshold value theta determined and how will it affect the final performance?
3. How does the trade-off between different terms in the objective function (5) affect the performance?
4. It will be helpful to provide more details about how the top-10 discriminative connections was identified in Fig. 3.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Contrastive learning is adopted to learn brain connectivity structure in an end-to-end, unsupervised way.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper proposes a novel unsupervised graph structure learning approach for brain functional connectivity analysis. Contractive learning is introduced to optimize the learning process of the functional network, which is interesting. In addition, validation is adopted on two psychiatric disorders. However, its weakness lies in the unclear usage of the learned connectivity structure for diagnosis/classification, missing experimental details, and unclear evaluation procedures. More details are expected. Overall, the paper has strengths in its novel approach and reasonable experimental design.

Author Feedback

Thanks to the reviewers for their time and insightful comments. We highly value each of your comments, as they provide crucial insights for improving our work. We have carefully considered your suggestions. Q1: How the learned brain connectivity structure is used for the final diagnosis/classification? (R1, R3 and Meta-R) The brain network structure generated by BrainUSL is utilized through a fine-tuning strategy for downstream classification. For the details of the classification task, we introduced an MLP classifier. Specifically, this stage includes two components: 1) Graph Generation Module, which is not involved in learning during the classification phase. 2) Topology-aware Encoder with MLP for classification, which employs a fine-tuning strategy. Q2: Hyper-parameter analysis. (R1) We have conducted extensive experiments to identify optimal parameter settings. Q3: How the top-10 discriminative connections were identified? (R1) We obtained the average functional brain network of MDD/BD from the pre-training. Then we show the top-10 connections of brain region pairs according to the values of generated brain networks. Q4: The details of classification loss. (R2) We used cross-entropy loss in our model for classification. Q5: Thresholding the correlation matrix may be unnecessary. (R3) Thresholding the correlation matrix is necessary for our approach. Unlike other contrastive learning works that employ data augmentation techniques, our method directly divides the original samples into positive and negative pairs required for contrastive learning. This requires binarization of the correlation matrix to obtain pseudo-labels for positive samples. In conclusion, we would like to express our great appreciation to you and the reviewers for comments on our paper.

back to top

BrainUSL: Unsupervised Graph Structure Learning for Functional Brain Network Analysis