Authors

Hye Won Park, Seo Yeong Kim, Won Hee Lee

Abstract

There is significant interest in using neuroimaging data for schizophrenia classification. Graph convolutional networks (GCNs) provide great potential to improve schizophrenia classification using brain graphs derived from neuroimaging data. However, accurate classification of schizophrenia is still challenging due to the heterogeneity of schizophrenia and their subtle differences in neuroimaging features. This paper presents a new graph convolutional framework for population-based schizophrenia classification that leverages graph-theoretical measures of morphometric similarity networks inferred from structural MRI scans and incorporates variational edges to reinforce the learning process. Specifically, we construct individual morphometric similarity networks based on inter-regional similarity of multiple morphometric features (cortical thickness, surface area, gray matter volume, mean curvature, and Gaussian curvature) extracted from T1-weighted MRI. We then formulate an adaptive population graph where each node is represented by the topological features of individual morphometric similarity networks and each edge models the similarity between the topological features of the subjects and incorporates the phenotypic information. An encode module is devised to estimate the associations between phenotypic data of the subjects and to adaptively optimize the edge weights. Our proposed method is evaluated on a large dataset collected from nine sites, resulting in a total sample of 366 patients with schizophrenia and 590 healthy individuals. Experimental results demonstrate that our proposed method improves the classification performance over traditional machine learning algorithms, with a mean classification accuracy of 81.8%. The most salient regions contributing to classification are primarily identified in the middle temporal gyrus and superior temporal gyrus.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43907-0_60

SharedIt: https://rdcu.be/dnwdH

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #3

Please describe the contribution of the paper

This paper proposed a novel graph convolutional network for population-based schizophrenia classification. The node features are constructed through graph-theoretical measures of morphometric similarity networks, where the inter-regional similarity of multiple morphometric features is captured as node features. The weights of edges between nodes model the similarity between subjects, which incorporates both phenotypic information and topological features. The proposed method outperforms several naive methods by a significant margin on a dataset that consists of 366 patients with schizophrenia and 590 healthy controls. The key hyper-parameters and modules are evaluated by sensitivity analysis and ablation study. It is very surprising that the paper interprets the results of the model and identifies the most salient regions contributing to classification, which may important to clinical diagnosis.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The paper verifies the morphometric similarity networks (MSNs) are useful for schizophrenia classification.
2. The paper provides the interpretability of the proposed method, which is important for a clinical diagnosis model, and is able to provide insights for guiding the diagnosis process.
3. The proposed method is novel for the modeling of schizophrenia classification. The novelty of the paper is mainly reflected in integrating MSN-driven features derived from structural MRI and phenotypic information.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The baselines might be a bit too simple. Stronger model need to be considered, such as MLP, and Transformer.
2. It would be better to compare the performance of the proposed method with that of human examination, which is better for evaluating the clinical feasibility.
3. As the number of subjects is limited, it would be better to present the standard deviation of the metric in Table 1.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility is good. Both the key hyper-parameters and the model of computational equipment are described in detail.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
This paper verifies the helpfulness of the MSNs to the schizophrenia classification. The presentation and the organization of the paper are good. I enjoy reading the paper.

It is very surprising that the paper can identify the most salient regions based on the results of the proposed method by using the interpretation method, this is very meaningful for a clinical diagnosis model.

However, I have the following comments to the paper.
1. It would be better to explain the importance of the early detection of schizophrenia for healing at the start of the introduction. This is an important aspect for us to justify the meaningfulness of your work.
2. At the beginning of Section 2, there are some confusing expressions, which are ‘‘… each edge models the similarity between the topological features of the subjects and incorporates the phenotypic information. The edge weights on a graph are adaptively determined by using an MLP-based encoder based on non-imaging data of the subjects.’’ You have mentioned that the edge models the similarity of topological features and phenotypic information earlier, but you say later that the edge weights are adaptively determined based on non-imaging. I observed that in Eq. (1), the weight is conditioned on both. Therefore, I think there needs a more clear expression.
3. It would be better to invite a group of experts to verify whether the interpretable results are useful.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The interpretability experiment, the model design, and the ablation study.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #1

Please describe the contribution of the paper

Paper introduces a strategy to differentiate between patients with schizophrenia and healthy individuals. The strategy consists in a new population graph convolutional network (GCN) model for integrating morphometric similarity networks (MSN)-driven features derived from structural MRI and phenotypic information. Authors provide a comprehensive evaluation that includes: a large cohort of data, sensitivity analysis for key GCN parameters, evaluation of feature selection methods (including a proposed one). Results demonstrated the proposed strategy outperformed baseline machine learning approaches.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- In general, the paper is well written and the methodology is well described.
- The use of a large schizophrenia dataset (n = 956 distributed as 366 patients with schizophrenia and 590 healthy individuals) coming from 6 public databases.
- Comprehensive evaluation including large dataset, sensitivity analysis for key GCN parameters, evaluation of feature selection methods (including a proposed one).
- A comparison with baseline methods (but there is a comment on this regard)
- According to the presented results, this is a promising strategy that demonstrates outperformance with respect to baseline methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Interpretability: Authors described a set of resulting relevant brain regions in differentiating the two classes, but a discussion about the meaning of the top-ranked GCN derived features should be done. How these features could be correlated with the histological findings of schizophrenia (eg, changes at level of cell density, cell size, etc.), and how clinicians could interpret these features for a comprehensive disease understanding?
- In Table 1, standard deviation should be also reported, since metric values are available for each k of the k-fold cross-validation.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Authors provide details about graph structure and parameter values used
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Authors should be careful about references, there is one citation placed as [REF]
- I suggest to include information about data quality control (as supplementary material)
- For comparing methods (SVM, RF, and KNN), did authors perform feature selection as made for the GCN model? In case of a negative answer, can this not be considered unfair with traditional machine learning methods?
- I suggest to run not only 1 iteration of 5-fold, at least 5 iterations, to avoid a bias due to data partitioning.
- It would be interesting to construct a more robust work if authors have an independent cohort to validate the model. For instance, use data from 4 databases (DecNef [24], COBRE [1], CANDI [11] and MCICShare [7]) for 5-fold cross validation, and data from 2 databases (UCLA CNP [3], and CCNMD [19]) to independently validate the model.
- It would be useful if authors are more specific in the main paper to reference a particular table/figure in the Supplementary Material instead of referencing the whole document as “can be found in the supplementary material”.
- Authors said that hand-crafted feature-based machine learning approaches suffer from disadvantages, but many of those hand-crafted features use to provide more interpretable information such as shape or texture features (biological meaning for a disease).
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper presents a novel approach with promising results. Some points should be addressed to improve work robustness. In fact, the presented comprehensive validation with the large cohort of data (composed of 6 public databases) has the potential for a journal paper.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

This paper proposes a new graph convolutional framework for population-based schizophrenia classification that leverages graph-theoretical measures of morphometric similarity networks inferred from structural MRI scans and incorporates variational edges to reinforce the learning process. The proposed method achieves reliable and generalizable classification of 81.8% and identifies the most salient regions in the middle and superior temporal gyrus. The study also investigates the influence of different graph structures and their components on classification performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. Novel formulation: The paper proposes a novel graph convolutional framework for population-based schizophrenia classification that leverages graph-theoretical measures of morphometric similarity networks inferred from structural MRI scans and incorporates variational edges to reinforce the learning process. This approach is unique and has not been explored before in the context of schizophrenia classification.
2. Original way to use data: The paper uses a large multi-site dataset comprising structural T1-weighted MRI scans from six public databases, including DecNef, COBRE, CANDI, MCICShare, UCLA CNP, and CCNMD. The authors extract 1540 morphometric features from each subject and construct individual morphometric similarity networks (MSNs) based on inter-regional similarity of these features. The MSNs are then used as input to the graph convolutional network for classification.
3. Demonstration of clinical feasibility: The proposed method achieves reliable and generalizable classification of 81.8% and identifies the most salient regions in the middle and superior temporal gyrus. This demonstrates the clinical feasibility of using MSNs and graph convolutional networks for schizophrenia classification.
4. Particularly strong evaluation: The paper conducts a comprehensive evaluation of the proposed method on a large schizophrenia dataset, including sensitivity analysis for key parameters in the GCN-based classification framework. The investigation of saliency patterns contributing to GCN classification also identifies the most salient regions in the middle and superior temporal gyrus, providing insights into the underlying neural mechanisms of schizophrenia.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Lack of comparison with state-of-the-art methods: The paper does not compare the proposed method with state-of-the-art methods for schizophrenia classification, such as support vector machines, random forests, or deep learning models. This makes it difficult to assess the performance of the proposed method relative to existing approaches.
2. Limited generalizability: The proposed method achieves reliable and generalizable classification of 81.8%, which is lower than the reported accuracies of some existing methods. The limited generalizability of the proposed method may be due to the use of a specific dataset and the lack of external validation.
3. Lack of clinical validation: The paper does not provide clinical validation of the proposed method, such as comparison with clinical diagnosis or correlation with symptom severity. This limits the clinical relevance of the proposed method.
4. Limited investigation of graph structures: The paper investigates the influence of different graph structures on classification performance but does not provide a comprehensive analysis of the impact of different graph structures on the proposed method. This limits the understanding of the optimal graph structure for schizophrenia classification.
5. Lack of novelty: The use of morphometric similarity networks (MSNs) and graph convolutional networks (GCNs) for disease classification has been explored in prior work, such as the identification of Alzheimer’s disease and mild cognitive impairment using networks constructed based on multiple morphological brain features (Zheng et al., 2018). The proposed method builds on this prior work but does not introduce significant novel contributions to the field.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Despite no information on the availability of code and data, it provides sufficient details to facilitate the replication of the experiments and the evaluation of the proposed method.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Please see my comments for strengths and weaknesses. BTW, a typo in the introduction in page 2: there is a [REF] in the paragraph, which may be a missing reference.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

4
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

While the proposed method shows promise for schizophrenia classification, the lack of comparison with state-of-the-art methods, limited generalizability, lack of clinical validation, and limited investigation of graph structures are major factors that led to my current recommendation.
Reviewer confidence

Somewhat confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper proposed a GCN framework for schizophrenia classification. A large dataset was used, and evaluation was sufficient. There are also some comments from reviewers could be further addressed, such as the interpretability of results, generalizability on other datasets, and clinical feasibility.

Author Feedback

We thank all reviewers and meta-reviewer for their helpful and valuable comments. Reviewers have provided insightful and constructive comments, mainly regarding the interpretability of results, generalizability on other datasets, and clinical feasibility, which we believe, will be taken into account for the final version. Detailed responses are as follows: [R1, R3 – Proofreading] We thank the reviewers for pointing out typos and missing reference. We rectify the typos and perform a full proof read of the submission. [R1, R3 – Standard deviation] We will add standard deviation to Table 1 in the revised version. [R1 – Quality control] We performed strict preprocessing quality control of morphometric neuroimaging data using the ENIGMA quality control procedure (https://enigma.ini.usc.edu/protocols/imaging-protocols/). This information will be added to the revised version. [R1 – Referencing supplementary material] We will be more specific in the revised supplemental material to reference a particular table/figure. [R1, R2, R3 – comparisons with state-of-the-art methods] Thank you for the insightful comments. [R1] We did not perform feature selection for machine learning (ML) based models. We used the upper triangle elements of morphometric similarity networks as input to machine learning models, while the input features of GCN were based the graph-theoretical measures such as strength and clustering coefficient. Future studies should consider the same input feature to ensure a fair comparison between ML-based and GCN models. Moreover, we plan to apply interpretable ML technique to handcrafted feature-based ML models to identify clinically-relevant features and improve our understanding of the underlying neural mechanisms of schizophrenia. [R2, R3] Thank you for giving us the opportunity for clarifying this point. We compared our proposed method to several traditional ML models, namely support vector machine, k-nearest neighbor, and random forests. However, we did not compare the performance of the proposed method to existing deep learning approaches such as CNN or transformer, which will be certainly considered in future studies. [R1 – Cross-valuation] Thank you for the suggestion. We will repeat 5-fold cross-validation 5 times to avoid a bias due to data partitioning. [R2 – Lack of novelty] We would like to highlight the novelty of our work by comparing to a prior study by Zheng and colleagues (2018) raised by the reviewer. They focused only on disease classification for Alzheimer’s disease and mild cognitive impairment while we focused on schizophrenia classification using structural T1-weighted MRI data. They used an AAL atlas to divide the brain into 78 regions to construct the MSNs. Also, they employed sparse linear regression (LASSO) to quantify inter-regional relationships and performed disease classification experiments using an SVM classifier. However, our work used the Desikan-Kilany atlas to divide the brain into 308 regions and extract five morphological measurements, namely cortical thickness, surface area, gray matter volume, mean curvature, and gaussian curvature, to construct the MSN. Our study proposed a new GCN framework for population-based schizophrenia classification that leverages graph-theoretical measures of MSNs inferred from structural MRI scans using large multi-site datasets. [R3 – texts] We will address the importance of the early detection of schizophrenia and also revise text for improved clarity in the revised version. [R1, R3 - Interpretability of results] Thank for this insightful comment. We identified clinically-relevant brain regions in differentiating schizophrenia from healthy individuals. We will certainly consider adding the interpretation of results by correlating graph-thersitical measures in ROIs identified with histological findings of schizophrenia and symptom severity in future studies.

back to top

Graph Convolutional Network with Morphometric Similarity Networks for Schizophrenia Classification