Authors

Joël Chavas, Louise Guillon, Marco Pascucci, Benoît Dufumier, Denis Rivière, Jean-François Mangin

Abstract

The human cerebral cortex is folded, making sulci and gyri over the whole cortical surface. Folding presents a very high inter-subject variability, and some neurodevelopmental disorders are correlated to local folding structures, named folding patterns. However, it is tough to characterize these patterns manually or semi-automatically using geometric distances. Here, we propose a new methodology to identify typical folding patterns. We focus on the cingulate region, known to have a clinical interest, using so-called skeletons (3D representation of folding patterns). We compare two models, beta-VAE and SimCLR, in an unsupervised setting to learn a relevant representation of these patterns. We add a decoder to SimCLR to be able to analyse latent space. Specifically, we leverage the data augmentations used in SimCLR to propose a novel kind of augmentations based on folding topology. We then apply a clustering on the latent space. Cluster folding averages, interpolation in the latent space and reconstructions reveal new pattern structures. This structured representation shows that unsupervised learning can help in the discovery of still unknown patterns. We will gain further insights into folding patterns by using new priors in the unsupervised algorithms and integrating other brain data modalities. Code and experiments are available at https://github.com/neurospin-projects/2021_jchavas_lguillon_deepcingulate.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_8

SharedIt: https://rdcu.be/cVD4P

Link to the code repository

https://github.com/neurospin-projects/2021_jchavas_lguillon_deepcingulate

Link to the dataset(s)

https://github.com/neurospin-projects/2021_jchavas_lguillon_deepcingulate/data

Reviews

Review #1

Please describe the contribution of the paper

Technically, the major contribution of this work is the introduction of topology-based augmentations in the SimCLR setting and adding a decoder to SimCLR and analyzing SimCLR and β-VAE reconstructions to recover folding patterns.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This work tackles a challenging task: encoding cortical folding patterns and using the latent layer features to represent and cluster cortex in a unsupervised manner. Adding a decoder to the methods of SimCLR and β-VAE helps to recover folding patterns.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Technical improvement is limited. Technical novelty can only be found in adding a decoder and proposing methods in the preprocessing steps.
2. Interpretation of the results is relatively shallow. Only description of the morphology of clustered folding patterns and whether they are present in the reports are found.
3. The choice of cluster number seems arbitrary.
4. There are no clear conclusion by comparing SimCLR and β-VAE. Also, its application in clinical scenarios is not very clear.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code was provided. Dataset used is publicly open.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

As an application, technical improvement to the existed approaches is acceptable. However, the authors may provide more solid conclusions on the findings, including the shape clusters and comparison across SimCLR and β-VAE. The authors may show the promise of this unsupervised clustering approach in some applications, such as abnormal shape detection.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper attempt to tackle a challenging task while the interpretation of results, conclusion and possible application are not well presented.
Number of papers in your stack

6
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

The author used unsupervised learning to learn latent representation and apply them in discovering folding patterns.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The topic is novel, important, and quiet significant in neuroscience.
2. The paper is clear, easy to follow.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The technique contribution is low. The authors seems to perform a contribution of two methods, in a special data form.
2. This paper seems to be more like exploring folding patterns, rather than developing a new method.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Excellent, the author provided their code, which is a plus.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

The revealing of novel folding patterns is not so convincing, what will you find if you simply increase the cluster number? Also, what does these new folding patterns suggests? do they have any relationship with human cognition or disease? I feel it is not so complete by only exploring patterns.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The topic is interesting. However, This paper is lack of completeness as I mentioned above.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

The paper develops an unsupervised learning framework based on β-VAE and SimCLR models, and affinity propagation clustering algorithm, to explore the folding patterns of cingulate cortex. The method is performed on HCP database with 550 subjects. The experiment results are interesting. Overall, the proposed method is reasonable, but the descriptions are not very clear.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The research topic of exploring cortical folding patterns of human brains is novel and of great importance.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Lack of validation experiments. The description is not very clear, especially for the comparison of patterns discovered by different models. Lack of a comprehensive comparison of cortical folding patterns discovered in this work and related papers.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

I believe that the obtained results can, in principle, be reproduced.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1. In introduction, please cite papers for some sentences, e.g., ‘Contrary to macaque…making it a fingerprint of each individual’. Papers about cortical folding features used as fingerprints for individual identification should be cited.
2. The introduction section is not well-structured, its logic is not very clear, 8 paragraphs in this section are too much for MICCAI paper.
3. In methods section, the hyperparameter βin 1st equation should be mentioned and described.
4. In experiments and results section, please perform cross validation to validate that whether the models applied on different subsets of the dataset can obtain consistent cortical folding patterns, which helps to convince the readers that the result is reliable and reproducible.
5. Please describe the relationship of the folding patterns discovered by two models in both fig. 2 and fig. 3. In my opinion, though the models are different, most of the latent folding patterns should be similar or consistent. But they are not very similar here, please explain why.
6. Lack of a comprehensive comparison of cortical folding patterns discovered in this work and related papers.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The research topic is novel and of great importance, and the methods are reasonable.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

3
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper develops an unsupervised learning framework based on β-VAE and SimCLR models, and affinity propagation clustering algorithm, to explore the folding patterns of cingulate cortex. The method is performed on HCP database with 550 subjects. The experiment results are interesting. Overall, the proposed method seems reasonable.

Strengths:
1. The topic is novel, important, and quiet significant in neuroscience.
2. The paper is clear, easy to follow.
3. The major contribution is the introduction of topology-based augmentations in the SimCLR setting and adding a decoder to SimCLR and analyzing SimCLR and β-VAE reconstructions to recover folding patterns.
4. The method is performed on HCP database with 550 subjects.
Weaknesses:
1. Interpretation of the results is relatively shallow.
2. The choice of cluster number seems arbitrary.
3. There are no clear conclusion by comparing SimCLR and β-VAE. Also, its application in clinical scenarios is not very clear.
4. Lack of a comprehensive comparison of cortical folding patterns discovered in this work and related papers.
Overall, an interesting paper where merits slightly weigh over weakness.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

2

Author Feedback

We thank the reviewers for the overall positive and constructive comments.

This paper is a first step that aims to resolve a challenging task, namely the determination and discovery of cerebral sulcus patterns. We tackled this challenge by using and adapting two classical unsupervised deep learning methods, beta-VAE, and SimCLR.

Regarding the minor points: We added the reference Wachinger et al., 2015 about cortical folding structures used as fingerprints on the camera-proof version. We described the hyperparameter beta. We changed the word « bucket » to describe the skeletons with “point cloud,” a term more generally used in the scientific literature.

The major reviewer points are the suggestions to use the method to find abnormal patterns, the suggestions to link what we found and known cortical folding patterns, and the clinical relevance of the study. We started to use one of the methods, beta-VAE, to detect generated anomalies in folding shapes (Guillon et al., 2021). Detecting natural folding shape anomalies is the object of future work. In the result section “deciphering the patterns,” we attempted to link the obtained results and known cortical foldings patterns in the cingulate region, particularly concerning the shape of the cingulate sulcus and the presence and shape of the paracingulate sulcus described in the literature. We agree that the interpretation remains visual. A quantified validation will require comparing with a manually labelled database, which will be the object of a future study. The clinical relevance of the found latent space or its correlation with other brain data modalities will be the final scope of the work. As described in the introduction, we can point out that several studies have shown a link between folding shape and clinical parameters; we thus expect to find soon such correlations by systematically applying our deep learning methods.

As a final general statement, we hope that the paper will be of general interest to MICCAI community. This paper is one of the first attempts to understand the usefulness of unsupervised deep learning techniques to tackle the challenging task of finding cortical folding patterns. Both approaches (beta-VAE and SimCLR) have their strengths: beta-VAE is good at finding a regularized latent space, in which there is a smooth evolution of cortical folding shapes across the latent space; SimCLR, with these new topological-based augmentations, seems better at obtaining a structured latent space similar to a manifold. This paper thus indicates what may be the next step forwards in such a challenging study. First, we need to get quantitative criteria (using a manually labelled database or correlating the findings with clinical or other brain modality parameters). Second, we want to combine the two techniques, beta-VAE and SimCLR, to get the best of both worlds: regularity and disentanglement.

back to top

Unsupervised Representation Learning of Cingulate Cortical Folding Patterns