List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Jiahong Ouyang, Qingyu Zhao, Ehsan Adeli, Wei Peng, Greg Zaharchuk, Kilian M. Pohl
Abstract
Interpretability is a key issue when applying deep learning models to longitudinal brain MRIs. One way to address this issue is by visualizing the high-dimensional latent spaces generated by deep learning via self-organizing maps (SOM). SOM separates the latent space into clusters and then maps the cluster centers to a discrete (typically 2D) grid preserving the high-dimensional relationship between clusters. However, learning SOM in a high-dimensional latent space tends to be unstable, especially in a self-supervision setting. Furthermore, the learned SOM grid does not necessarily capture clinically interesting information, such as brain age. To resolve these issues, we propose the first self-supervised SOM approach that derives a high-dimensional, interpretable representation stratified by brain age solely based on longitudinal brain MRIs (i.e., without demographic or cognitive information). Called Longitudinally-consistent Self-Organized Representation learning (LSOR), the method is stable during training as it relies on soft clustering (vs. the hard cluster assignments used by existing SOM). Furthermore, our approach generates a latent space stratified according to brain age by aligning trajectories inferred from longitudinal MRIs to the reference vector associated with the corresponding SOM cluster. When applied to longitudinal MRIs of the Alzheimer’s Disease Neuroimaging Initiative (ADNI, N=632), LSOR generates an interpretable latent space and achieves comparable or higher accuracy than the state-of-the-art representations with respect to the downstream tasks of classification (static vs. progressive mild cognitive impairment) and regression (determining ADAS-Cog score of all subjects). The code is available at https://github.com/ouyangjiahong/longitudinal-som-single-modality.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43907-0_27
SharedIt: https://rdcu.be/dnwcE
Link to the code repository
https://github.com/ouyangjiahong/longitudinal-som-single-modality
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper comes back to SOMs (self-organizing maps) as a way to make a deep AE more interpretable. Empirical results on prediction tasks for MRI data on AD and MCI.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper seems to develop some strong/consistent methodology in popular challenges (MRI for AD/MCI with DL/AE). The relatively large database is a plus, too. The return to SOMs (instead of the sometimes over-rated t-SNE) is an originality.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Clarity and structure are the main weaknesses here. The dataflow and the proposed architecture are impossible to understand from the text only. The text should be rewritten starting from a pipeline or data flow (including the AE and SOM).
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Error bars are missing on the results. Given the database size, this information should be of interest.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please improve clarity. The math notation is quite cryptic. The text is obscure and fails at compensating for the lack of a flowchart. The existing figures are not clear. Especially fig. 2 (what is this top 20?). At least a few sentences should be given to justify the choice of the SOM, quite old-fashioned, compared to fashionable alternatives like t-SNE. The R in LSOR is not really explained? R for representation? This is unclear.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The writing style is here a strong shortcoming and drastic changes, rewriting, and illustration effort are needed to make it attractive and understandable. The methodology is good.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The author proposes a stable, interpretable and self-supervised SOM-based framework for longitudinal MRIs, using a stop gradient operator and an extra commitment loss to ensure stable training. Longitudinal consistency regularization is also applied to capture the progression of brain aging and cognitive impairment.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The use of a gradient-stopping operator, which only updates the neighbors of a grid, and a commitment loss, which allows for the optimization of SOM embeddings through gradient descent, stabilizes the training of SOM on MRI data. Additionally, longitudinal consistency regularization effectively captures the process of brain aging.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There is a missing ablation study to demonstrate the importance of the gradient-stopping operator and commitment loss in stabilizing training. The downstream task performance is average.’
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The article provides a relatively detailed introduction on data processing, network structure, and training, which enhances reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The article focuses on stable training and it would be helpful to provide related ablation studies. It could be beneficial to add more datasets to further validate the results.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The article’s structure is clear, and while the level of innovation is moderate, it provides valuable downstream task results. Additionally, the article offers interesting visualizations with a high level of interpretability.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
In this paper, the authors proposed a SOM-based learning framework for diagnosis of AD and prediction of brain age using longitudinal MRIs. Specifically, a longitudinal consistency regularization was including in the model. The proposed model has been evaluated on the ADNI dataset and compared with several approaches.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method is technically sound and the results are convincing. The paper is very well structured and written.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The improvement on the BACC, AUC and other measures were limited.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I believe that the obtained results can, in principle, be reproduced.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In this paper, the authors proposed a SOM-based learning framework for diagnosis of AD and prediction of brain age using longitudinal MRIs. Specifically, a longitudinal consistency regularization was including in the model. The proposed model has been evaluated on the ADNI dataset and compared with several approaches. Generally, the method is technically sound and the results are convincing. My detailed comments are listed below: 1) According to the results in Table 1, the improvement on the BACC, AUC and other measures were limited.
2) There are three parameters in the objective function, how did the author choose the appropriate parameters. 3) The comparison methods are described without providing a context for how they will be implemented. The experimental setting (i.e., parameters) for these methods are missing, i.e., the number of batch size, iteration, etc. 4) In some places, abbreviations of the terms should be defined before use. - Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper has little overlap with my research focus but I am somewhat knowledgeable about some of the topics covered by the paper.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Reviewers agreed that the methods are strong, the ideas are novel, and the results were as expected. The reviewers agreed that the one of the contributions was a better interpretability of results based on the author’s approach. However, there were specific comments about clarity and the organization of the paper, absent error bars, and no large improvement of performance over existing methods, as well as a lack of ablation study. Another highlight of this approach is the use of the (now regarded as classical) SOM embeddings that are incorporated in learning of longitudinal representations. Overall, the method was deemed promising, however, the technical writing effort was lacking.
From the results (Figures 2 and 3), it was not clear in what way the longitudinal consistency term helped. While the SOM grid index showed strong correlations with age, cognitive status etc, no other independent validation was provided. The authors make a statement “Brain aging effects, e.g., enlarged ventricle and brain atrophy, were evident for brains towards the right.” It would be useful to independently show if the imaging attributes that got selected by this approach, yielded better performance when associated with age-related variables compared to other existing approaches. This is missing from the paper.
Author Feedback
We appreciate all the comments from the reviewers. Here is a brief response to the comments:
Response to Review #1: We will largely rewrite the paper to improve the clarity and readability in the final version.
Response to Review #2: Regarding the gradient-stopping operator and commitment loss, we find them very important to learning good SOM embeddings. Based on our experiments, we found the model stuck in a local minimum where most of the samples cluster to one or a few of the SOM embeddings without these two components.
Response to Review #3: (1) The major contribution of this work is providing an interpretable latent space regarding the brain aging process while maintaining good downstream tasks performance. (2) We choose the regularization weights based on experimental searches. To ensure meaningful latent representation, the reconstruction should be a major loss regularizing the training, which narrowed down the potential weight combinations. Based on this, we further did some grid search and selected the weights based on the correlation of the learned SOM embeddings with brain age. (3) We will include the related details in the final version. (4) We will revise them in the final version.