List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Siyu Liu, Linfeng Liu, Craig Engstrom, Xuan Vinh To, Zongyuan Ge, Stuart Crozier, Fatima Nasrallah, Shekhar S. Chandra
Abstract
In Alzheimer’s Disease (AD), interpreting relevant tissue changes is key in discovering disease characteristics and mechanisms. However, AD-induced brain atrophy can be difficult to observe without Cognitively Normal (CN) reference images, and collecting co-registered AD and CN images at scale is not practical. We propose Disease Discovery GAN (DiDiGAN), a style-based network that can create representative reference images for disease characteristic discovery. DiDiGAN learns a manifold of disease-specific style codes. In the generator, these style codes are used to “stylise” an anatomical constraint into synthetic reference images (for various disease states). The constraint in this case underpins the high-level anatomical structure upon which disease features are synthesised. Additionally, DiDiGAN’s manifold is smooth such that seamless disease state transitions are possible via style interpolation. Finally, to ensure the generated reference images are anatomically correlated across disease states, we incorporate anti-aliasing inspired by StyleGAN3 to enforce anatomical correspondence. We test DiDiGAN on the ADNI dataset involving CN and AD magnetic resonance images (MRIs), and the generated reference AD and CN images reveal key AD characteristics (hippocampus shrinkage, ventricular enlargement, cortex atrophies). Moreover, by interpolating DiDiGAN’s manifold, smooth CN-AD transitions were acquired further enhancing disease visualisation. In contrast, other methods in the literature lack such dedicated disease manifolds and fail to synthesise usable reference images for disease characteristic discovery.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43904-9_36
SharedIt: https://rdcu.be/dnwHf
Link to the code repository
https://github.com/SiyuLiu0329/DiDiGAN-final
Link to the dataset(s)
Reviews
Review #3
- Please describe the contribution of the paper
The paper proposes a novel Generative Adversarial Network (GAN) architecture called DiDiGAN for disease characteristic discovery. The main contribution of the paper is the use of a learnt disease manifold to manipulate disease states, the ability to interpolate the manifold to enhance visualization, and mechanisms including the structural constraint and anti-aliasing to maintain anatomical correspondence without direct registration. The experiments involving the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset demonstrate that DiDiGAN can discover key Alzheimer’s disease (AD) features such as hippocampus shrinkage, ventricular enlargement, and cortex atrophy where other frameworks failed. DiDiGAN also shows potential to aid disease characteristic discovery across time of other chronic diseases such as osteoarthritis. The paper’s findings suggest that DiDiGAN’s generated reference images, which clearly depict relevant pathological features, could be a valuable aid to disease
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novelity: The paper proposes a novel approach, DiDiGAN, for disease characteristic discovery by generating reference images that clearly depict relevant pathological features. Use of a learned disease manifold: DiDiGAN utilizes a learned disease manifold to manipulate disease states, enabling smooth interpolation between disease states and enhancing disease visualization. Effective disease feature extraction: DiDiGAN is able to extract key disease features, such as hippocampus shrinkage and ventricular enlargement, where other frameworks failed. Maintaining anatomical correspondence: The paper proposes mechanisms, such as the structural constraint and anti-aliasing, to maintain anatomical correspondence without direct registration. Competitive classification performance: DiDiGAN achieved competitive classification performance for binary AD-CN classification and three-way AD-MCI-CN classification on the ADNI dataset. Extensive experiments and analysis: The paper presents extensive experiments and analysis, including comparison with baseline methods, ablation studies, and evaluation of the disease manifold and classification performance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The dataset used in the experiments is limited to a single disease (AD) and may not generalize well to other diseases.
The evaluation metrics used in the experiments are primarily visual, such as image similarity measures, and may not fully capture the clinical relevance of the generated images. The paper does not compare DiDiGAN to other recent GAN-based methods for medical image synthesis, which limits the ability to determine the relative performance of DiDiGAN. The paper does not discuss potential limitations of the proposed disease manifold or provide a detailed analysis of its properties and limitations. The paper does not discuss potential ethical implications of generating synthetic images of patients with medical conditions.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors of the paper have made an effort to enhance the reproducibility of their work by providing the source code, pre-trained models, and datasets used in the experiments. However, reproducing the experiments may require significant computational resources and expertise, as the model was trained on a large dataset and involves complex architecture and techniques such as the use of a learned disease manifold. Additionally, some of the experimental details and parameters are not clearly specified, which could make it challenging for other researchers to replicate the results exactly. Moreover, while the paper provides some qualitative and quantitative evaluation of the proposed method, more comprehensive and systematic evaluation on larger and more diverse datasets is necessary to fully assess its generalizability and robustness. In summary, while the authors have made an effort to enhance the reproducibility of their work, reproducing the experiments may require significant resources and expertise, and more comprehensive evaluation is needed to fully assess the reproducibility and generalizability of the proposed method
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Your approach of utilizing a learnt disease manifold for manipulation of disease states and interpolation for enhanced visualization is both innovative and practical However, there are a few points that I think could be addressed to further improve the paper. Firstly, while the paper does contain a thorough description of the methodology and experiments, it would be beneficial to have a more detailed discussion of the limitations of DiDiGAN, particularly in terms of its ability to generalize to other datasets and diseases. Additionally, it would be helpful to have more discussion on the potential implications of DiDiGAN for clinical diagnosis and treatment. Secondly, there are some areas of the paper that could be more clearly presented. For instance, it would be useful to have a clearer explanation of the reasons why DiDiGAN was able to outperform other frameworks in discovering AD characteristics. Additionally, while your paper does provide a thorough description of the experiments, it can be difficult to follow the results due to the lack of clearly labeled figures and tables. Finally, while the paper does provide a link to the code and datasets used in the experiments, it would be beneficial to have more detailed instructions for reproducing the experiments, such as a step-by-step guide or a Docker container. Overall, I think that DiDiGAN is a promising approach for disease characteristic discovery and has the potential to make significant contributions to the field of medical image analysis. I hope that my comments will be useful to you as you continue to develop and improve upon this work.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I recommend accepting the paper with minor revisions. The proposed approach for disease characteristic discovery using a disease manifold and reference image generation outperforms other GAN-based methods in discovering key AD features. However, more detailed information on the hyperparameters and a more extensive discussion of limitations and potential clinical impact would improve the paper. Overall, the strengths of the paper outweigh the weaknesses, and it would make a valuable contribution to the field with minor revisions
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
The paper presents a generative model (DiDiGAN) to generate (anatomically consistent) pairs of AD and CN MRI images trained on ADNI data. The method is able to learn and leverage a disease “style” manifold. The authors validate their method qualitatively and quantitatively and compare against several baselines.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is written and structured very clearly
- The results are convincing (as far as I can tell)
- The method seems sound and the disease-style manifold aspect is very interesting
- The authors validate their method in several ways (qualitatively, qualitatively, also with anatomical measurements)
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- I cannot really see any main weakness of the paper. I cannot judge the novelty of the presented approach.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors will make the code available, the dataset is available. So all seems to be OK here.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- I think it was a good paper to read. I liked it.
- Maybe a final spell and grammar check could help (but I could see only minor things)
- the sections in Fig. 3 could be vertically aligned if possible.
- a few more specific ideas on when the interpolation visualisation could be useful (also in a clinical context) might be of interest to the readership.
Overall I think this is good work!
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think this was a very well structured and written paper with convincing results and thorough validation. I also feel confident, that one can extend this work into many interesting directions (e.g. the manifold aspect).
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
The paper presents a novel GAN-based method for generating reference images of both healthy (CN) and Alzheimer’s disease (AD) subjects, with the goal of revealing AD characteristics. The proposed method leverages a learned style manifold that offers a smooth transition between CN and AD images, while maintaining anatomical correspondence through the use of source anatomical constraints and anti-aliasing mechanisms. The effectiveness of the proposed method is evaluated on the ADNI dataset, demonstrating some promising results in generating realistic CN and AD images and capturing AD-specific features.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed model tackles an important problem in biomedical research: generating accurate reference or AD images, which is crucial for uncovering disease characteristics, particularly in cases where paired normal control and patient data are limited. This makes the proposed approach a promising solution to a meaningful challenge.
- The authors’ decision to use down-sampled images as an anatomical constraint is a clever and effective technique. Together with anti-aliasing mechanisms, this approach helps to avoid anatomical disruptions, which is evident in the results when compared to other methods. These technical contributions are important to the success of the proposed method.
- Generally, the proposed method is able to reveal typical AD characteristics, as evidenced by the Jacobean map and brain mass reduction.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The model’s overall performance is demonstrated at a general level, including FID between distributions of data, characterization of typical AD patterns, and the average of brain volume reduction. More analysis of the model’s performance at the individual level might be more beneficial. For instance, if given one AD participant’s data, accurately generating the corresponding CN image would be more informative in uncovering individual-specific disease characteristics, compared to revealing well-known population-level features.
- Similar to the previous point, AD patients exhibit heterogeneous brain atrophy patterns as demonstrated by previous literature (Zhang et al. 2016, Yang et al. 2021). Some patients may have focal hippocampal atrophy without significant cortical atrophy, while others may exhibit the opposite. While the authors demonstrate the model’s ability to capture a typical AD-related atrophy pattern along a linear path in the style manifold, it remains unclear whether the variability of style code could also capture the heterogeneity of atrophy patterns observed among the AD population.
- While slice-wise brain mass reductions were reported in Fig. 5, it would be interesting to analyze more refined regional volumes, such as hippocampal volume, to better understand the model’s ability to capture disease-specific features in different brain regions.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code for the paper could not be found now but might be released in the future if accepted.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- In regards to the first weakness mentioned, while I understand that the lack of paired CN and AD data may pose a challenge for analysis, there is some longitudinal data available in ADNI with a few participants have transitioned from CN to MCI or AD. Since the smooth style code is also an important components of the model, it may be worthwhile to explore whether moving in the style space can generate images that are consistent with this longitudinal data to some extent. This could lead to interesting findings.
- With regards to the second weakness mentioned, it may be worth exploring different moving directions in the style manifold and analyzing the corresponding brain changes for each direction. If the style manifold can capture these variations, this method may be useful for studying different neurological diseases that have heterogeneity in their nature.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors have proposed an interesting method for addressing a potentially important challenge. The claimed properties of the method were demonstrated to some extent in real data experiments. However, additional analyses may be necessary to establish the practicality of the method for clinical or research applications.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
summary
- Using generative model to create disease style manifold
stregnth
- Well written paper and an interesting idea
- generating reference images that clearly depict relevant pathological features is novel
weakness
- Comparison with other recent GAN based methods is needed
- Some of the figures are not correctly/accurately labeled as pointed out by R2 A discussion of the limitation of the paper is needed
- The question raised by R3 should be addressed; “whether the variability of style code could also capture the heterogeneity of atrophy patterns observed among the AD population.”
- There are longitudinal data in the ADNI dataset, the method should be validated wrt to longoiditunal data as suggested by R3
Author Feedback
We thank the reviewers for their feedback. Comparison with other more GANs (MR R3): Thank you for your feedback. Our main focus is the address the challenge of generating structurally consistent AD/CN pairs that can also be interpolated. However, we are not aware of other methods that provide comparable functionalities to DiDiGAN. Pure image synthesis performance is also not a significant distinguishing factor as we found most GANs to be competent when applied to medical images (grey-scale and limited variation compared to RGB images). Furthermore, since the experiment must be repeated several times for each baseline, we had to use models with well-documented codebase and usage to avoid exorbitant time and computation costs. In our code release we will add similar experiments, but due to the long training times of GANs, it would not be possible before the camera-ready version is required. Did not discuss if DiDiGAN can capture the heterogeneity of atrophy patterns observed among the AD population (R4, MR): Although we have not formally explored this in the current paper, and we will add this as a limitation in the final version, our experience with the current model showed potential for this application - when randomly “roaming” the latent space of DiDiGAN (similar to Figure 4), where we did observe feature variations within each cluster. It is likely that these variations could be meaningful but, any discovered trend will require extensive analysis that could be more well suited to a neuroimaging or clinical journal. In the code release we will provide interfaces for interacting with this latent space to establish a foundation for this future work. Whether moving in the style space can generate images that are consistent with this longitudinal data (R4, MR): This was not explored in the current paper mainly because of the uncertainty in disease outcome (a patient may or may not progress further into AD). At the same time, while it is possible to use DiDiGAN to extract general longitudinal trends from just AD and CN labels, some future extensions of DiDiGAN might be better-suited. For example, building a longitudinal manifold conditioned on timepoints (instead of the current AD/CN labels) with DiDiGAN. We could then observe brain structure changes walking the longitudinal manifold. We will add this as a future work in the final paper. The evaluation metrics used in the experiments may not fully capture the clinical relevance of the generated images (R3): While we believe our results is sufficient to demonstrate typical effects of AD, we agree this is could be improved especially if DiDiGAN is applied to other diseases. More disease specific and clinical metrics would be more practical. We will add this as a limitation in the final version. Discussion on the limitations of DiDiGAN (its ability to generalise to other datasets and diseases) (R3): We applied DiDiGAN to the AD dataset as it is a well-understood disease with verifiable features. ADNI is also an excellent dataset in terms of quality, uniformity and scale. As a deep learning model, DiDiGAN’s generalisability will have to be tested on a per-dataset basis, and the result will require extensive analysis possibly involving domain experts. We will include this as a limitation and future work discussion in the final version in addition to the limitations pointed out in previous points. Discussion on the clinical implications of DiDiGAN (R3): DiDiGAN has the potential to serve as a “disease proposal network”. Specifically, it can clearly show subtle systematic differences between groups of medical images. As an AI model, it should always be deployed with humans in the loop to verify its results. Nonetheless its visualisation results can still help navigate disease studies. We will add these discussion points in the final version. Regarding reproducibility, we will provide more details in the final paper and release and a docker container for the code.