Authors

Huy-Dung Nguyen, Michaël Clément, Boris Mansencal, Pierrick Coupé

Abstract

Alzheimer’s disease and Frontotemporal dementia are two major types of dementia. Their accurate diagnosis and differentiation is crucial for determining specific intervention and treatment. However, differential diagnosis of these two types of dementia remains difficult at the early stage of disease due to similar patterns of clinical symptoms. Therefore, the automatic classification of multiple types of dementia has an important clinical value. So far, this challenge has not been actively explored. Recent development of deep learning in the field of medical image has demonstrated high performance for various classification tasks. In this paper, we propose to take advantage of two types of biomarkers: structure grading and structure atrophy. To this end, we propose first to train a large ensemble of 3D U-Nets to locally discriminate healthy versus dementia anatomical patterns. The result of these models is an interpretable 3D grading map capable of indicating abnormal brain regions. This map can also be exploited in various classification tasks using graph convolutional neural network. Finally, we propose to combine deep grading and atrophy-based classifications to improve dementia type discrimination. The proposed framework showed competitive performance compared to state-of-the-art methods for different tasks of disease detection and differential diagnosis.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_6

SharedIt: https://rdcu.be/cVD4N

Link to the code repository

N/A

Link to the dataset(s)

https://ida.loni.usc.edu/

https://adni.loni.usc.edu/

Reviews

Review #2

Please describe the contribution of the paper

The paper describes creating a tool to differentiate between cognitively normal (CN), Alzheimer’s disease (AD), and Frontotemporal disease (FTD) since this is a tough task to do by just using cognitive tests or observing symptomatology. It describes using a deep grading (DG) framework with a support vector machine (SVM) to classify whether to assign a diagnosis of CN, AD, or FTD. Different groupings of data were also tested (AD+FTD+CN vs CN+FTD vs AD+CN vs AD+FTD) in order to test the best classification model.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. This method seems novel in its approach to classification, using an ensemble of 3D-UNets and then applying an SVM to classify the data.
2. Reproducing other methods that have been used in the past to solve the classification problem
3. Testing the DG framework + SVM against other methods, using the same data, in order to test performance of their method.
4. The authors demonstrate a clear understanding of the problem of disease classification, not just from a technical perspective, but from a medical perspective.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Not being as detailed about other models used in the pipeline, such as; everything used in the preprocessing pipeline, details of AssemblyNet used in image segmentation, and details about the SVM used.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
1. Used data from open source MRI databases (ADNI, OASIS, AIBL, MIRIAD, and NIFD)
2. Was not detailed enough in the preprocessing steps, for example, did not talk about algorithms used for each part of the preprocessing pipeline.
3. There is quite a bit of information missing with respect to hyper-parameters used for some of the algorithms used in the paper. For example, AssemblyNet and for the Support vector machine used.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Introduction
1. When mentioning that other methods may be “based on handcrafted features that may not fully exploit the image information”, it would be helpful to then give a few examples of features that these methods may be missing. It would also be helpful to also mention a few features that most methods take into consideration. After that, you could mention why this could be problematic (although this last part is not necessary).
Materials and methods
1. To help with reproducibility, you should talk about which methods you used for preprocessing. For example, for inhomogeneity correction, did you use FSL’s inhomogeneity correction, ANT’s inhomogeneity correction, or did you create your own intensity inhomogeneity correction algorithm? This should be done for each step of the preprocessing pipeline.
2. You mention that you used AssemblyNet for segmentation, however you don’t give any details about the hyper-parameters used for this. You should mention this in a few sentences.
3. The author should include hyper-parameters used for the SVM, it may seem obvious, however you should write the number of hyperplanes chosen.
Interpretation of deep grading map
1. In the first paragraph, instead of saying ‘regions around hippocampus’, you should say “regions included in the hippocampal formation”. I say this because, based on your Figure 2, it seems that the hippocampus, along with other areas around it, are highlighted in the grading map.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The approach that the authors use is unique even though the models that are used within their network have been used. The authors are able to articulate, not just the technical aspects, but also the clinical aspects of the problem. This seems like a tool that can be used in conjunction to approaches currently used in the clinic for deciding treatment methods.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

In this paper, the authors propose a new method to perform specific-disease diag- nosis (i.e., AD vs. CN and FTD vs. CN) and differential diagnosis (i.e., AD vs. FTD and AD vs. FTD vs. CN). Their purpose is to expand the knowledge about dementia sub-types and to offer an accurate diagnosis tool in a real clinical scenario. They extend the recently proposed Deep Grading (DG) framework by training it with multiple types of dementia (i.e., AD and FTD). Furthermore they propose an ensemble of the graph convolutional network and an SVM.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The combination of the graph convolutional network and an SVM for classification is interesting.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- They did not include subjects with early cognitive impairment. The images are available in the databases used.
- It is not clear how they partitioned the data in validation and final testing.
- They do not give details of the implementation of the SVM, kernel or optimization.
- There are no details of the equipment, or acquisition characteristics. There may be a bias because they are images of different characteristics.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors give details about the implemented neural networks, but not about the SVM. The databases used are available. They don’t have the code available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- I suggest adding quantitative results to the abstract
- Justify why subjects with MCI were not included
- What MNI space was used?
- Indicate what software was used for pre-processing
- What kernel did you use for the SVM? And how were the parameters optimized? -In tables 1 and 2 I suggest adding the standard deviation, how many repetitions were made for the results of table 2? Same case for tables 3 and 4. Give details about the data used, standard deviation (if it is the case) and number of repetitions.
- It is not clear how the data was partitioned in validation and final testing. Can you clarify this?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

3
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The contribution is not very novel, it is missing several details of the implemented classifiers and there may be a bias for using different databases without giving details of the images or equipment.
Number of papers in your stack

3
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

This paper aims at developing a machine-learning algorithm to classify multiple sub-types of dementia (e.g., Alzheimer’s disease and Frontotemporal dementia) based on T1-weighted MR images. The authors combine the classification results from a deep-learning-based grading framework that performs disease classification according to the structure-wised evaluation of the tissue abnormality, and an SVM algorithm that classifies the disease type by exploiting volumetric atrophy in dementia patients. The proposed method reveals the abnormal brain structures that are characteristic of each dementia sub-type and allows improved classification results over existing methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The authors explored the potential of using machine-learning methods to classify multiple dementia subtypes, while previous work mainly focused on the binary classification of either Alzheimer’s disease or Frontotemporal dementia.
- The authors performed a careful ablation study for the proposed method and showed some interesting findings such as aggregating data from multiple dementia sub-types may improve the binary classification results, and the deep-learning strategy can provide complementary diagnostic information to the hand-crafted features such as volumetric atrophy.
- The proposed method revealed the signature ROIs for each studied dementia sub-type. The agreement between their findings and previous research work indicated that the proposed method might be clinically useful.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The proposed method seems to be a combination of previously developed diagnostic tools. The innovation may need further improvement.
- The authors claimed that the “early” and accurate classification of dementia sub-types is desired. However, there isn’t sufficient clue of “early” detection demonstrated in their experimental setup and results.
- The number of subjects in the Frontotemporal dementia might be too small (45 for training and 29 for testing) to reach a concrete classification result. Furthermore, as shown in Fig. 2, the abnormality map of Frontotemporal dementia appeared asymmetric between left and right hemisphere, why is that?
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The data used are all from open-access databases. The implementation of the algorithm is clearly described in the method section for an expert to reproduce the main results. However, it might be better for the authors to further clarify how they perform “oversampling technique” to balance the number of training data from each category.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Integrating multi-modality MRI data (e.g., diffusion tensor imaging, functional MRI) might be helpful in further improving the classification results.
- Probably the authors can use these open-access data to demonstrate that their method is useful for the detection and differentiation of dementia subtypes in the early stage, i.e., MCI.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is well written and the experiments are well designed to demonstrate the efficacy of the proposed classification method. However, there’s not too much innovation in the proposed method. Good results may be leaded by the cohort bias, How to exclude the cohort bias among datasets should be considered.
Number of papers in your stack

6
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Strengths: differential diagnosis is important and unsolved problem, good comparison/ablation study, relatively novel method, good description of clinical aspects, interesting combination of graph network and SVM, well written, experiments well designed

Weaknesses: novelty of component of method is limited, method description is missing important details on preprocessing pipeline, data set choices unclear (split and excluding mild cognitive impairment), small dataset
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

1

Author Feedback

We thank the reviewers for the very interesting comments on our article. Their suggestions will be taken into account for the updated version. R#1/R#2: We will add further details about the SVM training protocol. We used a grid search strategy on the validation set between three kernels (linear, polynomial, and gaussian) and different soft margin values. R#2/R#3: Adding early cognitive impairment subjects is a very interesting question. However, while AD-MCI are quite common, to our knowledge, FTD-MCI do not exist in open access databases. Consequently, performing early differential diagnosis between AD-MCI and FTD-MCI is not possible yet with the available databases. We are currently looking for private database including FTD-MCI for further works. R#1: The section 2.2 details all the used preprocessing steps with adequate references. We used the default hyper-parameters for the AssemblyNet model (as in its original paper) R#2: During training, the training set was split into (80% train + 20% validation). The MNI space used was the MNI152 space. The number of repetitions for Tables 2, 3 and 4 was 10. R#3: The abnormality map of Frontotemporal dementia appeared asymmetric between left and right hemisphere. This result is in line with our current knowledge on this disease. https://pubmed.ncbi.nlm.nih.gov/24133287/

back to top

Interpretable differential diagnosis for Alzheimer’s disease and Frontotemporal dementia