List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Minhui Yu, Hao Guan, Yuqi Fang, Ling Yue, Mingxia Liu
Abstract
Growing evidence shows that subjective cognitive decline (SCD) among elderly individuals is the possible pre-clinical stage of Alzheimer’s disease (AD). To prevent the potential disease conversion, it is critical to investigate biomarkers for SCD progression. Previous learning-based methods employ T1-weighted magnetic resonance imaging (MRI) data to aid the future progression prediction of SCD, but often fail to build reliable models due to the insufficient number of subjects and imbalanced sample classes. A few studies suggest building a model on a large-scale AD-related dataset and then applying it to another dataset for SCD progression via transfer learning. Unfortunately, they usually ignore significant data distribution gaps between different centers/domains. With the prior knowledge that SCD is at increased risk of underlying AD pathology, we propose a domain-prior-induced structural MRI adaptation (DSMA) method for SCD progression prediction by mitigating the distribution gap between SCD and AD groups. The proposed DSMA method consists of two parallel feature encoders for MRI feature learning in the labeled source domain and unlabeled target domain, an attention block to locate potential disease-associated brain regions, and a feature adaptation module based on maximum mean discrepancy (MMD) for cross-domain feature alignment. Experimental results on the public ADNI dataset and an SCD dataset demonstrate the superiority of our method over several state-of-the-arts.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_3
SharedIt: https://rdcu.be/cVD4K
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper introduces a domain adaptation method for MRI-based Subjective Cognitive Decline (SCD) classification. Specifically, they devised a two-path framework and jointly trained them with activation-averaged attention and maximum mean discrepancy loss. Taking a relatively large ADNI dataset as a source domain, the proposed network was trained to reduce distribution gap to a target AAA dataset. The experimental results showed improved performance with their method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This work tries to solve a vital small-sample problem in medicine by means of domain adaptation.
- The paper is well organized, making their contribution clear.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The technical contribution is moderate because the proposed mechanisms are existing and not nobel.
- The proposed method is not persuasive.
- The experiments should be more rigorous and thorough by comparing with the state-of-the-art methods.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
NA
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The authors argue that this work could be among the first attempts employing domain adaptation for SCD progression prediction. However, many existing domain adaptation methods can also apply to the same task. In this regard, it needs to thoroughly compare the existing domain adaptation methods in the literature.
- In Eq. (2), the denominator should be $m\times n\times k$.
- It is unclear why is the tensor $A$ interpreted as an attention map? It is just a rescaled activation map with the averaged value of the small volumes. Note that large activation values don’t necessarily contribute to the task-related important features.
- What is the rationale for using entropy loss $L_{E}$ over the bottom branch with unlabeled target data? According to the description, it doesn’t even participate in the back-propagation.
- In the test phase, the top brain model is used. This case assumes that after training, the model learned to map the samples of sSCD and pSCD to CN and AD, respectively. However, to this reviewer’s understanding, no mechanism makes such a relation in the proposed framework.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Lack of technical novelty
- Non-persuasive framework to solve the problem
- Lack of comparison with SOTA methods, especially, domain adaptation
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This paper modifies the transfer learning methods using feature adaptation, and propose a domain-prior-induced structural MRI adaptation (DSMA) method for automated SCD progression prediction. Experimental results on 795 subjects from the public ADNI dataset and a small-scale SCD dataset demonstrate the superiority of the proposed DSMA method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) Feature adaptation is important for transfer learning, which is the study of this paper. (2) The paper is well organized, and the idea is easy to understand. (3) The performance is improved by the proposed method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) The proposed DSMA model is similar to the Inductive transfer learning (ITL) model except for the feature adaptation module, so the innovation of this paper is limited. (2) The paper employ the MMD based feature adaptation module to alleviate the inter-domain discrepancy, but still ignores the intra-domain data distribution gaps that may be caused by different imaging scanners and scanning protocols. (3) The references are not adequate and up to date.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It is difficult to judge the reproducibility of the paper as the parameter settings are unclear.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
(1) The authors should give details of parameter settings of the proposed model and the competing models. (2) The author should clearly show how the experimental data is divided into training, validation and test sets. (3) Comparison with more transfer learning methods should be given.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The motivation of this paper is clear, the innovation is limited, and the experiments are complete
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
In this work, the authors propose a method to predict the progression of Subjective Cognitive Decline (SCD) by accounting for the distribution gap between Alzheimer’s Disease and SCD datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Very well organized
- Technically thorough
- Great experiment design - appreciate the authors investigating different source dataset; performing ablation studies and hyperparameter analysis and benchmarking with other methods
- Strong evaluation of results
- Neat figures
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Does not seem reproducible only because the ‘AAA’ dataset (as referenced in this work) is not publicly available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Recommend the authors to compare attention maps obtained from the implemented attention module with other explainable AI method such as Grad-CAM - would be very interesting to see those results!
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
8
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Excellent submission - well-written; technically sound; thoroughly evaluated; high-quality figures; extensive investigations such as ablation studies, hyperparameter anlaysis
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Since the options of the reviewers are divided, I suggest the authors itemized the responses to Review#1.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
12
Author Feedback
We appreciate the Reviewers for the constructive comments. We are encouraged by the many positive comments about ‘technically thorough’ (R3), ‘improved performance’ (R2), ‘very well organized’ (R1&R2&R3), ‘great experiment design’ (R3), and ‘strong evaluation of results’(R3). We will address major concerns as follows.
R1&R2: Comparison with existing domain adaptation (DA) methods Besides the original competing methods, we further compare our method with two SOTA DA methods:
- Deep CORAL [1]: AUC=64.21 ± 2.05%, ACC=64.21 ± 2.05%, BAC=63.41 ± 1.6%, SEN=65.28 ± 6.37%, SPE=61.54 ± 5.09%, and F1=52.45 ± 2.1%.
- DANN [2]. The model achieves an AUC of 50% on our task. We think our task may be too difficult for this model to perform well. We will add these results in the final version. [1] DOI: 10.48550/arXiv.1505.07818 [2] DOI: 10.48550/arXiv.1607.01719
R1: Eq. 2 denominator Thank you for pointing that out. It will be fixed in the final version.
R1: Tensor A In Eq. 3, A is the product of the original activation map and the rescaled activation map. For the former, each of the neurons can be viewed as a local feature. For the latter, each of the rescaled values can be viewed as a global feature for each of the blocks. This operation enhances the important areas while weakening the unrelated areas. This is inspired by [3]. To get the attention map, we average all the activation maps in the layer, which represents the information the layer has learned, thus we can expect that it shows potential SCD-associated regions. [3] DOI: 10.48550/arXiv.1808.06670
R1: Entropy loss Although the entropy loss doesn’t participate in the backpropagation, we count it in the overall loss. This would work because of the characteristic of our data: 1) the target domain data is unlabeled; and 2) unlike other domain adaptation problem settings, we have not only different data sources but also different (but related) labels, making the problem more complicated. Both classes in target domain are closer to NC at the beginning, so they tend to be all classified as 0 and have large entropy loss. The decrease of entropy loss can to some extent show us the improvement in target domain classification, so we know the model is working. We will include the related presentation in the final version.
R1: Sample mapping According to the characteristic of those four classes of data, CN means no disease, sSCD means the subject is not a potential AD subject, which is close to no disease, pSCD is progressing to AD, which may have the mild biomarker of AD, and AD has the strong biomarker. Thus, sSCD and pSCD are in between NC and AD. If domain adaptation is trained well, by adjusting the threshold we can expect the sSCD to be classified on the side of NC, and pSCD to be classified on the side of AD.
R1: Moderate technical contribution Our task is interdisciplinary, aiming to use machine learning methods to solve vital biomedical problems. From the biomedical perspective, the proposed method is a novel application in solving SCD prediction. From the machine learning perspective, the method is applied to a multi-source multi-label task, and we propose to use 3 losses, which is also a novelty.
R2: DSMA similar to ITL The ITL model is trained on source domain and tested on task domain. In DSMA, it is trained to make the distribution of source and target data closer. Those two methods are different. We will clarify this in the final version.
R2: Intra-domain data distribution gaps This is a good point. Thank you for pointing that out. We will include this analysis in future work.
R2: Parameter Settings The parameter setting is mentioned in Section 2.2–Implementation, and we keep the same setup for competing methods.
R2: Partition of data We used a data partitioning strategy commonly used in DA problems, i.e., 90% of data are used for training and 10% for validation; all target data for test.
R3: Grad-CAM Great suggestion. We will add the comparison in the future.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I agree with the reivewer that the technical contribution of this work is very moderate. In the rebuttal, the authors did not thoroughly clarify this (the major concern of the reviewer).
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
While the reviewers criticised the novelty of the paper, the report in a clinical setting is important. The authors responded well in the rebuttal and showed additional results, which should be added to the revised manuscript, which will further strengthen the paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The key strength of this paper is to introduce a domain adaptation method for MRI-based Subjective Cognitive Decline (SCD) classification in order to solve a vital small-sample problem in medicine. The key weakness is limited technical novelty and contribution. The authors have tried to address the major concerns and provide more experimental results. Therefore, I would suggest acceptance of this paper and revise the final version of this paper to integrate all useful comments.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8