Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Jinpeng Li, Guangyong Chen, Hangyu Mao, Danruo Deng, Dong Li, Jianye Hao, Qi Dou, Pheng-Ann Heng

Abstract

Medical data often follow imbalanced distributions, which poses a long-standing challenge for computer-aided diagnosis systems built upon medical image classification. Most existing efforts are conducted by applying re-balancing methods for the collected training samples, which improves the predictive performance for the minority class but at the cost of decreasing the performance for the majority. To address this paradox, we adopt a flat-aware cross-stage distilled framework (FCD), where we first search for flat local minima of the base training objective function on the original imbalanced dataset, and then continuously finetune this classifier within the flat region on the re-balanced one. To further prevent the performance decreasing for the majority, we propose a cross-stage distillation regularizing term to promote the optimized features to remain in the common optimal subspace. Extensive experiments on two imbalanced medical image datasets demonstrate the effectiveness of our proposed framework and its generality in improving the performance of existing imbalanced methods. The code of this work will be released publicly.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16437-8_21

SharedIt: https://rdcu.be/cVRs7

Link to the code repository

N/A

Link to the dataset(s)

www.kaggle.com/c/diabetic-retinopathy-detection

endotect.com

Reviews

Review #1

  • Please describe the contribution of the paper

    In this paper, the authors propose a two-stage framework for imbalanced medical image classification. The proposed method sequentially learns the representative and discriminating features from the imbalanced data distribution and re-balancing strategies, respectively. In details, the method flattens the local minima to form a common optimal region and further employs the cross-stage distillation to facilitate the network optimization within this region.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Data imbalance is an important problem in medical image analysis. The proposed method is reasonable and novel. The paper is well written. The proposed method is evaluated on two public imbalanced datasets. The comparison experiments are comprehensive.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    The illustration of Fig. 1 could be improved.

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    None

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    See above

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    interesting paper where merits slightly weigh over weakness

  • Number of papers in your stack

    5

  • What is the ranking of this paper in your review stack?

    3

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #2

  • Please describe the contribution of the paper

    In this paper, the authors address imbalanced medical image classification with a flat-aware cross-stage distilled framework (FCD). FCD combines two-stage learning, flatten local minima and cross-stage distillation to re-balance classes while avoiding knowledge forgetting. FCD achieved good performance in two large imbalanced medical image calssification datasets.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The proposed methods are well motivated and constrcuted
    2. Good performance on two large datasets with well-designed experiments / ablations.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    There is still no guarantee that local optimum of the second stage can be found in the flatten local minimum of the first stage.

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Good. Experiments were performed on public datasets and the authors plan to release code after peer-review according to the checklist.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    Major: Flattening local minima is a clever idea yet there is still no guarantee that local optimum of the second stage can be found in the flatten local minimum of the first stage. It will definitely help to have at least one of the following (1) more theoretical analysis to prove this point (2) Visualization of the loss landscape in different stages with the proposed methods (3) experiments on more datasets with more variety of data distributions. It is also not clear how rho (radius of neibouring area) is selected and what’s the effect of rho.

    Minor: There are some typos in the last few pages of the manuscript. For example (1) Page 7, row 3, Ours(RS) and Ours(RS) should be Ours(RW); (2) Page 8, Table 3 should be Fig. 3.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Manuscript is well written and ideas and methodology well structured and presented. Experiments are solid and the proposed framework outperforms state-of-the-art methods.

  • Number of papers in your stack

    5

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Somewhat Confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    This paper proposed to adopt a flat-aware cross-stage distilled framework for imbalanced medical images classification. Extensive experiments have been done on two widely-used pubic datasets. The results demonstrate the effectiveness of the proposed methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. Good clarity and formulations.
    2. The proposed FLM technique is interesting and obtains significant improvments through the ablation study results.
    3. The proposed methods consist several main components, which can be well plugged into many other state-of-the-art methods with further improvments on the performance.
    4. Extensive experiments on two public datasets. The comparison study covers most of popular methods in long-tailed learning and imbalanced classification benchmarks.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. The technique novelty is limited, and most components exist in the literature and some have been widely used in many existing works.

    2. This paper claimed that most of existing methods improve the predictive performance for the minority class but at the cost of decreasing the performance for the majority. However, there is no related metrics or discussion (only in Figure 1) for this. For example, for the long-tailed dataset, you can use three groups “many”, “medium”, “few” to give a complete illustration on how your proposed methods can well improve the performance both on majority classes and minority classes, especially for a long-tailed distribution with more than 20 classes. Lacking related results may make the paper seem to be over-claimed.

    3. For the fundus dataset, DR grading is a common challenge, but I also expected to see how different methods affect the performance. Moreover, to the best of my knowledge, the data imbalance issue does not do damage to the performance as we imagined, i.e., the performance of minority classes may exceed that of major classes. For instance, NPDRIII always shows good performance even if there is a few samples available since it show obvious pathological features. I believe DR grading is not a good task. I recommend the following references for some related information. [1] Relational Subsets Knowledge Distillation for Long-tailed Retinal Diseases Recognition. [2] Automatic detection of rare pathologies in fundus photographs using few-shot learning.

    4. This paper studied and tested their methods on imbalanced dataset and its specicial condition - long-tailed. However, I suggest to evaluate it on more long-tailed benchmarks due to the reasons I mentioned above.

    Some other comments: Compared with RS, the two-stage methods (which contains RS component) such as cRT and DiVE did not obtain good results. To the best of my knowledge, two-stage methods show good robustness in many long-tailed benchmarks. Can you explain that?

  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The authors promised to release their codes and all the experiments are conducted on public datasets. I believe the results could be reproduced.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    See 5.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Due to the reasons above, I recommend a weak accept for this paper.

  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposed a two-stage method to address the imbalanced medical image classification, which is a significant and interesting topic. Given three consistent positive reviews, I recommend accepting this submission. However, there are some concerns arised in the reviews, especially in R3. The authors should address the detailed comments from the reviewers in the camera-ready manuscript.

  • What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    2



Author Feedback

We thank the AC and reviewers for their very valuable comments and positive options on our work. We will address the comments and fix the typos in the final version of our manuscript.



back to top