List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yi Zhou, Shaochen Bai, Tao Zhou, Yu Zhang, Huazhu Fu
Abstract
Unsupervised domain adaptation (UDA) has received significant attention in medical image analysis when labels are only available for the source domain data but not for the target domain. Previous UDA methods mainly focused on the closed-set scenario, assuming that only the domain distribution shifts across domains while the label space is the same. However, in the practice of medical imaging, the disease categories of training data in source domain are usually limited, and the open-world target domain data may have many unknown} classes private to the source domain. Thus, open-set domain adaptation (OSDA) has great potential in this area. In this paper, we explore the OSDA problem by delving into local features for fundus disease recognition. We propose a collaborative regional clustering and alignment method to identify the common local feature patterns which are category-agnostic. Then, a cluster-aware contrastive adaptation loss is introduced to adapt the distributions based on the common local features. We also construct the first fundus image benchmark for OSDA to evaluate our methods and carry out extensive experiments for comparison. It shows that our model achieves consistent improvements over the state-of-the-art methods.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_65
SharedIt: https://rdcu.be/cVRXD
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper describes a method for unsupervised domain adaptation with unknown/new classes in target domain, i.e. OSDA problem. The paper proposes a collaborative regional clustering approach to identify feature clusters, then a contrastive loss for better cluster boundary shaping.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- overall the paper is clearly written and well organized.
- it has a clear clinical application
- it has a good comparison study with existing arts.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- computationally expensive, could be infeasible for large data
- model details need clarification
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of the paper should be good since the authors will provide the code and the datasets are public. Important hyperparameters are specified in the paper as well.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Overall I think this paper is well presented. The model description is clear. Here are my comments.
-
Though the paper proposes IRS to reduce the computational cost, it seems to me it is still very expensive to train the model, considering a K-means clustering is extensively run to update the prototypes, which makes it even not feasible for large scale data. Please comment. In addition, I would recommend the paper to show computation time for model training.
-
It may be a minor issue or I misunderstand, in the experiment K^s=5 since there are 5 diseases in the source domain, \beta=1.5, so K^t=7.5, a fractional cluster number?
-
For the update of prototype, the paper proposes to use momentum controlled by \yita. A suggestion would be adding linear scheduling for \yita to make it variable instead of a fixed constant, this may improve convergence speed. Such approach is frequently used in existing models, such as mean-teacher exponential moving average.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- good presentation on the methodology
- clear evidence in numerical experiment on the advantage compared with existing art.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This paper proposes a collaborative regional clustering and alignment method to explore the open-set domain adaptation (OSDA) issues in the domain of medical images. The experiments show that the proposed model can achieve consistent improvements over the state-of-the-art methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.This paper investigates the OSDA problem in the medical imaging scenario, speci cally for fundus disease recognition.
- A novel method is proposed to perform positive transfer more precisely while avoiding negative transfer across domains, through delving into local features.
- The proposed method achieves state-of-the-art performance compared to previous methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1.Some technique details are not clear: a.The paper does not mention whether the training process of the CNN model is end-to-end or not since it contains multi-branch. b. In the Informative Region Selection part, it depends on the CAM to select patch features, but not mention the size of the patch. c. The experiments does not mention the input size. People may want to know it because fundus images are probably in higher resolution than natural images. 2.Datasets are not enough: I notice that the path cluster contains hard exudates, which are sign of diabetic retinopathy(DR). Try some DR datasets such as IDRID may make the paper more promising.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The CNN model is ResNet50 and the datasets are all public, thus the reproducibility of the paper is strong with more detail of experiments’ setting, for example, the input size.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Since you have mentioned that “they usually have similar global structures with uniformed appearances, such as the optic disks, vessels, and even common lesion patterns appeared in different diseases”, it may be a good idea to make use of the spatial information for OSDA methods on medical images.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The paper states that it is the first time to investigate the OSDA problem in fundus images.
- The performance of the proposed methods is considerable.
- The expression of the manuscript is good.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
This manuscript proposes an open-set domain adaptation (OSDA) method for fundus image classification. The proposed method employs contrastive learning based on clusteredl assigned by k-means to align source and target features, as well as to identify private classes. This paper can be interpreted as a sensible multi-scale version of to Domain Consensus Clustering (DCC, CVPR’21).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
This manuscript investigates a somehow overlooked problem in UDA for medical images: open-set domain adaptation.
-
The design of informative region selection makes sense considering that lesions in these datasets are often localized. It is shown to dramatically boost the performance of the baseline DCC.
-
The proposed method is tested on two pairs of datasets, and is shown to outperform baseline OSDA methods.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The proposed method seem to be relying on relatively strong assumptions: a) lesions (discriminative information) are localized; b) the source model performs reasonably well on the target data (small domain gap) so that initial clusters of the same class between the source and target can stay correctly close; c) there is little class-imbalance issue (otherwise majority classes may overshadow minority ones in the clustering); d) the number of private classes ($\beta$) is kind of known beforehand. The authors are encouraged to comment on whether these assumptions are too strong to be applied in practice?
-
Given that there is a k-means clustering in the method, what is the computational overhead of that, in comparisons with other steps during the training process?
-
There is little information regarding the source model: a) does it use a linear classifier in the end? If so, why are clusters assigned based on l2 distance, while the detection of private classifier are based on thresholding the softmax output of a linear classifier? b) will the source model to be updated during adaptation as well? If so, would it hurt the performance on source-domain data?
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
There is no significant issue with reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The authors are encouraged to comment on the relatively strong assumptions of the proposed method: to which extent can the proposed method to be deployed in practice?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed work is built on too-strong assumptions on the data and domains involved. Therefore, it is questionable whether the methodology is applicable in practice.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Most of my concerns are properly addressed. Still, the reviewer finds the assumptions to be rather strong. It is probably fine to accept this paper if taken as a specific solution to a specific clinical problem.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All the reviewers agreed that this work investigated an important but not well-studied problem in medical image analysis and recognized the technical contribution. The proposed methods were validated on two datasets and outperformed the alternative. However, reviewers raised several concerns about the algorithm design and missing details. The authors may want to address the comments during the rebuttal and improve the paper quality accordingly.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
To R1: Thanks for your support to our work. Q1: Computational Cost. A1: First, clustering is periodically (300 epochs) performed rather than running in every epoch (which is unnecessary). Second, 1D adaptive max-pooling is applied to reduce feature dimension before clustering. Thus, the (total clustering time / overall training time) only takes approx. 10%, which is scalable for large scale data. We will add the computation cost details in the revised paper.
Q2: K^s and K^t. A2: Clustering is performed in region-level rather than category-level. The K^s converges to 20, while K^t is 30.
Q3: Adding linear schedule. A3: Thanks for your insightful suggestion. We will take the advice to explore further improvements.
To R2: Thanks for your support. Your suggestion of spatial information usage will be explored. Q1: Technique Details. A1: a) The model training is end-to-end, and optimized by the final objective in Eq. 5. The only thing is that we pre-trained the backbone by 30 epochs to enable the IRS with good initial region selection ability. We will make this better clarified. b) The input size is 224x224. The spatial sizes of the last two conv layer feature maps are 7x7 and 14x14. The local patch sizes corresponding to the input image are 32x32 and 16x16, respectively. c) We adopt the common input size for fair comparison with baselines. The image is resized to 256x256, followed by a cropping to 224x224.
Q2: Datasets Selection. A2: We build two source-target domain pairs using three multi-disease datasets all containing DR. IDRiD only contains DR, which is not feasible to study the OSDA problem better requiring multi-disease scenarios. However, we will take your advice to extend our method to more medical tasks.
To R3: Thanks for your comments. Q1: ‘Assumptions’. A1: a) Fundus lesion localization has been studied by many works, such as CAM-based [1] and attention-based [2] methods, which are simple and useful. Our lesion localization doesn’t require any positional annotation, which is cheap but effective applied in practice. b) The source-only model doesn’t perform well on the target data (see Table 1). The cluster matching performance is progressively improved along with the domain adaptation learning, but not satisfied at the initial state. c) Both of the target domain datasets (ODIR and RFMiD) suffer the class-imbalance problem. However, the clustering is performed in region-level but not class-level, which can mitigate the issue. As we emphasized, one of our motivations is that many local regions with shared landmark patterns and lesion appearances (shared by majority and minority classes) can be fully exploited for domain adaptation. d) Since the clustering is done in region-level, the setting of beta will not be dramatically affected by the varying number of private classes. We validate it on different target datasets, and find a setting with enough capacity. We will add the ablation analysis in the revised paper.
Q2: Computational overhead of k-means. A2: Clustering is periodically (300 epochs) performed rather than running in every epoch (which is unnecessary). The (total clustering time / overall training time) takes approx. 10%. We will add the computation cost analysis in the revised paper.
Q3: Source Model Information. A3: a) Linear classifier is adopted in the end. However, the cluster-aware learning is based on local region-level features while the final classification is still on image-level. There is no direct link between the two training losses. b) The source model is updated during adaptation. A relatively large weight for L_cls (Eq. 5) makes the training signal from the source data stable and can correctly guide the target data.
References: [1] H. Jiang, et al., 2020, A Multi-Label Deep Learning Model with Interpretable Grad-CAM for Diabetic Retinopathy Classification, EMBC, IEEE. [2] Z. Wang, et al., 2017, Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection, MICCAI, Springer.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After carefully reading the authors’ rebuttal, I will vote for acceptance, given the major concerns raised by reviewers were well-addressed.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper studies open-set unsupervised domain adaptation for medical imaging, specifically for fundus image analysis. Although open-set UDA is not a new problem in general CV, but this is new in medical imaging field and provides a good basis for further investigation. The reviewers also acknowledge the technical contribution and experimental comparison of this paper. Thus, the AC votes for accepting this paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This work investigates open-set unsupervised domain adaptation in medical imaging, with a focus on fundus image processing. While open-set UDA is not a entirely new, it is novel in the field of medical imaging and provides a solid foundation for subsequent research. The reviewers also recognize this paper’s technical contribution and experimental comparison. The AC voted to accept this paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR