List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Canran Li, Dongnan Liu, Haoran Li, Zheng Zhang, Guangming Lu, Xiaojun Chang, Weidong Cai
Abstract
Unsupervised domain adaptation (UDA) methods have been broadly utilized to improve the models’ adaptation ability in general computer vision. However, different from the natural images, there exist huge semantic gaps for the nuclei from different categories in histopathology images. It is still under-explored how could we build generalized UDA models for precise segmentation or classification of nuclei instances across different datasets. In this work, we propose a novel deep neural network, namely Category-Aware feature alignment and Pseudo-Labelling Network (CAPL-Net) for UDA nuclei instance segmentation and classification. Specifically, we first propose a category-level feature alignment module with dynamic learnable trade-off weights. Second, we propose to facilitate the model performance on the target data via self-supervised training with pseudo labels based on nuclei-level prototype features. Comprehensive experiments on cross-domain nuclei instance segmentation and classification tasks demonstrate that our approach outperforms state-of-the-art UDA methods with a remarkable margin.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_68
SharedIt: https://rdcu.be/cVRXG
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
In this paper, the authors propose a class-aware feature alignment method in domain adaptation for nuclei segmentation and classification. They also propose to select a certain branch to generate pesudo labels to reduce the negative effects of incorrect prediction in pseudo labels. Experiments show the effectiveness of the method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- It is a simple but effective way to use the class-aware feature alignment for domain adaptation to handle different types of nuclei, because the appearance and distributions among different nuclei are large.
- The ablation studies clearly present the effectivenss of the proposed methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1). The details of the method part are not clear enough. The losses are not explained either in the main paper or the supplementary materials. 2). It could be better to provide other methods’ results in the Dpath -> GlaS task as those in the Dpath -> CRAG task
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors do not provide the description of results with central tendency (e.g. mean) & variation in the paper, which they answered Yes in the checklist. It would be a plus to add mean and variance to the results.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
(1) There are details missing in the method section, which affects the overall quality of the paper because the readers have no idea how the method works. It is better to make sure that all necessary infomation is provided, either by clear description in the paper or adding references. (2) About the pseudo labels, I am wondering if the authors generate pseudo labels for all images/patches no matter what the probablity scores are. It may be better to only use the pseudo labels that have high probability scores (i.e., close to 1 or 0), which are more trustable. (3) It is interesting to see how the method works in the reverse direction, e.g. CRAG -> Dpath.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is an overall good paper but the method part is not clear enough. Thus I tend to give weak accept.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
- Proposes a new unsupervised domain adaptation approach for nuclei instance segmentation and classification.
- The approach uses class-level feature alignment with class-specific adversarial discriminators and self-supervised learning from pseudo-labels predicted by the model.
- The proposed approach outperforms the baseline methods on segmentation and classification.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The approach is well-motivated. Although class-aware domain adaptation has been done in the machine learning communities, it is claimed to be the first time for nuclei instance segmentation and classification.
- Empirical results and ablation studies show that the proposed approach outperforms other baselines.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Some details of the proposed methodology is missing. For the class-aware discriminator, it is unclear how the classes are assigned for the target domain due to the lack of labeled data. How to handle prediction errors that could interfere with the discriminator.
- The paper doesn’t adequately discuss the limitations of the proposed approach. Extensive use of pseudo-labels (in both the class-aware discriminator and in self-supervised learning) could result in model deterioration and poor calibration due to enforcing the model to optimize towards what it has already know. What are the motivations and practical considerations for using pseudo-labels? Is there any issue encountered/resolved due to mistakes in the pseudo-labels?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Not sure how easy/difficult to reproduce this paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
In the experiments, why Dpath is always used as the source domain in both the main paper and the supplementary materials? It would be interesting to know the adaptation performance from the other direction, i.g., CRAG -> Dpath.
Not clear what is the difference between Baseline+CA+PL and the proposal.
It is typically not trivial to learn the loss weighting parameter in Eq. 2. Would it be possible to share what is the weighting parameter after training and if any special optimization technique is used.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents an interesting approach using feature alignment and self-training with competitive results over the baselines. The main limitation of this paper is that it doesn’t adequately discuss the limitations of the proposed approach, e.g., it is well-known that the type of self-supervised learning in this paper could result in model deterioration and poor calibration.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
Revise the class-awre feature alignment compared with Ref.12
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper proposed a category-aware prototype pseudo-labelling architecture for unsupervised domain adaptive nuclear instance segmentation and classification
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- What’s the main difference between the proposed method and Ref.12, so what’s the highlight of your paper.
- There are a lot of papers about Zero-shot or one-shot image segmentation or classification, why do not you directly use these techniques and select the UDA, which more complexity than others?
- How about the failures?
- There are some grammatical and sentence expression errors in this manuscript, pls revise them carefully.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I think it can be reproduced
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- What’s the main difference between the proposed method and Ref.12, so what’s the highlight of your paper.
- There are a lot of papers about Zero-shot or one-shot image segmentation or classification, why do not you directly use these techniques and select the UDA, which more complexity than others?
- How about the failures?
- There are some grammatical and sentence expression errors in this manuscript, pls revise them carefully.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
reproduced
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper received three weak accepts. The reviewers gave positive comments on the idea of using class-aware feature alignment for unsupervised domain adaptation as well as the experiments to justify the proposed method. However, the reviewers also raised some questions or concerns, e.g., the presentation of the method is not clear (Reviewers #2 and #3), the motivation of using pseudo-labels and how to deal with noisy pseudo-labels are not well explained, the reason why choosing Dpath as the source domain is not given, and why not consider other relevant methods that may have lower complexity is not clearly described (Reviewer #4). Please improve the manuscript by considering these comments when preparing the final version.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
We sincerely appreciate the reviewers and area chair for the positive and valuable comments, and we have updated our manuscript accordingly.
We also would like to take this opportunity to clarify the concerns raised by the reviewers:
Q1. The highlight of our work. Compared with Ref.12, our work focuses on the alignment of class-level feature distributions and achieves good results when the target domain data has no labels. The performance improvement of Ref. 12 relies on weak labels and fails to get good training results without target domain labels (R4, Q5.1). Unsupervised domain adaptation and zero-shot/one-shot approaches address different problems. UDA method is proposed to minimize the gap between different data distributions, which belongs to transfer learning. Zero-shot/one-shot mainly considers the problem of insufficient training samples, and they always don’t consider the distribution discrepancy between different datasets. Therefore, we think the complexity of the UDA and low-shot learning tasks are not comparable. We will consider extending our method for low-shot learning in future work (R4, Q5.2).
Q2. Details of losses (R2, Q5.1) Following Hover-net, we use the same loss functions for each branch. For the adversarial domain discriminator, we set 1 (0) as the Ground-Truth of the source (target) domain and apply binary cross-entropy as the loss function. Due to the page limit, these loss functions are not described in detail in the main paper. We will add the details of the loss function in the final version.
Q3. Supplement on the experimental results. 1) Regarding the results of the Dpath -> GlaS task (R2, Q5.2), the nuclei classification results of Yang et al. are: 0.565, 0.000, 0.621, 0.141, 0.152, 0.018, 0.240. Their nuclei segmentation results are: 0.639 0.294 0.377 0.735 0.275. Following the main paper, we didn’t do classification tasks on PDAM. The nuclei segmentation results of PDAM are: 0.596, 0.351, 0.467, 0.676, 0.316. 2) The description of Baseline+CA+PL can be found in Section 3.4 (R3, Q8). The difference between it and the proposed method is that our proposed method only trains the target domain of the HV branch during the pseudo-labelling process. By contrast, Baseline+CA+PL directly trains the model on the target images with all predictions in the first stage as the pseudo labels. 3) We believe it is important to describe the results with the central tendency (R2, Q7) and the loss weighting parameter (R3, Q8.3). It will be included in our main paper or supplementary material. For more basic training details, please refer to Hover-net.
Q4. The supplementary description on pseudo-labels. Our motivation for using the pseudo-label is to improve the model’s performance through supervised training. Pseudo-learning is one of the self-supervised methods. An essential advantage of self-supervised methods is that, although the obtained labels have noises, the model performance is improved by providing more supervision labels to the training data. However, the low quality of some pseudo-labelling classes may cause the model trained with all pseudo-labels not to perform well. In order to overcome this obstacle, we propose the nuclei-level prototype in Section 2.3, in which we only train the target domain through the HV branch during the second stage (pseudo-labelling process) (R3, Q5.2). We chose threshold=0.9 as our pseudo-label probability score (R2, Q8.2).
Q5. The limitations of our proposed method. We chose Dpath as the source domain since the Dpath dataset has the largest number of nuclei in the Lizard dataset (R3, Q8.1). Experimentally, our method shows good performance when the source domain contains more data than the target domain, but does not work well when the source domain has less data than the target domain (e.g., transferring from CRAG to Dpath). It is a limitation of our work, and we will further explore it in our future work (R3, Q5.2; R3, Q10; R4, Q5.3).