List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Qiangguo Jin, Hui Cui, Changming Sun, Jiangbin Zheng, Leyi Wei, Zhenyu Fang, Zhaopeng Meng, Ran Su
Abstract
Acquiring pixel-level annotations for histological image segmentation is time- and labor-consuming. Semi-supervised learning enables learning from the unlabeled and limited amount of labeled data. A challenging issue is the inconsistent and uncertain predictions on unlabeled data. To enforce invariant predictions over the perturbations applied to the hidden feature space, we propose a Mean-Teacher based hierarchical consistency enforcement (HCE) framework and a novel hierarchical consistency loss (HC-loss) with learnable and self-guided mechanisms. Specifically, the HCE takes the perturbed versions of the hierarchical features from the encoder as input to the auxiliary decoders, and encourages the predictions of the auxiliary decoders and the main decoder to be consistent. The HC-loss facilitates the teacher model to generate reliable guidance and enhances the consistency among all the decoders of the student model. The proposed method is simple, yet effective, which can easily be extended to other frameworks. The quantitative and qualitative experimental results indicate the effectiveness of the hierarchical consistency enforcement on the MoNuSeg and CRAG datasets.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_1
SharedIt: https://rdcu.be/cVRqY
Link to the code repository
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a novel semi-supervised segmentation method applied to cells and glands in regions of histopathology images. The method follows a teacher-student model where the teacher is regularised with stochastically altered intermediate features.The results on two datasets of nuclei and glands segmentation show competitive, and in some cases better, results than state of the art SSL methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Writing quality is good.
- The experimental setup is solid and design decisions of each stage are well-motivated with the aim of improving the standard semi-supervised mean teacher.
- The method technicalities are well explained.
- The method is evaluated on two state-of-the-art datasets for histopathological structures segmentation.
- The comparison with strong SSL baselines makes the paper relevant and the results promising.
- Is remarkable that with such a limited amount of data (1 labeled image region), the proposed method reaches a dice >0.7 for nuclei segmentation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The paper novelty is to support the idea that applying perturbations in the feature space is preferred than doing them in the input or in the output space. Nevertheless, the authors do not compare or explain properly how it improves against such SSL methods, which weakens the paper.
- Details and discussion on pixel-segmentation results are lacking.
- No statistical analysis of the results, or average performance over several repetitions are reported, which makes difficult to evaluate the robustness of the methods.
- The qualitative results show that the method struggles to separate the boundary of touching structures, would have been interesting to discuss deeper and propose further enhancements for the method to cope with this. State-of-the-art methods such as [1] make this the central innovation in their methods, I think HCE could be further improved if you take this type of domain knowledge into account.
- The qualitative results show that the method struggles to separate the boundary of touching structures, would have been interesting to discuss deeper and propose further enhancements for the method to cope with this. [1]: He, Hongliang, et al. “CDNet: Centripetal Direction Network for Nuclear Instance Segmentation.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Methods: Highly reproducible as all the technical details are well-explained. Datasets: Both datasets are open-access which
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
The datasets used in the experiments are relatively small regions of whole slide images. Provide the computation time (inference and training) and the average computing time for the instances (glands, cells). Providing the computation times can help evaluate the feasibility to apply this method directly on whole slide images in the routine work of pathology laboratories and researchers in computational pathology.
-
Please elaborate why the learnable HCL of equation (4) is “capable of providing a more reliable prediction as guidance for the student model”. Did you test this experimentally? Such statements should be accompanied with evidence to some degree at least. Is this the first row of the ablation results table? If so, please refer to it properly in the sentence.
-
The writing can be improved in some parts: In section 2, the last paragraph that introduces the notation, should be the first, I think.
-
Figure 1. Is quite cluttered and difficult to understand at a first glance, making it simpler probably would draw the attention of more readers.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- Few incremental novelties that fit together well.
- Strong baselines.
- State-of-the-art open access datasets.
- Relevant problem for computational pathology.
- Well written manuscript.
- The only drawback of the paper is that do not discuss the boundary separation problem, which is important in this segmentation task.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper describes a framework to utilize both labeled and unlabeled data for nuclear and gland segmentation in a hierarchical fashion. The method uses a teacher-student network setup where labeled data is fed to a student network for supervised optimization, while training with unlabeled data is based on consistency between multiple outputs from student network and a single output from the teacher network. Specifically, latent space perturbations are applied at multiple layers of the student decoder with prediction certainty maximized with self-supervised mechanisms and hierarchical losses using the Teacher model outputs. Results on existing public benchmark datasets show competitive results, including ablations on the proposed losses.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
• This work introduces several novel losses tailored to the hierarchical setting i.e., hierarchical consistency loss and self-supervised enforcement. • The model is able to increase segmentation performance when with a limited amount of data e.g., 50% labeled data reports results comparable to fully-supervised baselines. • The human based labeling process of histopathology images is an intensive task that also requires special expertise making it costly and difficult to scale. Therefore, the development of methods that leverage unlabeled data are important for larger scale medical studies and applications. • The framework presented does not depend on a special network architecture making flexible and perhaps easy to adapt in other models.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
• The paper is based on the idea that decoders at different layers of the network can reconstruct the exact same output as the last decoder. However, there is no clear proof that this consistency enforcement does not result on identity functions for the last layers of the encoder leading to a network that under-fits the already small training data. • The current reported results employ 3 hierarchical layers in the student model. It is unclear if the improvement stems solely from the use of multiple layers. Including ablations/evaluations for the number of layers may further support the need for hierarchical learning.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The presented ideas are clear and can be reproduced.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Part of the optimization process is based on the measurement of uncertainty from the teacher’s network output, but the explanation is limited and there are no additional experiments that validate its importance. It will be advisable to give more information to clarify and validate the design choice. In addition, I am concerned that the model is sensitive to the number of layers employed for HCE and learnable HC losses.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a simple framework to exploit unlabeled data alongside limited labeled with HCE modules and novel learnable HC losses. I appreciate the extensive experiments and ablations with recent works, to both validate the proposed losses and their contributions in learning. According to results the model has the most improvement when the amount of data is very small, but it is not proved that the model has performance gains due to the proposed HCE module or it is also due to under-fitting.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
In this paper, the authors propose a semi-supervised histological image segmentation method based on the Mean-Teacher model with a hierarchical feature consistency loss. In experiments, the proposed method outperforms previous methods, including a SOTA method using transformation consistency.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) The problem is important and should be interesting to the MICCAI community. 2) The idea and method presented in the paper are intuitive and easy to follow, and the organization is clear. 3) Experiments are conducted on two datasets, and comparisons include several previous semi-supervised methods. Compared with semi-supervised methods, the proposed HCE generally shows better performance; It is encouraging to see that HCE achieves comparable or better performance even when compared with the full-supervised method, indicating the great potential of semi-supervised segmentation methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) Even though I agree with the authors that the proposed HCE is simple, the novelty is limited. It’s more of a marginal improvement over the current perturbation-based Mean-Teacher method such as [8]. More importantly, I think it is necessary to introduce more details about the perturbations, including both types and degrees, as I believe the essence of the method is to learn perturbation-invariant features for a more efficient learning process. It would be interesting to see more discussions on how would different feature-level perturbations affect the HCE performances. Could authors also comment on why transformations as in [6] or prediction-level perturbations as in [8] are not used in the paper? If possible, an ablation study on perturbations would be constructive. 2) Authros could consider including more recent related works, in the discussion and/or comparisons, such as Extreme Consistency: Overcoming Annotation Scarcity and Domain Shifts by Fotedar et al. and SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation by You et al. 3) In Fig. 2, it would be helpful to better locate the visual differences if authors use the same or similar colormaps for different methods.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Authors claimed that the code will be released, and results shown in the paper seem reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please refer to the weaknesses above.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the novelty is limited, the results are encouraging. I may raise my rating if authors could resolve my concerns regarding the perturbations.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a novel histological image segmentation method based on a student-teacher model which uses latent space perturbations for self-supervised learning and hierarchical consistency loss. All three reviewers agreed that the paper is well-written and the result is promising for real cases where data is not sufficient. Although there are several weak points, a number of strong points ouweigh the weakness. Therefore, I recommend acceptance. Below is the summary of the reviews:
Strength:
- Well-written, methods are well explained (R1, R3)
- Technically novel (new loss functions) (R2)
- Clinically useful for larger scale studies and application (R2)
- Flexible, easy to adapt to in other models (R2)
- The experimental setup is solid and design decisions of each stage are well-motivated (R1)
- Validation and experiments are well conducted (R1)
- Demonstrated good performance with limited data (R1, R2, R3)
Weakness:
- No explanation or rationale behind the performance improvement (R1, R2)
- Details and discussion on pixel-segmentation results are lacking (R1)
- No statistical analysis of the results (R1)
- Weak technical novelty (R3)
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Author Feedback
N/A