List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Qixiang Zhang, Yi Li, Cheng Xue, Xiaomeng Li
Abstract
Designing deep learning algorithms for gland segmentation is crucial for automatic cancer diagnosis and prognosis, yet the expensive annotation cost hinders the development and application of this technology. In this paper, we make a first attempt to explore a deep learning method for unsupervised gland segmentation, where no manual annotations are required. Existing unsupervised semantic segmentation methods encounter a huge challenge on gland images: They either over-segment a gland into many fractions or under-segment the gland regions by confusing many of them with the background. To overcome this challenge, our key insight is to introduce an empirical cue about gland morphology as extra knowledge to guide the segmentation process. To this end, we propose a novel Morphology-inspired method via Selective Semantic Grouping. We first leverage the empirical cue to selectively mine out proposals for gland sub-regions with variant appearances. Then, a Morphology-aware Semantic Grouping module is employed to summarize the overall information about the gland by explicitly grouping the semantics of its sub-region proposals. In this way, the final segmentation network could learn comprehensive knowledge about glands and produce well-delineated, complete predictions. We conduct experiments on GlaS dataset and CRAG dataset. Our method exceeds the second-best counterpart over 10.56% at mIOU.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43901-8_27
SharedIt: https://rdcu.be/dnwDb
Link to the code repository
https://github.com/xmed-lab/MSSG
Link to the dataset(s)
https://warwick.ac.uk/fac/cross_fac/tia/data/mildnet/
https://warwick.ac.uk/fac/cross_fac/tia/data/glascontest/
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a two-stage unsupervised method for gland segmentation. The first stage is based on pixel-wise similarity to generate gland segmentation pseudo labels. The second stage utilizes the pseudo label generated from the first stage as ground truth, and pushes pixels inside gland boarders together.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The high-level idea of the method sounds interesting.
- Extensive experiments show the effectiveness of the method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The writing is unclear and makes it difficult to comprehend the method being presented.
- Several details regarding the method are unclear. How to select proposals? It is very confusing how to convert Fig.2. candidate region Yi to proposal map Pi. Why there are five different clusters in Yi? Why only two clusters are used to generate pseudo labels, while it generates five clusters at the beginning?
- The mathematic functions are not clear, like Function (1) and (2).
- The method has specific assumptions on gland regions, such as that boarder part has high intensity. The strong constrain may reduce the generalizability of the method.
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
meet the requirement
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
see weakness
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a new method for unsupervised gland segmentation. But the presentation of the method is too confusing. The current version is not good enough to be accepted.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
The selection process of gland border proposals from the five candidates is indeed unclear even after the rebuttal process. The relationship between the proposal map Pi in Fig. 2 and the candidate region Yi in Fig. 2 is not apparent. The meaning and calculation of the loss functions, Eq (1) and Eq (2), are still ambiguous. It is also not clear how the one-hot cluster label Ci is obtained. The lack of essential method details hinders a proper understanding of the proposed approach for readers.
Review #2
- Please describe the contribution of the paper
In this paper, the authors propose the MSSG pipeline, which incorporates an empirical cue about gland morphology as additional knowledge to guide unsupervised gland segmentation. First, they propose a SPM module to generate a proposal map. Second, they propose a MSG module to merge the sub-regions of a gland.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The strategy that the SPM module adopts to generate the proposal map combines both the network features and the empirical cues into consideration. The strategy adopted in the MSG module further refines the pseudo labels and facilitates model optimization.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
According to Fig. 2, the initial labels Y are produced through clustering based on the features from an extra shallow encoder. But the differences between using different encoder architectures are not discussed or evaluated in this paper. According to Section 1, avoiding the proposed method from being extremely sensitive to color is one of the motivations of this paper. However, using such a shallow encoder to generate pseudo labels seems to be contrast to the motivation.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors claim to release the codes.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The author should discuss the reasons for using a shallow encoder to generate the initial pseudo labels.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method is meaningful, and according to the experimental results, it is effective. However, the authors should provide more discussion about the sensitiveness to color.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The motivation of this paper makes sense. But it would be better to provide more discussion on the depth of the encoder f, because it seems that a shallow decoder trained with the self-supervised method will make very noisy predictions.
Review #3
- Please describe the contribution of the paper
This work proposed a novel unsupervised gland segmentation model using extra color prior knowledge and morphology information as guidance. In image space, selective proposal mining (SPM) is proposed to generate pseudo segmentation labels, achieved by feature clustering, refined by color intensity differences between gland boundary and interior region. In embedding space, morphology-aware semantic grouping (MSG) is used to regulate features to reduce ambiguities. The model is validated on two datasets and achieves a large performance gain.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
It is a novel task-specific unsupervised segmentation solution. It integrates color intensities as prior knowledge which makes it different from other works. This work achieves a large performance gain which is about 10% measured by F1, Dice, and mIoU. The paper is clearly organized and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
For me, the motivation and the practical application of unsupervised model seems unclear. Why we need unsupervised model for gland segmentation in practical scenario?
In unsupervised setting, without validation set, how to finetune the hyper parameters in the model.
Effectiveness of some hyper-parameter setting is not validated. Such as the K in K-means is set to 5, and threshold beta is 0.7.
Effectiveness of Spatial Continuity Loss is just demonstrated by the visualization of an image. Maybe appending some numerical results would be better.
In unsupervised setting, performances could be sensitive to random seeds and initialization. Would you like to share more randomness evaluation regarding the model?
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Code will be available upon acceptance.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
It is a novel and effective method. Adding more explanations about motivations and practical applications would be better. More ablation studies on hyper-parameters may be required.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The motivation and application seem not fully explained but it is a novel and effective method. I suggest accept.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Most of my concerns are resolved but the clinical applications of unsupervised models still require further explorations. I would like to keep my rating.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
We have received mixed reviewer comments. While reviewers confirmed the merits of the paper on interesting idea and improved performance, they also raised major concerns including poor clarity, unclear motivation, lack of enough clarity on methodology and experimental analysis, etc. Therefore, a decision of Invite for Rebuttal is recommended for the authors to address the reviewers’ comments.
Author Feedback
We appreciate the reviewers and area chair for their valuable feedback. All 3 reviewers recognized the effectiveness of our method and significant performance improvements. They found our method interesting (R1), meaningful (R2), and novel&effective (R3). R3 considered our paper well-organized and easy to follow. However, R1 raised concerns regarding unclear writing in the method. R2 expressed concerns about rational of shallow encoder, while R3 has some concerns regarding practical application and experimental details. Q1: Clarify motivation and task significance (AC,R3). Current gland segmentation methods mainly rely on supervised & weakly-supervised methods. However, even weakly-supervised gland segmentation can be laborious, requiring expert knowledge. This is particularly pronounced when dealing with WSIs, ranging in size from gigabytes to terabytes. Therefore, this paper aims to propose an unsupervised gland segmentation method that eliminates the need of manual labeling. Q2: Clarify the generalizability (R1). This paper focuses specifically on gland segmentation, a critical task of medical image analysis. Additionally, the idea of semantic grouping is applicable to various tasks, given that numerous tissues&organs consist of sub-parts with distinct appearances. Q3: Method details: how to select proposals from candidate regions, why 5 clusters in Yi and why 2 clusters are used for pseudo labels (R1). We discussed proposal selection in Section 2.1.2. We used K-means to cluster feature map of each gland image into 5 (determined by Elbow Method) candidate regions. Next, based on prior that gland comprises two sub-regions with distinct appearances, we initially selected gland border proposal from 5 candidates, then expanded the selection to encompass the surrounding area as proposals for the remaining gland regions. As a result, the proposal map contained only 2 proposal labels. Q4: Explanations of Eq. 1&2 (R1). Eq.1 & 2 are loss functions to train the encoder. Eq.1 refers to self-supervised loss which uses cluster labels as pseudo labels to train the encoder. Eq.2 aims to minimize variance among pixels within a specific area, thereby enhancing completeness of each candidate region. Q5: Why use a shallow encoder (R2). We chose a shallow encoder emphasizing low-level appearance features over a deep encoder focusing on high-level semantic features. The rationale behind this is as follows. The aim of our SPM is to use cues about gland appearance to extract gland sub-region proposals for subsequent semantic grouping stage. We aim for each extracted candidate region to reflect the appearance of specific partitions within the gland image. Therefore, we opt for a shallow encoder that prioritizes the extraction of appearance features. Q6: Will shallow encoder be sensitive to color (R2). The shallow encoder may be sensitive to color and segment one gland into multiple parts. However, please note that its produced candidate regions are not final predictions. By selecting gland sub-region proposals from candidate regions, we can use two MSG modules in the next stage to compel final segmentation network to recognize these sub-regions as parts of the same tissue. Q7: Numerical results of spatial continuity loss (R3). Numerical results, showing Lsc enhances the mIOU of gland proposals from 44.35% to 51.45%, will be added. Q8: Hyper-parameters tuning-cluster number K and threshold β (R3). For SPM, we determine cluster numbers via Elbow Method, eliminating the need for validation set. Moreover, we observed that proposal quality is not significantly affected by the cluster number. In fact, the mIOU variation of gland proposals remains within 1% for cluster numbers ranging from 3 to 8. To tune β, we use pseudo-labels of a segmentation network trained solely with MSG for Variation. Q9: Robustness towards seeds (R3). The result of robustness test is in section Q7, where the results are the average of 10 tests with different seeds.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Although reviewers confirmed the merits of interesting idea and improved performance, the clarity on the methodology part remains ambiguous and the generalization capability to other tasks remains unclear. Overall, I feel the weakness is over than the merits given the current version.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal has addressed the reviewers’ concerns regarding the method’s details and motations. Overall, this paper has novelty, and the performance is promising. The final version should further address reviewers’ concerns regarding the selections of some important hyperparameters, and include some discussions on clinical applications.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper proposes a two-stage unsupervised method for gland segmentation. The first stage involves generating gland segmentation pseudo labels based on pixel-wise similarity. The second stage utilizes the pseudo labels as ground truth and focuses on pushing pixels inside gland borders together. The results seem to be good but the reviewers have raised concerns about the clarity of the paper, the lack of explanations in certain areas, and the need for further validation and evaluation. The method relies on specific assumptions about gland regions, which may limit its generalizability. However, given the promising results on gland segmentation, I am probably still slightly toward accepting the paper but also okay with rejection based on other ACs’ decisions.