List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Faquan Chen, Jingjing Fei, Yaqi Chen, Chenxi Huang
Abstract
By fully utilizing unlabeled data, the semi-supervised learning (SSL) technique has recently produced promising results in the segmentation of medical images. Pseudo labeling and consistency regularization are two effective strategies for using unlabeled data. Yet, the traditional pseudo labeling method will filter out low-confidence pixels. The advantages of both high- and low-confidence data are not fully exploited by consistency regularization. Therefore, neither of these two methods can make full use of unlabeled data. We proposed a novel decoupled consistency semi-supervised medical image segmentation framework. First, the dynamic threshold is utilized to decouple the prediction data into consistent and inconsistent parts. For the consistent part, we use the method of cross pseudo supervision to optimize it. For the inconsistent part, we further decouple it into unreliable data that is likely to occur close to the decision boundary and guidance data that is more likely to emerge near the high-density area. Unreliable data will be optimized in the direction of guidance data. We refer to this action as directional consistency. Furthermore, in order to fully utilize the data, we incorporate feature maps into the training process and calculate the loss of feature consistency. A significant number of experiments have demonstrated the superiority of our proposed method.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43907-0_53
SharedIt: https://rdcu.be/dnwdA
Link to the code repository
https://github.com/wxfaaaaa/DCNet
Link to the dataset(s)
ACDC: https://www.creatis.insa-lyon.fr/Challenge/acdc/databases.html
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a novel approach to consistency-based training of segmentation neural networks. During training, the authors compare two segmentations obtained based on two independent augmentations of the input image. Unlike prior work that encourages the two segmentations to be consistent uniformly across the images, the proposed training method treats areas of high confidence and low confidence differently. Specifically, in the areas where the model is confident in its decision (the probability of any label is greater than a threshold), the two segmentations are forced to agree via a separate loss term. In areas where one segmentation is of high confidence and the other segmentation has low confidence, the low confidence segmentation is forced to be similar to the high confidence segmentation. The authors demonstrate the advantages of this approach on two different image sets, achieving substantial improvements in segmentation accuracy.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper presents a novel and interesting idea. It’s the best paper in my stack in terms of novelty. The experimental results support that the proposed method has a potential to significantly improve the quality of segmentation when few annotations are available, along with a lot of unlabeled images.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper is confusing and the presentation could use improvement. In spite of this, I support acceptance of the paper as it has a new idea, which seems to be a rare event.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Knowledgeable reader would be able to reproduce the approach.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I really like the idea, but the paper needs a careful editing pass to make the presentation of the method more clear.
While changing the model updates based on some indication of what is likely to be a correct segmentation is a great one, the model’s confidence might not be a great measure for this. The models are often confident and wrong, especially at the beginning of training. Instead, it might be better to estimate the areas of high uncertainty and low uncertainty by performing a set of augmentations and obtaining a set of segmentations for the same image. Then segmentations that are close to consensus labeling can be used as a “pseudo ground truth” (like the segmentations of high confidence in the paper).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See above.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
Mine is the most positive review and the authors did not dissuade me from the original recommendation. :)
Review #2
- Please describe the contribution of the paper
Authors present a method to perform semi-supervised learning that makes full use of the unlabeled data by decoupling consistent and inconsistent sections and then apply them according to function.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The use of decoupled set of consistent and inconsistent unlabelled data is best of both pseudo labeling and consistency based methods for semi-supervised modeling. This is a novel idea. Furthermore, defining guidence data to move the low confidence samples towards hig density ares is also an intuitive and smart idea,
The motivation and the methodology is presented clearly and is easy to understand.
The experimental section is also rich and has a good supporting set of ablations. And they provide results on two different anatomies leading support to the generality of the method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Needs to address
– For eq 5 how are the lSP and hSP are obtained, the paper does not specify. Additionally, after sharpning these are values between 0-1 where CE is more suited. If there a reason why L2 was preferred?
– Motivation for using two different decodes with slightly different architectures is not clear. Similarly, it is not clear why the feature maps are only used for decoder B and not A. Please elaborate.
Minor nitpicks
– Please mention how the hyperparameters were chosen, and how sensitive is the method to their choice. – PROMERE12 -> PROMISE12
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
very good, results on public dataset and all the details are specified.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The authors have presented a well constructed method for using all the unlabelled data in semi-supervised learning for medical segmentation. The results and ablations are comprehensive.
Minor explanations (as highlighted in weaknesses) are missing for the choices made by the authors which might provide more context, reproducibility, and intuition behind the method.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think this paper presnts a simple yet novel combination of ides, resulting in an intuitive and general semi-supervised model for segmentation. The results and ablation are comprehensive giving me trust in the method.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Considering is authors present the arguments they presented in the rebuttal to the final review + supplemental would enhance the paper.
I consider this an accept.
Review #3
- Please describe the contribution of the paper
This paper investigates semi-supervised medical image segmentation, by combining pseudo labelling with tuned threshold and SimSiam-style consistency regularisation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper is addressing an important issue and the components of the methods are well validated in previous literature.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The motivation of the proposed methods are not very clear and the claims are not rigorous. For example, “We initially sharpen the confidence of these pixels to bring the high-confidence pixels closer to the high-density area”, I am not sure if sharpening confidence can actually do that, as the cluster assumption is violated at pixel-wise of images.
Equation 5 appears to be consistent regularisation in SimSiam. If that is the case, then I am not very convinced that it is necessary to decouple the data.
The key terminologies are confusing and not well defined. For example, the authors make up terms such as unreliable data without defining it carefully. And there is no explanation of guidance data at all.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code is NOT promised to be released so I encourage the authors to release the code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The Figure 1 is very confusing, which is the main figure of the paper.
No standard deviation is reported in the results.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper has many flaws from a foundational level.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a neural network architecture for image segmentation by utilizing consistency-based training. This paper received mixed ratings. Please the authors address the questions/concerns raised by the reviewers, mainly for: (1) explain how the equations obtained pointed by R#2; and (2) how the presented method is motivated pointed by R#2 and R#3, especially for the comments from R#3.
Author Feedback
Many thanks to the reviewers for their comments on our work. Here, I will explain the questions raised by the reviewers.
To Reviewer 2 & Reviewer 3
Q1: How were the lSp and hSp obtained. The specific acquisition process is as follows: mask = (SpA > SpB) hSpA = torch.mul(mask, SpA) lSpB = torch.mul(mask,SpB) Similarly, hSpB and lSpA can be obtained. The guidance data is hSpA (hSpB), and the unreliable data is lSpA (lSpB).
To Reviewer 3
Q1: About the motivation of proposed method. Our goal is to fully utilize unlabeled data; Because the traditional pseudo label method will filter out the pixels with low confidence, the consistency regularization method does not divide the functions of the data although it uses all the data, so it does not make full use of the data in a real sense. Our method not only utilizes all the data, but also divides the data into different functions, with data from different functions playing different roles. We have demonstrated the effectiveness of data partitioning through experiments, as shown in Tables 3 and 4 of the experimental section.
Q2: About Figure 1. Divided into left and right sides by the red threshold in the graph. The left side of the graph contains supervised loss lseg and feature consistency loss lf. On the right side of the graph are the consistent and inconsistent parts obtained through threshold filtering. For consistent parts, cross supervision is used, and for inconsistent parts, directional consistency strategy is used.
Q3: About the necessity of decoupling the data. Please see Q1.
Q4: About releasing code. During the blind review stage of the paper, personal links are not allowed. After the paper is accepted, it will be open-source code.
Q5: About sharpening. Softmax can normalize data to [0,1]. We set a temperature coefficient T (0<T<1). The temperature coefficient T can affect the predicted distribution, thereby controlling the model’s discrimination of samples. The larger the T, the smoother the distribution; The smaller the T, the sharper the distribution. We have demonstrated that the effect of sharpening will be better, as shown in Appendix Table 1.
To Reviewer2
I am very sorry that additional experimental results is not allowed for this rebuttal. But the following conclusions have been experimentally proven.
Q1: Why L2 was preferred in the inconsistent part ? Both CE and L2 are optional, but L2 performs better. We will supplement the experimental results in the appendix for the camera-ready version.
Q2: Why use a dual decoder architecture ? We use two heads in the training phase for higher accuracy, but in the inference phase, we only use one head, which has no additional computational overhead compared to the single-head method. And compared to a dual encoder-decoder architecture, a dual-decoder architecture only requires one forward propagation, and less time.
Q3: Why the feature maps are only used for decoder B and not A ? Decoder A employs bi-linear interpolation for up-sampling, and decoder B employs transpose convolution for up-sampling. We found that using only decoder B for feature maps yields better results. We guess that it may be because transposed convolution can restore images better than bilinear interpolation.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal has largely addressed the concerns raised by R#2 and R#3. Accept
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After thoroughly evaluating the authors’ feedback and the final decisions of the reviewers, it is apparent that the majority of the reviewers lean towards accepting the paper. They recognize the value of the work and the idea it brings to the community.
While one reviewer leans towards rejection, they still acknowledge the strengths of the paper.
After careful consideration of the reviewers’ feedback and opinions, the Meta Reviewer agrees with the majority sentiment. The positive reception of the paper by the majority of the reviewers reinforces the belief that it holds significant value.
Based on this assessment, the Meta Reviewer suggests accepting the paper. The recognition of the paper’s value and the positive reception it has received indicate its potential to make a meaningful contribution to the field.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This manuscript introduces a pioneering and thought-provoking concept that utilizes semi-supervised learning to maximize the potential of unlabeled data. It cleverly partitions the data into consistent and inconsistent portions and employs them based on the inherent characteristics of the task at hand. The authors, in their rebuttal, adeptly addressed the primary concerns put forward by the reviewers, making a compelling case for the acceptance of the paper. If the manuscript is accepted, I recommend that the authors incorporate the insights and discussions elucidated in their rebuttal into the final version of the paper. Moreover, the overall presentation of the paper could be further elevated.