Authors

Xiangde Luo, Minhao Hu, Wenjun Liao, Shuwei Zhai, Tao Song, Guotai Wang, Shaoting Zhang

Abstract

Medical image segmentation plays an irreplaceable role in computer-assisted diagnosis, treatment planning and following-up. Collecting and annotating a large-scale dataset is crucial to training a powerful segmentation model, but producing high-quality segmentation masks is an expensive and time-consuming procedure. Recently, weakly-supervised learning that uses sparse annotations (points, scribbles, bounding boxes) for network training has achieved encouraging performance and shown the potential for annotation cost reduction. However, due to the limited supervision signal of sparse annotations, it is still challenging to employ them for networks training directly. In this work, we propose a simple yet efficient scribble-supervised image segmentation method and apply it to cardiac MRI segmentation. Specifically, we employ a dual-branch network with one encoder and two slightly different decoders for image segmentation and dynamically mix the two decoders’ predictions to generate pseudo labels for auxiliary supervision. By combining the scribble supervision and auxiliary pseudo labels supervision, the dual-branch network can efficiently learn from scribble annotations end-to-end. Experiments on the public ACDC dataset show that our method performs better than current scribble-supervised segmentation methods and also outperforms several semi-supervised segmentation methods. Code and data at: https://anonymous.4open.science/r/MICCAI22-4E2B.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_50

SharedIt: https://rdcu.be/cVD66

Link to the code repository

https://github.com/HiLab-git/WSL4MIS

Link to the dataset(s)

https://acdc.creatis.insa-lyon.fr/

Reviews

Review #1

Please describe the contribution of the paper

In this paper, the authors present a technique for scribble-based medical image segmentation. Their approach features a dual branch network that implements a perturbation-consistency strategy, encouraging the network to produce sensible segmentations despite having only scribbles as supervisory signal. Experiments and ablation studies suggest that the proposed technique is robust and accurate w.r.t several baselines.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The paper addresses a very relevant problem, i.e. the elevated cost of pixel-wise annotations for medical image segmentation.
2. The methodology is novel, sound and interesting.
3. The literature review and comparisons with other available approaches, both in the realm of weakly supervised and semi-supervised methods, are detailed and sensible.
4. The experimental set-up is (mostly) well-devised and the results are convincing.
5. Different settings, ablation studies and comparisons against a rather ample set of baselines allow a rather good assessment of the robustness of the proposed approach.
6. The paper is very well-written and generally easy to follow.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Single-dataset experimental set-up: the authors limited their experiments to a single dataset (ACDC). While this is a very well-known dataset, experiments with additional datasets are required to better assess the real performance of the described technique. This being said, I don’t think additional experiments are strongly necessary for a MICCAI submission, also considering the ample comparisons with different types of baselines that are available in the paper.
2. Lack of discussion: the authors reported their results against many baselines in Table 1. However, there is no critical discussion of the achieved results. Specifically, do the authors have any idea why their method outperforms RLoss in DSC rather substantially, but do much worse in HD? Also, how can a method that produces rather poor DSC scores like RW provide instead such remarkable HD results? These are important aspects that need to be clarified.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility of the paper is very high. Methods, implementations and settings are quite clear. Codebase is released. Dataset used is public.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1. Page 6: Table 1 is barely legible. Please improve the layout.
2. There are a few parts of the paper that I find unclear. Page 6-7: “we trained networks with partially supervised and semi- supervised fashions, respectively. We used a 10% training set (8 patients) as labeled data and the remaining as unlabelled data, as the scribble annotation also takes similar annotation costs [29]”. Page 8: “3) the proposed approach dynamically mixes two outputs to generate hard pseudo labels for two decoders training separately”. Please re-phrase/clarify these parts.
3. Please consider adding supplementary materials with more qualitative results (as Fig.3).
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is highly interesting. It presents a novel technique for scribble-based segmentation, which favourably compares to several baselines. The experimental design is restricted to a single dataset, but thanks to ablation studies and further analyses the results are convincing.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper proposes a simple yet efficient dual-branch network with one encoder and two slightly different decoders for image segmentation, which combines the scribble supervision and auxiliary pseudo labels supervision and performs better than current scribble-supervised segmentation methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is well-written and easy to follow. The proposed method adopts dual-branch network and a dynamically mixed pseudo labeling strategy to train segmentation models with scribble annotations, which reduces annotation costs and makes good use of a small amount of supervision information. Experiment results demonstrates the effectiveness of the propose method.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

To increase persuasiveness of the proposed method, the experimental settings should be illustrated more detailed, include comparison methods. The authors did not mention how much supervision information used of each comparison WSL methods. The proposed method adopts a dual-branch network, include main decoder, and an auxiliary decoder, the authors should give a comparative experiment between w/o auxiliary decoder to illustrate the effectiveness of auxiliary decoder. The network architecture is critical, it could influence the final results significantly. However, main decoder+auxiliary decoder has been used in previous work(UDC-Net: Dual-Consistency Semi-Supervised Learning with Uncertainty Quantification for COVID-19 Lesion Segmentation from CT Images, MICCAI’21), which is not a new idea, and the authors should make some analysis between UDC-Net and the proposed method in the experiment.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors released the source code of this paper, and it is not hard to reproduce the experiment.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

The proposed dual-branch network is similar to the mean-teacher architecture, I prefer more discussions about the differences and strengths of the proposed algorithm compared to the mean-teacher architecture. Besides, the authors should add more experiments on other datasets with scribble annotation(such as PASCAL-Scribble Dataset) to improve the persuasiveness.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper proposes a simple yet efficient dual-branch network with one encoder and two slightly different decoders for image segmentation. And the paper is well-written and easy to understand. However, the proposed method is not a new idea, and the experiment is insufficient. There are several concerns above in the current version of the paper that addressing them will increase the quality of this paper.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper presents a dual branch network for MR cardiac segmentation using scribbles and auxiliary pseudo-labels. The paper is well written and well explained. The proposed idea is novel and the results achieved are good. The reviewers agree on the value of the work. However, they have some minor critics that the authors are adviced to address.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

2

Author Feedback

Sincerely thanks to all reviewers and meta-review for their positive and constructive comments. We believe that the constructive feedback will help us improve the quality of the paper and promote further studies on this weakly-supervised segmentation topic. First, we apologize for the unclear experimental settings. They will be clearly clarified for better understanding. Second, we will evaluate the proposed method on more datasets. In addition, we will provide the detailed code of the proposed and all the comparisons and examples on public datasets. Finally, we will provide more details descriptions and discussions in the next version.

back to top

Scribble-Supervised Medical Image Segmentation via Dual-Branch Network and Dynamically Mixed Pseudo Labels Supervision