Authors

Heng Cai, Lei Qi, Qian Yu, Yinghuan Shi, Yang Gao

Abstract

Medical image segmentation typically necessitates a large and precisely annotated dataset. However, obtaining pixel-wise annotation is a labor-intensive task that requires significant effort from domain experts, making it challenging to obtain in practical clinical scenarios. In such situations, reducing the amount of annotation required is a more practical approach. One feasible direction is sparse annotation, which involves annotating only a few slices, and has several advantages over traditional weak annotation methods such as bounding boxes and scribbles, as it preserves exact boundaries. However, learning from sparse annotation is challenging due to the scarcity of supervision signals. To address this issue, we propose a framework that can robustly learn from sparse annotation using the cross-teaching of both 3D and 2D networks. Considering the characteristic of these networks, we develop two pseudo label selection strategies, which are hard-soft confidence threshold and consistent label fusion. Our experimental results on the MMWHS dataset demonstrate that our method outperforms the state-of-the-art (SOTA) semi-supervised segmentation methods. Moreover, our approach achieves results that are comparable to the fully-supervised upper bound result.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43898-1_59

SharedIt: https://rdcu.be/dnwBT

Link to the code repository

https://github.com/HengCai-NJU/3D2DCT

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

This paper targets reducing the amount of annotation required for medical image segmentation. Existing approaches have you sparse annotation meaning large glass gaps to preserve slash reduce rater involvement this paper proposes a framework that robustly learns from sparse annotation. Moreover, it focuses on teacher networks for both three dimensional and two dimensional approaches. Results are shown to be comparable to fully supervised approaches.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Prior contexts include image level bounding box scribble and point wise annotations. The key area of the field is weak label and weak supervision in deep learning training. The key limitation is the existing performance gap between supervised and semi supervised slash weakly supervised methods.

The work follows up on reference number two.

Reference to was published in 2020, and IEEE Journal of biomedical and health informatics for 3d image segmentation with sparse labels.

Both references to and 10 train on sparsely labeled data through slides gaps registration models have been key to their prior performance.

This work expands preference 13 By adapting different network dimensions

The key linkage is utilizing both three dimensional and whodunit jewel networks to reduce pseudo labels, which are then used to cross train

ad hoc strategies are introduced to balance two dimensional and three dimensional training and two dimensional three dimensional uncertainty. The methods appear effective, but the theoretical underpinnings of these approaches are not well explored.

Traditional v net and unet backbones are used for the three dimensional and two dimensional networks respectively.

An effective set of baselines including mean teacher, uncertainty aware mean teacher cross pseudo supervision and crossed teacher between CNN and transformer approaches are offered as baselines the transformer network is based on unit R.

All methods are evaluated with 16 labelled slices per volume.

Interestingly, the results outperform a fully supervised v net

The ablation experiment shows a reasonable dependence on the specified parameters but no overt sensitivity.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

In terms of constructive criticism, the primary concern is the ad hoc nature of the proposed optimization criteria. Demonstration on a single test case is effective but introduces concerns relative to generalizability of the approach, and generalizability of the ad hoc rules. A second example, even a second toy example in a different context would greatly improve stability and assurance of the reproducibility of these results.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Methods are clearly presented. No concerns.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

In terms of constructive criticism The primary concern is the ad hoc nature of the proposed optimization criteria. demonstration on a single test case is effective but introduces concerns relative to generalizability of the approach, and generalizability of the ad hoc rules. A second example, even a second toy example in a different context would greatly improve stability and assurance of the reproducibility of these results.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Overall this paper is interesting, the results are promising.

The explanation of ad hoc solutions is clear. And the organizational structure of the paper leads to a reasonable level of reproducibility

The lack of a second dataset is slightly problematic.

However, the extension is not absolutely essential for a publication in this venue.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

This paper demonstrates a method on 3D segmentation with sparse annotation via cross-teaching between 3D and 2D networks. This is a practical problem setting in medical imaging field. It 1) trains 2D segmentation network with sparse 2D annotation and fuses 2D segmentation results in different views into 3D pseudo segmentation masks; 2) trains 3D segmentation network with sparse 2D annotation; 3) the prediction of 2D and 3D networks are used as pseudo label for the other network after selection.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) The problem setting is practical 2) The method is novel
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The major concern is on the experiment section. For MMWHS dataset, the data split of 12/4/4 gives results for only 4 testing samples. It is difficult to prove any improvement with such a small testing set. It would be necessary to perform k-fold testing or experiment on more datasets.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

No code provided
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

The major concern is on the experiment section. For MMWHS dataset, the data split of 12/4/4 gives results for only 4 testing samples. It is difficult to prove any improvement with such a small testing set. It would be necessary to perform k-fold testing or experiment on more datasets.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The method is novel
Reviewer confidence

Somewhat confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper presents a novel framework for effective learning from sparsely annotated data, utilizing cross-teaching of 3D and 2D networks. To account for the unique properties of these networks, this paper devises two pseudo label selection techniques: the hard-soft confidence threshold and consistent label fusion. Through experimental evaluation on the MMWHS dataset, the proposed method outperform existing state-of-the-art techniques.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

(1) 3D CNNs can capture inter-slice relationships and 2D CNNs are efficiently for inner-slice information. The 3D and 2D network can benefit from each other.

(2) The pseudo label selection strategies are novel. For the 3D network, a hard-soft threshold is used to estimate the quality of predictions and select high-quality ones as pseudo labels. Those exceeding the hard threshold are used to supervise 2D networks. For 2D networks, consistent predictions of two 2D networks are used instead of calculating uncertainty.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

(1) Lack the results of fully supervised 2D (U-Net) for comparison.

(2) Lack of analysis why outperform Fully-supervised V-Net.

(3) More ablation studies of Labeled Slices like 8 would make this paper better.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Through Implementation Details.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

please refer the weaknesses
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

To further explicate the proposed methodology’s superiority over the fully-supervised model, it is imperative to conduct an in-depth analysis of the underlying mechanisms of the 2D network’s functionality. Additionally, conducting extensive experimentation with various labeled slices would strengthen the findings of this study.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Based on the reviews provided, I recommend accepting this paper. The reviewers acknowledge the novelty of the proposed framework for learning from sparsely annotated data in medical image segmentation. The method effectively utilizes cross-teaching between 3D and 2D networks and introduces novel pseudo label selection techniques. The experimental results demonstrate that the proposed method achieves comparable performance to fully supervised approaches. The paper is commended for its clear writing, organization, and the inclusion of relevant references. Although there are some weaknesses, such as the ad hoc nature of the optimization criteria and the limited evaluation on a single dataset, they are outweighed by the interesting nature of the paper and its potential contributions to the field. The reviewers suggest further improvements, such as exploring additional datasets and conducting in-depth analyses of the underlying mechanisms, which should be appropriately discussed in the final version. Overall, the consensus among the reviewers is to accept this paper for publication.

Author Feedback

N/A

back to top

3D Medical Image Segmentation with Sparse Annotation via Cross-Teaching between 3D and 2D Networks