Authors

Zhenqi He, Mathias Unberath, Jing Ke, Yiqing Shen

Abstract

Nuclei appear small in size, yet, in real clinical practice, the global spatial information and correlation of the color or brightness contrast between nuclei and background, have been considered a crucial component for accurate nuclei segmentation. However, the field of automatic nuclei segmentation is dominated by Convolutional Neural Networks (CNNs), meanwhile, the potential of the recently prevalent Transformers has not been fully explored, which is powerful in capturing local-global correlations. To this end, we make the first attempt at a pure Transformer framework for nuclei segmentation, called TransNuSeg. Different from prior work, we decouple the challenging nuclei segmentation task into an intrinsic multi-task learning task, where a tri-decoder structure is employed for nuclei instance, nuclei edge, and clustered edge segmentation respectively. To eliminate the divergent predictions from different branches in previous work, a novel self-distillation loss is introduced to explicitly impose consistency regulation between branches. Moreover, to formulate the high correlation between branches and also reduce the number of parameters, an efficient attention sharing scheme is proposed by partially sharing the self-attention heads amongst the tri-decoders. Finally, a token MLP bottleneck replaces the over-parameterized Transformer bottleneck for a further reduction in model complexity. Experiments on two datasets of different modalities, including MoNuSeg have shown that our methods can outperform state-of-the-art counterparts such as CA-2.5 by 2-3% Dice with 30% fewer parameters. In conclusion, TransNuSeg confirms the strength of Transformer in the context of nuclei segmentation, which thus can serve as an efficient solution for real clinical practice. Code is available at https://github.com/zhenqi-he/transnuseg.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43901-8_20

SharedIt: https://rdcu.be/dnwC4

Link to the code repository

https://github.com/zhenqi-he/transnuseg

Link to the dataset(s)

https://github.com/lu-yizhou/ClusterSeg

Reviews

Review #3

Please describe the contribution of the paper

This paper propose three techniques to improve nuclei segmentation on microscopy and histology images: (1) a multi-task transformer-based segmentation network that both segment nuclei contours and detect nuclei edges; (2) a novel self-distillation loss that enforce edge and contour boundary consistency; (3) an attention sharing scheme among decoders. Experiments show that all of the proposed techniques are effective and the final model advance SOTA results.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The paper is generally well-written and easy-to-follow;
2. The motivations and ideas of attention sharing and self-distillation loss are interesting;
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Some of the proposed designs are incremental and trivial. For example, the MLP bottleneck are directly borrowed from [16], and the multi-task training framework are largely based on [6];
2. In table 2, while the designed models use less parameters than SwinUNet, the computation complexity in terms of FLOPs is significantly larger, which weakens the value of the proposed design;
3. The effectiveness of self-distillation loss needs to be further justified. The edge detection modules have already been supervised by the edge-related loss, the reasons why the additional supervision from the contour branch would benefit the final segmentation performance is unclear.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The method should be easy to reproduce.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Please see the weakness section.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper successfully combines several techniques from other fields and achieve new SOTA results. The quantitative experiments are done thoroughly. The motivation of self-distillation consistency loss is interesting but the effectiveness needs further justification.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

This paper proposes TransNuSeg, a network that predicts segmentation mask, nuclei edges, and cluster edges. Multiple techniques are proposed to share information and reduce computations among the three decoders. Results on two datasets as well as complexity are presented.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The flow of the paper is pretty well. Readers can follow the logic easily.
- The proposed method is somewhat new.
- The results are clear state-of-the-arts.
- Complexity comparison and ablation studies are thorough.
- Source code is provided.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Although the authros did a good job in Figure 1. demonstrating high level ideas of the proposed method. The legend of the figure is really confusing. It took me a while to find where the consistency regulation mark is. The authors should improve this figure for better clearness.
- After decoding the three properties for nuclei, how should one combine them together? Also, are there ground truth nuclei edges and cluser edges that can be directly used as supervision in the datasets? These important details are skipped in the paper.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Reproducible, code is provided.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

If there is a rebuttal for this paper, I encourage the authors to answer the questions I raised in the weakness section.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

After fixing the clarification issues and add more details, I think this paper is ready to be published at MICCAI.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #1

Please describe the contribution of the paper

This work introduces TransNuSeg, a novel pure Transformer framework for nuclei segmentation, which addresses the challenges of the task by decoupling it into a multi-task learning problem with a tri-decoder structure for nuclei instance, nuclei edge, and clustered edge segmentation. To ensure consistency of predictions across branches, a self-distillation loss is employed between the nuclei branch and edge branch, while an attention-sharing scheme reduces the number of parameters by sharing self-attention heads among the tri-decoders. A token MLP bottleneck further decreases model complexity. Experiments demonstrate the superiority of TransNuSeg over state-of-the-art methods on the Fluorescence Microscopy Image Dataset and Histology Image Dataset.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The strengths of this work can be summarized from two aspects. From the architectural perspective, the shared Swin Transformer across three decoders and the use of a bottleneck token MLP to improve model efficiency are novel. Regarding learning tasks, the joint learning of nuclei instance, nuclei edge, and clustered edge is reasonable for making better use of the nuclei object labels.

The paper is generally well-written and easy to follow. The visualizations are also informative.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

There lacks a clear description on how the clustered edge is defined. From the visualization the clustered edge contains the pixels between closely touching instances. But it will be better if it can be defined mathematically.

There are existing works that segment nuclei/cells by learning multiple targets like semantic mask, edge mask, distance transform, flow map etc. The authors should cite those literature [A, B] and discuss the similarity and differences.

[A] Nucmm dataset: 3d neuronal nuclei instance segmentation at sub-cubic millimeter scale. MICCAI, 2021 [B] Cellpose: a generalist algorithm for cellular segmentation. Nature Methods, 2020
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The code is shared. I have no concern on reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Please see the discussion on weaknesses.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This work presents a nuclei segmentation framework that improves existing methods from both architectural and learning perspectives. The experiments are also comprehensive to demonstrate the effectiveness of each component. Therefore I would recommend an accept.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The work addresses the challenges of the task by decoupling it into a multi-task learning problem with a tri-decoder structure for nuclei instances. All reviewers agreed that the proposed method is moderately novel, motivations and ideas of attention sharing and self-distillation loss are interesting, clear performance improvement, thorough complexity comparison and ablation studies, well-written and easy to follow. Metareviewer agrees that the work would make an interesting addition to the conference and strongly recommend the authors address the reviewers’ further comments before the camera-ready submission.

Author Feedback

N/A

back to top

TransNuSeg: A Lightweight Multi-Task Transformer for Nuclei Segmentation