Authors

Zudi Lin, Donglai Wei, Aarush Gupta, Xingyu Liu, Deqing Sun, Hanspeter Pfister

Abstract

Objects with complex structures pose significant challenges to existing instance segmentation methods that rely on boundary or affinity maps, which are vulnerable to small errors around contacting pixels that cause noticeable connectivity change. While the distance transform (DT) makes instance interiors and boundaries more distinguishable, it tends to overlook the intra-object connectivity for instances with varying width and result in over-segmentation. To address these challenges, we propose a skeleton-aware distance transform (SDT) that combines the merits of object skeleton in preserving connectivity and DT in model- ing geometric arrangement to represent instances with arbitrary structures. Comprehensive experiments on histopathology image segmentation demonstrate the state-of-the-art performance achieved by SDT.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43898-1_51

SharedIt: https://rdcu.be/dnwBL

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The authors propose a loss function that takes into account the distance of each pixel to the medial axis of a binary shape, thus enforcing the importance of shape bootle necks in cell segmentation. They show how at inference time to use watershed transform to recover the different instances.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- simple idea which adds to the concept of boundary and watershed distance.
- the figures illustrate well the benefit of the skeleton aware approach
- compelling quantitative analysis
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- the authors should provide examples where it is better to use boundary-based or watershed-based losses than skeleton-based losses. What are the limitations of the approach ?
- Computing a skeleton is sensitive to noise and there are many ways to compute it. The authors only provide a single method and should discuss this issue more thoroughly
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

ok
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- please provide i) limitations of the approach : when is it better to use this loss compared to boundary or watershed losses ii) discuss the issue of skel;eton extraction which is sensitive to noise. Is it necessary to have clean segmentation masks ?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

the paper is easy to read, and present their idea with good illustrations. The proposed concept nicely add-up to the already existing approaches
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

This paper proposes a skeleton-aware distance transform (SDT) that combines the merits of object skeleton in preserving connectivity and DT in modeling geometric arrangement to represent instances with arbitrary structures.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The writing is clear and well organized with clear motivation.
2. The figure quality is good.
3. Experiment is relatively comprehensive.
4. The idea of taking connectivity and distance transform in modeling geometric arrangement is novel.
5. The method is easy to follow and technically sound.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The performance improvement is limited.
2. Instance segmentation performance should be provided which should achieve better improvement.
3. Reference [18] shares a similar idea but for a different task which should be discussed in details.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Easy to reproduce.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

I like the idea of this paper though it shares a similar idea with [18] but for a different task. The different should be discussed in detail. Another flaw is the limited improvement. The main reason is that the paper adopted a very old and popular dataset which has been explored for many years. Another reason is that only segmentation performance is provided. In fact, in my view, the method is more useful in instance segmentation, which should obtain much better improvement. More experiments should be performed in 2D and 3D datasets for instance segmentation.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The novelty of the method.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

A new segmentation method that combines skeleton connectivity and geometric arrangement is proposed and validated.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-The integration of skeleton connectivity with geometric segmentation is a novel design to address the issue of oversegmentation in traditional method
- The performance is excellent in comparison with SOTA in the histology dataset.
- The paper is well-written and easy to follow.
- Hyperparameter is well discussed.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

-The application of the proposed segmentation method is limited to the case when multiple objects are intra-connected. The performance may not differ that much in task with single objects. For example, segmenting single cancer region within a medical image.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

This paper is based on public dataset.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-Cross-validation. It is not clear how training and testing are set-up
- It will be great to highlight the best number in ablation study.
- As discussed in weakness, a discussion on the application of proposed study should be provided.
- It is not clear whether this method will fit a multi-class problem setting.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The score driving factor is the novelty in which the paper brings new concept (skeleton-awareness) in segmentation task.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

Based on the reviews provided, I recommend accepting this paper. The reviewers appreciate the novel formulation of a skeleton-aware distance transform (SDT) for cell segmentation. The proposed method combines the advantages of object skeleton in preserving connectivity and distance transform in modeling geometric arrangement, resulting in a comprehensive representation of instances with arbitrary structures. The paper is commended for its clear writing, well-organized presentation, and compelling quantitative analysis. The integration of skeleton connectivity with geometric segmentation is deemed a novel design that effectively addresses the issue of over-segmentation. The performance of the method is shown to be excellent, particularly in comparison to state-of-the-art approaches on histology datasets. The reproducibility of the paper is rated positively. Although there are minor weaknesses, such as the need for further discussion on limitations and different applications of the proposed method, they do not significantly undermine the overall quality and contributions of the paper. Consequently, the consensus among the reviewers is to accept this paper for publication. The weaknesses should be carefully addressed in the final version though.

Author Feedback

We thank the reviewers and meta-reviewer for the constructive feedback. We are encouraged by the positive comments on the novelty of our SDT algorithm and the informativeness of our illustrations and experiments. We address the remaining questions in this response, and will incorporate the feedback in the final version.

[R1] Sensitivity to noise in skeleton generation. “We agree that when the object mask contains small perturbations on the boundary, the skeletonization algorithm can produce unwanted branches. In our implementation, we actually smooth the masks before computing the skeleton by applying Gaussian filtering and thresholding. We will add these details to the final version.

[R1] Limitations. One scenario that is challenging for SDT to handle is when an object has two or more connected components due to occlusion, which would require an additional linking step. We will discuss such cases in the revision.

[R2] Evaluation metrics and performance improvement. We reported the instance segmentation performance in Tables 1 and 2 (see also Fig. 5) for the instance masks. Unlike Mask R-CNN models that usually report mAP, we report F1 score, Dice Index, and Hausdorff distance at the instance level. Table 1 shows that under the Hausdorff distance metric for evaluating morphological similarity, our SDT performs significantly better than existing approaches, which verifies that learning skeleton-guided energy can better preserve shape.

[R2] Connection and comparison with [18]. The DDT method in [18] generates the energy map using the distance to the object surface, which is similar to the DT baseline shown in Fig. 1(c). Since [18] targets the segmentation of tubular structures, which have relatively uniform widths, the energy peaks in distance transform maps are close to the skeleton. However, our SDT method explicitly uses the skeleton in energy calculation, which makes it more robust for objects with thin connections (Fig. 1(d)).

[R3] Experimental settings. The Gland Challenge dataset has predefined train/test splits covering 85 and 80 images, respectively. During training, we use a small set of training images for validation. The evaluation uses the official scripts provided by the Gland Challenge organizers.

[R3] More applications. We believe SDT is a good fit for segmenting structures with non-convex shapes and non-uniform widths, which are common in biomedical imaging tasks. For multi-class problems, we can use class-aware semantic segmentation to mask the SDT energy trained for all objects that is agnostic to their classes.

back to top

Structure-Preserving Instance Segmentation via Skeleton-Aware Distance Transform