Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Siwoo Nam, Jaehoon Jeong, Miguel Luna, Philip Chikontwe, Sang Hyun Park

Abstract

Recently, weakly supervised nuclei segmentation methods using only points are gaining attention, as they can ease the tedious labeling process. However, most methods often fail to separate adjacent nuclei and are particularly sensitive to point annotations that deviate from the center of nuclei, resulting in lower accuracy. In this study, we propose a novel weakly supervised method to effectively distinguish adjacent nuclei, and maintain robustness regardless of point label deviation. We detect and segment nuclei by combining a binary segmentation module, an offset regression module and a center detection module to determine foreground pixels, delineate boundaries and identify instances. In training, we first generate pseudo binary masks using geodesic distance-based Voronoi diagrams and k-means clustering. Next, segmentation predictions are used to repeatedly generate pseudo offset maps that indicate the most likely nuclei center. Finally, an Expectation Maximization (EM) based process iteratively refines intial point labels based on the offset map predictions to fine-tune our framework. Experimental results show that our model consistently outperforms state-of-the-art methods on public datasets regardless of the point annotations accuracy.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43907-0_51

SharedIt: https://rdcu.be/dnwdy

Link to the code repository

N/A

Link to the dataset(s)

https://monusac-2020.grand-challenge.org/

https://drive.google.com/drive/folders/1l55cv3DuY-f7-JotDN7N5nbNnjbLWchK

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper proposes a weakly-supervised nuclei segmentation method that estimates each nuclei region given nuclei center positions as weakly-supervised data. An issue of current weakly-supervised methods is that annotated nuclei centers may be shifted from the ground truth, and thus, current methods often fail to capture nuclei shape information; resulting in inadequate boundary delineation between adjacent nuclei. To solve this issue, this paper introduces an EM algorithm-based refinement process to encourage model robustness on center-shifted point labels. The experimental results show the effectiveness of the proposed method.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    • The idea of point refinement using a shape-guided offset map is interesting and looks reasonable.

    • The proposed method achieved better performance than comparative methods.

    • Ablation study shows the effectiveness of each module.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    • There are some unambiguous points, in particular, descriptions of math. I described the details on the detailed comment. Please clarify these points.

    • They follow the framework of related works for the proposed method, which uses color clustering and Volonoi. The difference is that they alternatively optimize both model parameters and point labels instead of the standard fixed-point label-based model optimization.

    • One of the comparative methods [21] was designed for `partial’ point annotation but points are not fully annotated. Did you modify the method to apply it for fully annotated centroid points? Otherwise, it is a bit unfair.

    • Heatmap that represents a centroid point as Gaussian distribution is a more robust method [*] for centroid shift problems. I’m wondering about the advantage of the proposed method compared to the method that applied Gaussian-based centroid heatmap to the backbone method, which uses Volonoi and color clustering.

    [*] K. Nishimura et al., Weakly supervised cell instance segmentation under various conditions. Medical Image Analysis, 2021.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    If the code will be provided after accept, it is O.K.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    • In Eq. 1, the number of (’ and )’ is different. In addition, the value of I is decided by the coordinates (x, y). Please check it.

    • There is no definition of `\hat{\mathcal{C}}_i’ in Eq. 1. What is it? I guess it may be the region of $i$-th nuclei. How did you obtain the region from the heatmap? To understand the method, this is important.

    • In Figure 1, \mathcal{O}’ has a color. Does \mathcal{O}’ have multiple channels or a single channel? If it contains multiple channels, which channel is used for Eq. 1?

    • Similarly, `\mathcal{K}’ is a color in Figure 1. How to compute Eq. 2?

    • In general, `*’ indicates a convolution operation mathematically. Did you use this symbol for a product operation? If so, please use it consistently. In other parts, the description is not consistent.

    • In Eq. 2, L_v and L_k have the same structure. Basically, the Volonoi boundaries \mathcal{V}’ is used for penalty and \mathcal{K}’ may be used as the pseudo labels of the foreground regions. Is it correct?

    • The definition of \mathcal{O}’ is unclear. In the paper, we define an offset vector O(x, y) that indicates the displacement of a point (x, y) to the center of its corresponding nucleus.’ It is computed for any (x, y)? If so, it does not fit to Figure 1. The example in Figure 1 indicates `\mathcal{B} \times \matcal{O}’?

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While there are some unclear points, the idea of point refinement using a shape-guided offset map is interesting and looks reasonable. Therefore, currently, my decision is slightly above the borderline. If the issues are clear after rebuttal, the decision may be upgraded.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    5

  • [Post rebuttal] Please justify your decision

    Some points of my concerns have been addressed. However, the following rebuttal is incomplete. My concern is that since [21] was originally designed for learning from partial labels and not for supervised learning, i.e., [21] was not designed to maximize the performance in the problem setup. So, I consider that is a bit unfair. (R1) Unfair comparisons with [21]: In fact, our method and [21] use the entire set of point labels. There is no use of a partial set of points in [21]. While it remains such a concern, I feel the merits of the method slightly weigh over the weakness. Thus, I keep my decision `weak accept’.



Review #2

  • Please describe the contribution of the paper

    They study the problem of imperfect annotations in deep-learning-based cell segmentation using only point annotations during training.

    To address this problem, they propose a deep learning pipeline using 3 main modules (this part is not novel):

    • prediction of the binary segmentation of cells
    • prediction of the centers of cells
    • prediction of the offset

    Contributions:

    • use Geodesic distance to generate pseudo segmentation masks from point annotations (sec 2.1)
    • a refinement process based on an expectation maximization algorithm (sec 2.2). The overall optimization problem alternate between segmentation model optimization and point annotations refinement optimization
    • improved robustness and segmentation accuracy for the proposed pipeline as compared to previous methods
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • outperform three published methods on two public datasets
    • the paper is overall clearly written and organized
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • the method is incremental as compared to the literature
    • some key implementation details are missing in the experiments section
    • the ablation study is incomplete
    • the reproducibility needs to be improved
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
    • the code is not available
    • the method used to set the hyperparameter values is not specified. In particular for the hyperparameters: 20 epochs before using the offset map loss, learning rate, Gaussian kernel size r, weight decay, threshold, weights of the three loss functions
    • the method to obtain the shifted center point annotations is not described
  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    1. Incremental methodological contributions. The two main methodological contributions are:
      • use Geodesic distance to generate pseudo segmentation masks from point annotations (sec 2.1)
      • a refinement process based on an expectation maximization algorithm (sec 2.2). However, geodesic distance and EM are commonly used tools in the medical image computing literature and consist of only a small part of the segmentation pipeline. This being said, I think this work could be acceptable for MICCAI if the experimental section is polished (see more details below).
    2. Experiments
      • The ablation study reported in Table 2 is incomplete. In Table 2 the baseline configuration with neither offset, nor geodesic distance and refinement used is missing.
      • How long does the alternate optimization for the proposed EM-based training last? Can you compare to other models without point refinement? A limitation of the alternative optimization schemes like EM is that they can be slow. If the proposed training method with point refinement is slower than training of baseline models please mention it in the limitations
      • There is no direct comparison of the geodesic distance with the euclidean distance to generate pseudo-segmentations. The use of the geodesic distance to generate the pseudo segmentations used for training from point annotation is one of the main contributions. Therefore, this evaluation would support the effectiveness of the method.
      • no statistical tests has been performed
      • no measure of dispersion of the results (e.g. standard deviation) has been reported
    3. Critical implementation details missing.
      • It is not explained how the point annotations and the annotation shifts were simulated for CPM17 and MoNuSeg. However, this is a key step of the experimental design and should be described. I also have not found the unit used for the point shifts (Table 1 and 2). Is it in terms of number of pixels?
      • How many epochs for each iteration on the segmentation model in the EM algorithm?
    4. Others points that I found unclear.
      • What is bold in the tables 1 and 2? If it is the best value, please put all the best values in bold (see Table 2)
    5. Minor.
      • there is one missing right parenthesis in equation 1
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    My recommendation is based on the major strengths and weaknesses listed above

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    5

  • [Post rebuttal] Please justify your decision

    The authors have addressed most of my concerns on the experiments.

    As promised in my review, I therefore upgrade my recommendation to weak accept.

    I still think that the method itself is a bit incremental as compared to previous work. However, the gain of performance and ablations studies are satisfying.



Review #4

  • Please describe the contribution of the paper

    This paper proposes a novel weakly supervised method for nuclei instance segmentation based on low-precision point annotation. The key contribution is the use of geodesic distance-based Voronoi diagrams and k-means clustering to generate pseudo binary segmentation labels, which are then used to train a binary segmentation module, an offset regression module, and a center detection module. The paper shows that this approach can effectively distinguish adjacent nuclei and maintain robustness regardless of point label deviation. The authors also introduce an Expectation Maximization algorithm to refine the initial point labels using pseudo binary masks. The experimental results demonstrate the superiority of this method compared to other weakly supervised and fully supervised methods in terms of both accuracy and efficiency. This paper provides a valuable contribution to the field of nuclei instance segmentation with low-precision point annotation.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1.Novel approach for weakly supervised nuclei instance segmentation: The paper presents a novel approach for nuclei instance segmentation based on low-precision point annotation. The method effectively distinguishes adjacent nuclei and maintains robustness regardless of point label deviation. This is a significant contribution to the field, as low-precision point annotation is often encountered in nucleus segmentation annotations, and this paper proposes an essential solution to address this problem. 2.Geodesic distance based Voronoi diagrams and k-means clustering: The authors propose to use geodesic distance based Voronoi diagrams and k-means clustering to generate pseudo binary segmentation labels. They show the effectiveness of Geodesic distance in Fig. 2, and the results demonstrates that geodesic distance does work better than Euclidean distance. 3.Strong experimental results: The authors show the superiority of their method by multiple experiments. They compare their method to state-of-the-art weakly supervised and demonstrate that their approach achieves better results. The experiments also show that the proposed method is robust to variations in point annotation precision and is effective in segmenting overlapping nuclei.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1.The clarity and organization needs to be improved: The paper lacks a clear structure, and it is not easy to follow the authors’ thought process. The authors should consider reorganizing the paper, perhaps by using a more logical and structured approach, to make it easier for readers to understand. 2.Better organization of the figures should be used: Although the paper includes many figures to illustrate the authors’ approach, the figures are not always well-organized, and some of them lack clear captions or explanations. For example, Fig. 3 could benefit from a general caption before the captions of each image to indicate its purpose. Additionally, more descriptions under the figures could help readers understand what the figures are trying to convey. 3.Improve the pipeline illustration: The paper’s pipeline illustration, which is intended to summarize the authors’ approach, is not clear and could be confusing for readers. The authors should consider revising the illustration to provide a more comprehensive overview of their approach, perhaps by including more detailed explanations or additional diagrams.

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The authors have indicated that they will release both the training code and evaluation code, which is a positive step towards ensuring reproducibility of the results presented in the paper.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    1.Figure organization: The authors could improve the organization of their figures by adding a general caption for each figure that provides context and purpose. Additionally, the authors could add more descriptive text below each figure to explain the key findings or takeaways. This would make it easier for readers to understand the figures and their relevance to the paper’s main contributions. 2.Pipeline illustration: The pipeline illustration in the paper could benefit from more clarity. The authors could consider re-organize the pipeline illustration as the current version seems a bit messy. 3.Clarity and organization: The paper’s clarity and organization could be improved. The authors could consider reorganizing their text and figures to better present their methodology and results.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    While the clarity and organization of the paper could be improved, the paper’s contribution of addressing the issue of imperfect annotation in nuclei instance segmentation valuable. The proposed method, which uses geodesic distance based Voronoi diagrams and k-means clustering, shows promising results in handling low-precision point annotation. This paper provides an essential solution to a common problem in the field of nuclei segmentation. The experiments conducted in the paper are thorough, showing the superiority of the proposed method. Despite the unclear figures and pipeline illustration, the novelty of the approach and its potential impact on the field justify a weak accept recommendation.

  • Reviewer confidence

    Somewhat confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposes a weakly-supervised nuclei segmentation method that estimates each nuclei region given nuclei center positions as weakly-supervised data. The experimental results show the effectiveness of the proposed method. The three reviewers also affirmed this is an interesting work. The paper will furtherly be improved. The issues include adding the descriptions of math, adding the related work, adding the ablation study, adding the experimental implementation details, and some other details mentioned by reviewers. Please address these concerns in the final version.



Author Feedback

Summary: R1(WA), R2(WR), R4(WA). We thank the reviewers for their valuable and encouraging feedback! We are pleased they find our work interesting(R1), well organized(R2) and novel(R4). We clarify the main concerns below.

(R2) No statistical tests performed: We performed t-test’s between Ours and SPNIEN, MixedAnno, and MIDL. On CPM dataset, Ours significantly outperforms all methods (p-values<0.05) with similar results on Monuseg, except for MixedAnno. We also measured dispersion(std) and will include results in Tab.1.

(R2) EM optimization time: EM starts at epoch 30 and repeats every epoch. In Tab.2, we compared to other models without refinement. Though EM introduces increased training(~1.2x), other methods often rely on a two-stage approach, whereas Ours is end-to-end, allowing for simpler training. In fact, training time is not a major concern, as testing time is more crucial and refinement is absent in inference. Ours requires 72sec on CPM compared to other methods (2150, 67, 66.sec) during testing.

(R2) Implementation details missing: The unit of the point shift is pixel. To create shifted points, we construct a donut-shaped mask radius with shift range and randomly select one of the points that overlap with the nucleus region. We will add it to the paper.

(R1) [] seems robust for point shift problems: While Gaussian filtering is used in Ours & [] for foreground expansion, our work introduces a refinement process and an offset correction strategy making our model robust to point shifts. We compared with [] but it failed in the point shifted setting as it was designed for weak-sup. nuclei segmentation. [] K. Nishimura et al., Weakly supervised cell instance segmentation under various conditions. MedIA, 2021.

(R1) How \mathcal{K}’ and \mathcal{V}’ are used in Eq2?: \mathcal{V}’ has foreground, background pixels, and \mathcal{K}’ additionally has ignore pixels. Only foreground and background pixels are used to compute L_v and L_k. \mathcal{V}’ provides a small number of high confidence pixels. \mathcal{K}’ employs a large set of points but with lower confidence.

(R1) How \mathcal{O}’ is defined?: To clarify Eq4, we changed ‘\hat{B}(x,y)\times\mathcal{O}(x,y)’ to ‘\mathcal{O}(x,y)’ that denotes the offset of a point (x,y) to the center of its corresponding nucleus. Multiplying ‘\mathcal{O’}’ with ‘\hat{\mathcal{B}}’, we get \mathcal{O}’ which is a 2-channel (x,y) offset map only on nuclei pixels. In Fig1 the Hue represents the angle of the vector.

(R1) How \mathcal{C}_i’ is used in Eq1?: To clarify Eq1, we replaced ‘\hat{\mathcal{C}_i}’ with ‘(x_{\hat{\mathcal{C}}_i},y_{\hat{\mathcal{C}}_i})’. In inference, \hat{\mathcal{C}}i’ is used to identify nuclei, one for each predicted centroid heatmap. Thresholding `\mathcal{C}_i’, we obtain connected components, find the center of the components and defined it as nucleus center (x{\hat{\mathcal{C}}i}, y{\hat{\mathcal{C}}i}). Then, for every pixel, after computing (O^(x,y) + (x,y)), pixels are assigned to the closest nuclei center (x{\hat{\mathcal{C}}i}, y{\hat{\mathcal{C}}_i}).

(R2) Incomplete ablations and geodesic vs Euclidean distance: In Tab.2, note that results w/o geodesic(x) denote using euclidean distance to obtain `\mathcal{K}’ and Fig.2(c,d) highlights the benefit. Regarding missing lowerbound (w/o {offset,geo,refine}), we have evaluated all settings and will update the Table, including related text (Ablation studies).

(R1) Unfair comparisons with [21]: In fact, our method and [21] use the entire set of point labels. There is no use of a partial set of points in [21].

(R1, R4) Error in equations, Organization & Clarity: We will fix the typo in Eq1, and unclear ‘*’ symbol. Fig.3’s caption will include a general description and will improve overall clarity in related sections.



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    In this paper, the use of a shape-guided offset map to achieve point refinement is interesting. They use Geodesic distance to generate pseduo masks from point annotations. However, reviewers think the clarity and organization could be improved. Besides, ablation study, implementation details and mathematical expression details need to be provided. In their rebuttal and supplementary, authors explain how they create shifted points and add more details of mathematical expression. A new figure of ablation study is provided.



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    All reviewers confirmed the merits of the manuscript (three weak accept). I do agree with reviewers that the merits of this paper slightly over weakness and may be worthy being discussed by the MICCAI community. I hope the reviewers’ comments can help to improve the quality of final version by including more details and ablation studies.



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The paper proposes an end-to-end weakly supervised segmentation model for accurate identification and nuclei segmentation. The authors utilized geodesic distance to delineate the boundary between adjacent nuclei and introduce an EM algorithm to encourage the model robustness. All reviewer appreciated the presented method and the strong experimental results however, they raised some weaknesses about comparisons with other methods such as [21], some missing experimental details and some unclear sections. The authors addressed the raised from the reviewer’s comments addressing most of the raised points from the reviewers. The metareviewer even if he/ she agrees with R1 and the comparison with [21], he/ she thinks that the method is interesting and could have an impact for the community.



back to top