List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Liu Li, Qiang Ma, Cheng Ouyang, Zeju Li, Qingjie Meng, Weitong Zhang, Mengyun Qiao, Vanessa Kyriakopoulou, Joseph V. Hajnal, Daniel Rueckert, Bernhard Kainz
Abstract
Despite recent progress of deep learning-based medical image segmentation techniques, fully automatic results often fail to meet clinically acceptable accuracy, especially when topological constraints should be observed, e.g., closed surfaces. Although modern image segmentation methods show promising results when evaluated based on conventional metrics such as the Dice score or Intersection-over-Union, these metrics do not reflect the correctness of a segmentation in terms of a required topological genus. Existing approaches estimate and constrain the topological structure via persistent homology (PH). However, these methods are not computationally efficient as calculating PH is not differentiable. To overcome this problem, we propose a novel approach for topological constraints based on the multi-scale Euler Characteristic (EC). To mitigate computational complexity, we propose a fast formulation for the EC that can inform the learning process of arbitrary segmentation networks via topological violation maps. Topological performance is further facilitated through a corrective convolutional network block. Our experiments on two datasets show that our method can significantly improve topological correctness.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43901-8_7
SharedIt: https://rdcu.be/dnwCH
Link to the code repository
https://github.com/smilell/Topology-aware-Segmentation-using-Euler-Characteristic
Link to the dataset(s)
http://www.developingconnectome.org/
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a Euler Characteristic based model to preserve topological correctness of image segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- It proposes a new EC loss to calculate the local topological patterns.
- It proposes a new topology-aware feature synthesis network to repair topological errors.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The EC loss is still based on pixel-wise supervision. Why the EC loss is better than the previous topological loss?
- The formulation of EC = N0-N1+N2, which is used to evaluate the topological correctness of segmentation algorithms, has its limited ability to provide an accurate representation of topological correctness. This is because a single value of EC may not be sufficient to capture the complex nature of topological correctness. As a result, the effectiveness of the EC metric in real-world applications is questionable.
- It appears that the comparison methods used in a particular study are not sufficient as they do not include recent related works [1][2].
- It appears that the CREMI and dHCP datasets only focus on segment boundaries and do not provide a standard setting or compare state-of-the-art results.
[1] Hu, Xiaoling. “Structure-Aware Image Segmentation with Homotopy Warping.” Advances in Neural Information Processing Systems 35 (2022): 24046-24059. [2] Liu, Weiquan, et al. “TopoSeg: Topology-aware Segmentation for Point Clouds.” IJCAI, 2022.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
meet the requirement
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
See weakness
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper proposes a new loss and a new error correction model, however, the motivation of the method and the effectiveness of the method is not well supported.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
I hold the view of a weak rejection for several reasons. Firstly, the method contains certain limitations. Although the EC value is computed in a patchwise manner using sliding windows, a single value assigned to a particular path fails to accurately reflect its topological correctness. Additionally, its overall effectiveness remains unproven. The method has not been compared to state-of-the-art methods in the CREMI and dHCP datasets.
Review #2
- Please describe the contribution of the paper
This paper addresses the problem of incorporating topological constraints in image segmentation. The proposed method is a way of computing Euler Characteristic within a convolutional neural network. The network is designed to be differentiable, which makes the overall architecture computationally efficient.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper proposes a novel way of computing Euler Characteristic (EC) within a convolutional neural network.
- Using EC, the proposed method can find topological violations and feed the violation map back to segmentation network. As a result, it successfully incorporate topological constraints in image segmentation task.
- Comparison with the latest segmentation techniques show that the proposed method obtained better performance on the Betty error metrics.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
In the verification, the number of test cases were small (25). So it is unclear that the results is statistically significant. Side effects or limitation of enforcing topological constraints are not discussed well. I do not fully understand the following points, 1) the method tend to generate closed surface, 2) the method is applicable for tree structures.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The main idea of calculating Euler Characteristic in CNN is simple and seems easy to re-implement.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Because the main idea is simple and clear, it is interest to discuss a possible applications and impacts (e.g. surface reconstructions).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper proposes a novel computationally efficient approach to enforce topological constraints. Researchers in this field will benefit from this insight.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
(1) This paper presents a novel topology constrain utilizing the method of Euler Characteristic in image segmentation. This works also implement DL-compatible Euler Characteristic computation. (2) Leveraging the topology violation map, a topology-aware segmentation network to correct regions with topological errors. (3) experiments results demonstrate the effectiveness of the proposed method on two public datasets with different topological structure.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
For medical image segmentation, the topological correctness is important. Many work proposed topological aware constrain to solve a specific structure segmention and many utilizeing persistent homology to design loss to improve segmention performance. This work proposed a novel method to utilize the concept of Euler Characteristic to describe the topological characteristic in binary image segmentation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Compared to the PH-based methods, the proposed EC computation implemented on binary image segmentation. The PH-based methods can also apply on multi-class image segmention [1]. If EC computation in multi-class image segmentation, can the EC-based method also maintain the better efficiency?
However the topological error map rely on the ground-truth segmentation, the topology error map cannot be generated and guide to correct topology errors. If the error map can modify the prediction results，the performance of topology may be better. The comparison methods lack the PH-based methods, such as reference [1][2][3]. - Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code will be public. The experiment performed on two public datasets.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
If EC computation in multi-class image segmentation, can the EC-based method also maintain the better efficiency? However the topological error map rely on the ground-truth segmentation, the topology error map cannot be generated and guide to correct topology errors. If the error map can modify the prediction results，the performance of topology may be better. The camparison method lack some PH-based methods in medical image segmentaion. In experiment results, the statistical significance should be added.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work propose a novel model to describe the topology-aware constraint. Compared to the existed PH-based method, the new model has a real-time performance. The experiment performed on two public datasets to demonstrate the effectiveness.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
In rubuttal, the authors added the statistical significance of results and describe the difference between the proposed loss and PH-based methods. The content is convincing to me.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper proposes a Euler Characteristic based loss to preserve topology in image segmentation networks.
Strengths
- novelty of the approach
- highly relevant topic in medical image segmentation
Weaknesses:
- the proposal is restricted to boundary segmentation and not region segmentation; how can multi-class image segmentation be handled?
- the statistical significance of the results can be questioned: number of test cases is small, and no statistical significance is computed with a p-value for instance
- comparison to some state of the art methods is missing: ref 1 and 2 give by Rev1 and ref 1,2,3 of the paper itself
- the limitation of enforcing topological constraints is not well discussed.
Rebuttal
- please add statistical significance of the results
- please comment on the applicability of the EC loss, with respect to region segmentation and multi-class image segmentation
- answer to questions 1 and 2 of Rev 1
- please discuss why there is not comparison to some state of the art methods is missing: ref 1 and 2 give by Rev1 and ref 1,2,3 of the paper itself
- improve the discussion on limitation of enforcing topological constraints
Author Feedback
We thank the AC and reviewers for acknowledging the novelty of our Euler Characteristic (EC) based topology-aware segmentation.
Motivation [AC-3; R1-1] Both EC-based and persistent homology (PH)-based methods apply a pixel-wise loss term. However, ours is substantially more efficient: 30 to 46 times faster than PH. This is because (1) PH-based methods only detect and adjust a small portion of topology-related pixels, and (2) PH has O(n^3) calculation complexity. In contrast, our method corrects the entire regions with topology errors by EC visualization, being more efficient for optimization.
Sufficient representation of EC [AC-3; R1-2] As stated in the Topology-violation Detection section, our method preserves the local topological correctness by calculating EC’s patchwise via sliding windows, rather than via a single value per image. This patching enforces local topology and allows to correct fine-level topological errors. We validate the effectiveness on two real-world datasets with intricate topological structures (Fig. 2 and Tab. 1).
Comparisons to SOTA’s [AC-4; R1-3] [1] incorporates the homotopy warping. We now show that [1] is inferior to ours in all metrics on both datasets: with 88.11% and 2.495 in Dice and Betti error on dHCP, and 83.03% and 11.781 on CREMI. Their training time is 0.46 and 1.78 s/batch on both datasets, around 4.2 to 9.4 times slower than ours. [2] is not comparable because it is for point cloud data modeled by Rips-complexes. In contrast, we focus on image segmentation modeled by cubical-complexes in dense pixel grids.
[AC-4; R3-3] Instead of comparing with different PH implementations [1,2,3], we compare to the representative SOTA PH-based method [9] and also other topology-aware methods [13,22], achieving better accuracy and efficiency (Tab. 1). [2] shares the same idea as [9]: Both of them increase the lifetime for the persistent components. [1] and [3] are minor extensions of [9] towards ring-structures and multi-class segmentation. Meanwhile, [1,2,3] do not solve the main PH efficiency problem. Therefore, we only compared to the SOTA’s [9,13,22].
4.1 Applicability to region segmentation [AC-2; R2-2] Our method is applicable to various topological structures, including regions: It learns the topology directly from the GT, rather than expecting a specific topological genus. R2 comments on our method ‘tends to generate closed surfaces’ and is ‘applicable for tree structures’. We agree, this is true because the dHCP and CREMI datasets have such properties.
4.2 Beyond segmenting boundaries [AC-2; R1-4] We argue that 1. the quality of boundary segmentation is heavily subject to topology correctness and it remains an unsolved problem. 2. Existing topology-aware methods [1,2,3,9] and (1) from R1 also focus on boundary segmentation: neuron boundary, cardiac ventricle, and road boundaries. 3. As discussed in 4.1 above, our method can also be applied to other topological structures.
Multi-class segmentation [AC-2; R3-1] Following [1], ours can be extended to multi-class by calculating the EC class by class. The computation time for both EC and PH methods increases linearly with the number of categories. But our EC method is more efficient than the PH method in individual classes, the efficiency improvement is even magnified for multi-class segmentation.
Limitations [AC-5; R2-2] Enforcing topological constraints on topology-agnostic tasks only increases compute time, e.g., hepatic ascites segmentation prioritizes the volume of the segmented fluid over its topology. In addition, topology-aware methods are sensitive to labeling noise; even a few mislabeled pixels inside an object can lead to entirely wrong topologies. We will add this to discussion.
Statistical significance [AC-1; R2-1; R3-4] We conducted Wilcoxon signed-rank tests comparing our method to all other baselines, and the p-values are <0.05 across all metrics in Tab. 1. We will add this to the manuscript.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Even after rebuttal, there are mixed feelings towards this paper. The authors were able to clear up the issues raised by the reviewers, especially regarding comparison to SotA and statistical significance. Hence I recommend acceptance of this paper- even if one reviewer is still concerned with some limitations of the proposed method.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper presents an interesting segmentation method. The authors addressed most of the concerns raised by the reviewers. However, the main weakness of the proposed framework, i.e., being limited to binary segmentation has not been addressed in a convincing manner. To this M.R, opinion an extension to multi-class segmentation could not be resolved in a straight-forward manner (by calculating the EC class by class) and would possibly require handling of conflicting, neighboring regions. The authors are encouraged to extend their work in this direction.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper proposes a topologically valid segmentation by leveraging a multi-scale Euler characteristic. The reviews appreciate the task of proposing a topology-aware segmentation, an unsolved problem, but had concerns on its methodological correctness and missing evaluation with topology-aware methods.
R4, upgrading to a strong accept may sound disproportionate since this decision is based on new results of a statistical significance, which should have been provided before the deadline for fairness with the other submissions. The rebuttal may also be considered insufficient in securing the topological correctness when stating that this is preserved by calculating the Euler characteristics in image patches rather than on images.
For all these reasons, and situating the work with respect to the other submissions, the recommendation is towards Rejection.