List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Ainkaran Santhirasekaram, Karen Pinto, Mathias Winkler, Andrea Rockall, Ben Glocker
Abstract
Deep learning based methods have become the most popular approach for prostate segmentation in MRI. However, domain variations due to the complex acquisition process result in textural differences as well as imaging artefacts which significantly affects the robustness of deep learning models for prostate segmentation across multiple sites. We tackle this problem by using multiple MRI sequences to learn a set of low dimensional shape components whose combinatorially large learnt composition is capable of accounting for the entire distribution of segmentation outputs. We draw on the language of cellular sheaf theory to model compositionality driven by local and global topological correctness. In our experiments, our method significantly improves the domain generalisability of anatomical and tumour segmentation of the prostate.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43901-8_24
SharedIt: https://rdcu.be/dnwC8
Link to the code repository
https://github.com/AinkaranSanthi/A-Sheaf-Theoretic-Perspective-for-Robust-Segmentation
Link to the dataset(s)
http://medicaldecathlon.com/dataaws/
https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=21267207
Reviews
Review #1
- Please describe the contribution of the paper
This paper tackles the domain shift problem by using multiple MRI sequences to learn a set of low dimensional shape components whose combinatorially large learnt composition is capable of accounting for the entire distribution of segmentation outputs. The authors draw on the language of cellular sheaf theory to model compositionality driven by local and global topological correctness. The proposed method significantly improves the domain generalisability of anatomical and tumor segmentation of the prostate.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The problem this paper wants to address is interesting and the proposed method is novel.
-
The proposed method achieves good performance compared to the baseline methods.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
I would expect the authors to provide more implementation details to reproduce the results. Open-sourced codes are encouraged to benefit the community.
-
The method section is kind of difficult to follow. More specifically, it’s not very clear to me how the shape dictionary D is constructed and how we obtain the sampled components z.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors provide some details.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
N/A
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See both the strengths and weaknesses sections.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
I tend to keep my original score.
Review #2
- Please describe the contribution of the paper
This paper uses sheaf theory to provide a robust way to proceed to prostate images segmentations. The goal is mainly to make the algorithm able to segment prostates whatever the context behind the prostate. They aim also at preserving the topology at local and global levels. Persistent Homology is used in this approach.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
As said before, the strength of this approach is mainly the topology preservation. The final Dice attest that proceeding this way leads to better results (see the Betti number and so on). Also, the way they segment is original: dictionary learning and the use of sheaf theory are not commun in such a context.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I do not know if it is really a “weakness”, but I do not feel that sheaf theory is really used. The same for posets: we know that the parts of R^2 constitute a poset, but this fact is not really used to benefit of anything in this approach. We have relations between sheaf theory and this paper, but which theorem is used and makes the approach so robust ? Was sheaf theory really necessary to obtain these results? It is very confusing.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Equations are clear and thanks to Figure 2, all the details seem to be given for reproducibility, nothing to say.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I consider that the contribution of sheaf theory to this paper is not clear enough, I even think that sheaf theory could have been ignored to provide these results. I think that the authors should explicit which tool coming from sheaf theory is so much useful here (to preserve topology at he global and local levels and so on).
The same applies for posets. Furthermore, at page 3, the authors speak about a “poset”, when it seems sufficient to talk about total orders, it is confusing.
I also wonder about the relation between the proposed approach and attention models which develop filter specific to the region of interest; perhaps it coud be nice to add a section to compare both approaches.
“noisy topological descriptor” -> do the authors mean “ill-defined”?
Section 2.2 : local homology exists, and it is local topology, so saying that homology is global is not completely true.
“a topological space, Y” -> it is a set but not a topological space since the authors did not supply the domain with any collection of open sets.
Many references about the works of Edelsbrunner, Harr, Tierny, and peers miss relatively to persistent homology.
Section 3 : the literature is rich -> the authors cite only two papers, so this phrase is not convincing. Please add other references.
About the notations relative to the contrastive loss: z^1 -> z_1 and so on, to avoid the reader to interpret that z^2 is equal to z \times z.
stop gradients: do the authors speak about early stopping? or some parameters are frozen? Please develop.
page 5: “not one-hot encoder” seems not to be relevant there, please remove or develop.
P(R^2), the set of parts of R^2, is a poset, but no poset theory seems necessary to say that “(0,0) < (0,1)” and so on. In fact, I do not understand why it is important in this context.
The authors obtain a dice of 0.51, which is the best score in Table 2. Why is this problem about the prostate so hard? (It is an open question).
Did some approach based on super resolution has been used in this context? I see that segmentations in Figure 3 are not so good (it is another open question).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I chose weak reject because I consider that sheaf theory keyword is used like a buzz word when in practice, it is not really used in the paper.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors present a novel way to segment prostate regions and tumours on datasets with a domain shift by focussing on low dimensional shape components. An encoder network is trained to project the image into a low dimensional, shape equivariant and texture invariant representation which is utilised to train a shape dictionary. Based on cellular sheaves theory, the shapes are composed to create anatomically plausible segmentations.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- (To my knowledge) this is the first application of cellular sheaves in the context of medical image segmentation. By focussing the network on shape features, changes in texture will be less detrimental when applied to a new domain and thus should aid when deploying the model on datasets with a domain shift.
- The evaluation is performed on several datasets, including two public datasets, which shows general performance benefits of the proposed method when applied to prostate and prostate tumour segmentation. Furthermore, several baselines and metrics are reported to provide a through comparison against other augmentation based domain generalisation methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The presented method is only applied in the context prostate and prostate tumour segmentation while the concept should be applicable to a broader range of segmentation tasks. Extending the experiments to other tasks, such as Brain Tumours, would demonstrate the generalisability.
- The setup of the baseline methods and experiments needs to be explained in more details since not all aspects are entirely clear: a. Were all methods only trained with the T2 images? If so, reporting numbers with both modalities as an additional upper bound might be necessary and should be discussed as a potential shortcoming of the presented method (since it requires T2 + ADC during training) b. There is one statement “all experiment were validated on 10 samples” which needs additional elaboration, were those samples used as validation and if so were they selected form the training or testing sets? c. How were the standard deviations generated for the evaluation metrics? d. Did the cropping to a fixed shape of 256 x 256 x 24 degrade the performance for cases with more slices? Is this a requirement for the method to work correctly?
- (minor) The manuscript is quite math heavy which may make it difficult to understand by readers without a strong mathematical background.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The presented results should be easily reproducible since the source code will be made publicly available and one of the experiments is based solely on public data.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The authors have developed a very interesting method and applied it to several datasets to demonstrate its robustness. Since the concept should be applicable to other problems as well, extending it to other datasets might be the most promising direction to further strengthen the provided results. Some additional details should be added to the manuscript to clarify the previously raised points (in the weaknesses section).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The presented work proposed a novel way to learn more shape specific features and demonstrated the usefulness on several datasets. The selection of the baselines and provided evaluation (given the current information without clarifying some of the previously mentioned points in the weaknesses section) are sound and well thought out.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Additional information regarding the experimental setup (used modalities of baselines and validation set) were provided by the authors during the rebuttal and show a sound evaluation scheme. In combination with the code release and the presence of public datasets the results of the paper should be reproducible in the future. Due to the limited number of target structures (only prostate) and smaller missing details (e.g. the computation of the standard deviations) the review score remains unchanged.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper uses computational topology to provide robust prostate image segmentation.
The reviewers have mixed feelings about this paper. On the one hand, they find the paper well written, and appreciate the quest to preserve topology. On the other hand, a number of concerns are also brought up, which need to be addressed in the rebuttal:
- The language of sheaf theory and posets makes the work less accessible and likely reduces the impact of the work, but at the same time might make it seem more novel than it actually is. Could the authors please comment on whether it is necessary (cf reviewer 2)?
- The related work is lacking
- There are a number of concerns regarding clarity and detail (reviewers 1 and 2)
- There are a number of concerns regarding the experimental setup (see in particular reviewer 3)
Author Feedback
We thank the reviewers for their constructive feedback. R1/R2: Why are we using sheaf theory? Shapes are sampled independently from the shape dictionary, and this can lead to anatomically implausible segmentation maps. We tackle this issue by modelling the connectivity of patches/shapes in an image to provide a connectivity-based loss function. Specifically, we track how topological features changes as we glue patches in a specific order to compose the segmentation map. We glue each patch adjacent to the previous patch to assess the topological relationship between patches which is not possible if patches are glued together in a random order. Hence, we model and compare topological patch connectivity between the output and label via inclusion maps in a well-defined manner. This cannot be defined by a total ordered set but a poset because the relations between elements are inclusion maps. For example, there is no inclusion map between elements corresponding to patch 1 and patch 2 but between patch 1 and patch 1 + patch 2. Next, for each poset element which corresponds to a patch or connected patches, we calculate the persistent homology to get a vector space (topological feature). Therefore, in category theory we have a functor from the category of elements and partial relations in the poset to the category of vector spaces over a certain field. This is defined as a cellular sheaf of vector spaces on a poset which naturally arises in our approach and provides a mathematically precise justification for using cellular sheaves in our method. In order to be in keeping with the context of the paper, we changed the title of our paper to; ‘A sheaf theoretic perspective for robust prostate segmentation’. By approaching the composition of segmentation maps through this lens, we have significantly improved the robustness of segmentation models. There is a trend towards using category theory in deep learning and we hope this approach will provide the MICCAI community with a fresh perspective for building robust segmentation models. R1/R2: Missing related work. Related to our work, [1] provides a sheaf theoretic construction of a shape space. Further related work on sheaves and persistent homology is included in the revised paper as suggested by R2. R1/R2: Clarity and missing details. Clarity: All code will be released upon acceptance. We improved the clarity of the methods section by providing better motivation behind the use of cellular sheaves to better model the connectivity/composition of segmentation maps. How is the codebook learnt? The codebook is learnt in an identical fashion to the VQ-VAE. Firstly, the continuous latent space is divided into vectors of size c x 1 x 1 x 1 (c = number of channels). Next, each vector replaces itself with the nearest codebook vector by Euclidean distance. Vector quantisation minimises the distance between each latent vector and the codebook vector which it is replaced by. What are stop gradients? Stop gradients provides a way to not compute gradients with respect to some variables during back-propagation. R3: Experimental setup queries. Further experiments: We agree this method can be extended to other datasets and will constitute future work. This method can also easily be used to construct structural priors for test-time adaptation. Clarification of comparison with other methods: Our method does not use ADC maps for inference. The other methods are trained and tested on both T2 and ADC images as input. This makes our method even more convincing. Validation and pre-processing clarification: We used 10 samples in the validation set which is acquired from the training set. Finally, we did not notice a degradation in performance by cropping to a fixed shape of 256x256x24 (not a strict requirement). We used 24 slices because no training images had more than 24 slices.
- Arya S, Curry J, Mukherjee S. A Sheaf-Theoretic Construction of Shape Space. arXiv preprint arXiv:2204.09020. 2022 Apr 19
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This was a borderline paper, where the reviewers had a number of concerns:
- The language of sheaf theory and posets makes the work less accessible and likely reduces the impact of the work, but at the same time might make it seem more novel than it actually is. Could the authors please comment on whether it is necessary (cf reviewer 2)?
- The related work was lacking
- There are a number of concerns regarding clarity and detail (reviewers 1 and 2)
- There are a number of concerns regarding the experimental setup (see in particular reviewer 3)
While the authors addressed some of these concerns in their rebuttal, some concerns remain regarding in particular the use of very small datasets and missing explanations of how metrics were computed. As a result, the paper remains borderline, but just short of recommending acceptance.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Following the first round of review, there were missing information and justification regarding the authors contribution. In the rebuttal, the authors were convincing in explaining why and how sheaf theory was an essential component to their methodology, and could clear up the clarity issues, as requested per the reviewers.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes to segment prostatic images using cellular sheaf theory on topological correctness. The reviews appreciated the originality of the approach but had major concerns on its understandability, notably on the direct contribution of sheaf theory. The rebuttal has provided useful clarification on the motivation and use of sheaf theory, notably on how posets are exploited in a vector space of topological feature.
While the methodology may still benefit from further explanation and clarification, as well as providing more exhaustive experiments, the work still constitutes an original approach with a reasonable validation, worth publishing in the miccai community.
For all these reasons, and situating the work with respect to the other submissions, the recommendation is towards Acceptance.