List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yinglin Zhang, Ruiling Xi, Huazhu Fu, Dave Towey, RuiBin Bai, Risa Higashita, Jiang Liu
Abstract
Robust and accurate segmentation for elongated physiological structures is challenging, especially in the ambiguous region, such as the corneal endothelium microscope image with uneven illumination or the fundus image with disease interference. In this paper, we present a spatial and scale uncertainty-aware network (SSU-Net) that fully uses both spatial and scale uncertainty to highlight ambiguous regions and integrate hierarchical structure contexts. First, we estimate epistemic and aleatoric spatial uncertainty maps using Monte Carlo dropout to approximate Bayesian networks. Based on these spatial uncertainty maps, we propose the gated soft uncertainty-aware (GSUA) module to guide the model to focus on ambiguous regions. Second, we extract the uncertainty under different scales and propose the multi-scale uncertainty-aware (MSUA) fusion module to integrate structure contexts from hierarchical predictions, strengthening the final prediction. Finally, we visualize the uncertainty map of final prediction, providing interpretability for segmentation results. Experiment results show that the SSU-Net performs best on cornea endothelial cell and retinal vessel segmentation tasks. Moreover, compared with counterpart uncertainty-based methods, SSU-Net is more accurate and robust.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43901-8_31
SharedIt: https://rdcu.be/dnwDt
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
– proposed an Gated Soft Uncertainty-aware Moduleand a Multi-scale Uncertainty-aware Module, which have improved the performance of U-Net for elongated physiological structure segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
– A novel semgentation structure based on uncertainty. The methodology looks sound. – A strong evalution. The method has been evaluated on three public available dataset.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
– The quantative improvment is marginal to the SOTA methods. – The authors deleted the lines of Authors and Affiliations, which would be a violation of the anonymity rule and typesetting rules.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
– The datasets are public available – The algorithm is clearly presented and easy to implement. The code will not be released referring to the reproducibility table.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Objective functions are built on the ground truth and multiple outputs of the network. I wonder whether all of these constraints are useful. Please conduct experiments to verify the design.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
– The modeling is novel and complete. – The algorithm is technically sound. – The improment is marginal.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proposes a method of focusing the information of the feature map in consideration of uncertainty. Robustness has been validated on benchmark datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper deals with uncertainty, an important issue in the medical field. They propose two uncertainty-based attention modules. The proposed GSUA module measures spatial uncertainty and uses it to focus more on uncertainty regions in the feature map in the next layer. The MSUA module was proposed to adaptively fuse information of all scales in consideration of uncertainty, considering that there are differences in information depending on the scale of the feature map.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Since the proposed method is based on sampling for uncertainty estimation, a decrease in speed occurs in training and inference.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I believe that it is possible to reproduce.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The proposed method is believed to have novelty and is legibly and comprehensively described. The role of the proposed two modules is interpreted through the ablation study. It would be better if the author provided an explanation and analysis of the role of the two parallel poolings [Psi_avg, psi_max] in GSUA.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed uncertainty-based modules are considered novel, and experiments on benchmark datasets show performance improvement.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
This paper proposed a spatial and scale uncertainty-aware network (SSU-Net) for elongated physiological structure segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) This paper proposes a gated soft uncertainty-aware (GSUA) module to adaptively highlight ambiguous areas based on spatial uncertainty maps. 2) This paper extracts the uncertainty under different scales and propose the multi-scale uncertainty-aware (MSUA) fusion module to integrate hierarchical predictions for enhancing the final segmentation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- In Fig. 1(a), there are S2 and S3, but not S1.
- How exactly are y ̃ and v in Fig. 1(b) obtained? Is the network already trained? If so, how was it trained?
- You have done lots of work on uncertainty aware, including using Bayesian network to obtain u_e and u_a, and designing the GSUA module. However, it is not obvious from the results that this module is useful.
- The MSUA module is essentially more like an ensemble learning, and it is suggested to add a set of ablation experiments, i.e., a direct max operation on y ̃_1,y ̃_2,y ̃_3 to demonstrate the role of uncertainty perception (Sigmoid+Multiplication+Add).
- In the testing phase, you infer 16 times. Are the network parameters the same for each inference? How do you get the 16 different results?
- What is your backbone? Why are the results of backbone already better than UNet, D-LinkNet, AttentionUNet, TransUNet and BayesianNet? Is it fair to make a comparison in this case?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This work is reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-Please provide the details of the experimental. -Give the more results to verify the effectiveness of the provided module.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
An interesting work
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This work proposes novel components for incorporating uncertainty into segmentation, which is an interesting and important area. The results are evaluated well. It would be useful to expand the results to demonstrate the contributions of the individual components to the results and some more explanation of the experimentation is warranted. The computational and time complexity should also be discussed, in relation to competing methods.
Author Feedback
reviewer1:
- In Fig. 1(a), there are S2 and S3, but not S1.
- How exactly are y ̃ and v in Fig. 1(b) obtained? Is the network already trained? If so, how was it trained?
- You have done lots of work on uncertainty aware, including using Bayesian network to obtain u_e and u_a, and designing the GSUA module. However, it is not obvious from the results that this module is useful.
- The MSUA module is essentially more like an ensemble learning, and it is suggested to add a set of ablation experiments, i.e., a direct max operation on y ̃_1,y ̃_2,y ̃_3 to demonstrate the role of uncertainty perception (Sigmoid+Multiplication+Add).
- In the testing phase, you infer 16 times. Are the network parameters the same for each inference? How do you get the 16 different results?
- What is your backbone? Why are the results of backbone already better than UNet, D-LinkNet, AttentionUNet, TransUNet and BayesianNet? Is it fair to make a comparison in this case?
To reviewer1:
- We apologize for the confusion. In fact, the convolution operation acts as a segmentation head at scale 1. We will clarify the concept and supplement more details in the extended work.
- Yes, the network is already trained. Due to the limitation of paper length, we do not describe this model in detail, but give the reference. Please refer to [7] for more details on the model architecture and training process.
- The GSUA module is a crucial component for performance and robustness across a range of situations. In the ablation study, we observed that the benefit of GSUA appeared to be less than MSUA. This may be due to the fact that, in the context of our TM-EM3000 corneal endothelium cell segmentation task, scale is a more important factor than space. We will take this feedback into account when designing future experiments. Thank you for your comment.
- We will take this feedback into account when designing future ablation experiments.
- Since we add droupout layer after each UNet block, we can obtain different output for each inference.
- The backbone of our model is a UNet with a dropout layer after each block. The primary difference between the backbone and the BayesianNet is the placement of the dropout layers. The BayesianNet only adds a dropout layer in the encoder, while the backbone uses dropout layers throughout the entire network.
To reviewer2:
- We appreciate your comment. While we acknowledge that the improvement we observed may be marginal in some cases, we would like to point out that our proposed SSU-Net consistently achieved the best performance and robustness across various situations, for example, experiments on the FIVES retinal vessel segmentation task and the Rodrep dataset. On the FIVES task, SSU-Net outperformed UNet by increasing the Dice score by 10.08%, mIoU by 7.29%, and mAcc by 7.51%. Although Lee’s method is comparable to ours on this task, on the Rodrep dataset, SSU-Net performed considerably better than Lee’s uncertainty method, improving the Dice score by 4.98%, mIoU by 3.48%, and mAcc by 3.14%. We believe that these results demonstrate the effectiveness of our proposed method and its potential for wider applicability beyond the specific task of corneal endothelial cell segmentation.
To reviewer3:
- Many thanks for the comment. We acknowledge the issue you have raised and plan to address it in future work to improve the efficiency of our proposed method.