List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yanzhou Su, Yiqing Shen, Jin Ye, Junjun He, Jian Cheng
Abstract
Accurate segmentation of polyps is a crucial step in efficient diagnosis for colorectal cancer during screening procedures. The prevalent UNet-like encoder-decoder frameworks are commonly employed, due to their capability of capturing multi-scale contextual information efficiently. However, two major limitations hinder network achieving effective feature propogation and aggregation. Firstly, the skip connection only transmits a single scale feature to the decoder, which can result in limited feature representation. Secondly, the features are transmitted without any information filter, which is inefficient for performing feature fusion at the decoder. To address these limitations, we propose a novel feature enhancement network that leverages feature propagation enhancement and feature aggregation enhancement modules for more efficient feature fusion and multi-scale feature propagation. Specifically, the feature propagation enhancement module transmits full stage features from the encoder to the decoder, while the feature aggregation enhancement module performs feature fusion with gate mechanisms, allowing for more effective information filtering. The multi-scale feature aggregation module provides rich multi-scale semantic information to the decoder, further enhancing the network’s performance. Extensive evaluations on five datasets demonstrate the effectiveness of our method, particularly on challenging datasets such as CVC-ColonDB and ETIS, where it can outperform the previous state-of-the-art models by a significant margin (3\%) in terms of mIoU and mDice.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43904-9_61
SharedIt: https://rdcu.be/dnwH7
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposes a novel feature enhancement network that improves feature propagation and aggregation in polyp segmentation by introducing novel feature propagation modules that can be employed in a encoder-decoder network. The modules allow multi-scale feature propagation thus enhancing the network’s performance. The paper reports extensive evaluations on five polyp segmeneation datasets and demonstrates the effectiveness of the proposed method, particularly on challenging datasets such as CVC-ColonDB and ETIS, where it outperforms the previous state-of-the-art models by a significant margin (3%) in terms of mIoU and mDice. Overall, the paper presents a valuable contribution to the field of polyp segmentation and the proposed modules are generally applicable to other medical imaging tasks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novelties: The main novel contribution are the 3 modules that improves upon the standard skip connection in an encoder-decoder network. Simplicity and extendability: The proposed modules are simple and easy to understand, they can also be easily deployed to existing encoder-decoder frameworks. Performance gain and validation: The authors also show that the proposed method, overall works better on several datasets that current best standard approaches.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Missing and vague details: The overall structure of the paper is good and easy to follow. There are however some minor details that I think should be addressed by the authors. (i) The FPE example in Fig. 2 (a) seems odd, is p here p4? It would simpler to pick one specific example, maybe the one before the example shown in (c). (ii) How are the inputs (C1’, C2’, C3’) to the CU layer in Fig. 2 (c) merged? Are they summed but why are they not summed using the element-wise summation layer prior to the CU? (iii) How is the reference feature X generated in practice, is it always Y? I think adding a few more sentences how X is generated would help. (iv) “… followed by a CU to produce the final output feature C1.” Is the final output not O1? (v) In the Multi-Scale Aggregation Module section, there is no explanation how and why there exists supervision after the MAE as show in Fig. 2 (d). Is the extra supervision necessary and beneficial and why is it not discussed?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Authors have not referred to the code in the paper but will presumably release the code post-acceptance.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Overall, the paper is well written and easy to follow. The main idea is also novel and easy to deploy to other medical segmentation tasks. I have outlined several suggestions in the Weaknesses section but I do have one minor suggestion. In the main results Table i.e. Table 1, I think it would be beneficial to highlight the top 2 best results. I believe the proposed method is consistently within the top 2 best results across the 5 datasets tested and should be highlighted as such.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, the paper proposed an easy to follow idea that can be easily adapted to various segmentation tasks, not just for polyp segmentation. The experiments ran, both comparison to other existing methods and the ablation study, are valid and justify the proposed method. There are however some minor details, if addressed, would change the paper from being just a good enough paper to a great one.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper proposed three modules - Feature Propagation Enhancement (FPE), Feature Aggregation Enhancement (FAE), and Multi-Scale Aggregation (MSA) to improve the performance of UNet’s encoder-decoder architecture. Experimental results demonstrated the outstanding performance of the proposed segmentation model.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper analyses the limitation of the existing UnNet-like architecture and proposed feature enhance modules to address the limitation. The paper also provided a strong evaluation of the proposed model using 5 datasets. These are the strengths of the paper.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Although there are some novel aspects in the proposed modules, they are limited to incremental improvement of UNet-like architecture.
- There are some new methods published which show better performance than the proposed method on the same datasets such as: https://paperswithcode.com/sota/medical-image-segmentation-on-cvc-clinicdb. Those top performing methods in the leader board shall be used for comparison.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The benchmark datasets used in the paper are publicly available, however, there is no mention in the paper about the availability of the code for reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
This paper presented an important research topic of broad interests - how to further improve the performance of those commonly used networks such as UNet. The paper is well written in clear English. The proposed modules have some novel aspects. The paper can be further improved by referencing some latest papers and comparing the proposed method with the models published in these leader boards: https://paperswithcode.com/sota/medical-image-segmentation-on-cvc-clinicdb https://paperswithcode.com/sota/medical-image-segmentation-on-kvasir-seg https://paperswithcode.com/sota/medical-image-segmentation-on-cvc-colondb https://paperswithcode.com/sota/medical-image-segmentation-on-etis
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the proposed method has some novel aspects, they only present an incremental advance over previous work. Some latest SOTA methods need to be compared and referenced as listed above in the constructive comments.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
The authors have largely addressed my comments, I am happy to revise my score to 6.
Review #3
- Please describe the contribution of the paper
The authors designed a novel approach for polyp segmentation. Differently from strategies based on the UNet model, the authors introduce three novel modules to enhance the decoder via feature propagation and aggregation. The former is achieved by substituting the single-scale skip connections with the FPE module that considers features at all scales. The latter is obtained via the FAE and MSA modules that filter relevant information and consider multi-scale high-level features. The proposed approach is validated on five public collections, where it either considerably increases the state-of-the-art or achieves performance that are in line with existing solutions.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
High novelty: the manuscript contains several interesting ideas to better exploit features extracted by a neural network. Via their three novel modules, the authors manage to considerably improve the state-of-the-art on three datasets while also achieving in-line performances on the other two collections, fully demonstrating the effectiveness of their proposal. In detail, the modules account for multi-scale information deriving from the encoder feature maps and can filter relevant cues to generate more robust segmentation maps. Sound experimentation: the authors present both quantitative and qualitative results. The former through a SOTA comparison and a well-thought ablation study, which can highlight the efficacy of each module; the latter through a visual comparison of segmentation maps, where the proposed model can extract more accurate polyps maps.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Improvable formality: although the paper presents its methodology clearly, it might be helpful to also use a more formal presentation of the method so that it is easier to find new and improved strategies. This aspect can also be noticed by the use of misleading language, e.g., “full stages” when referring to the feature maps extracted by the encoder or the (easily fixable) figures where it is hard to understand the feature propagation at a glance.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors provide detailed information on their methodology, train/test procedure, and used datasets, resulting in a reproducible work.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The authors presented a well-thought and well-written approach that can be easily re-implemented since all details are provided. Possible improvements would be associated with the presentation of their work. For instance, referring to the multi-scale feature maps extracted by the encoder as “full stages” can be misleading, whereas referring to them as “all encoder-extracted feature maps” (or similar) would immediately identify the input of the FPE module. Similarly, the presented images provide enough information but can be improved by using diverse lines. For example, in Fig. 2(b), it is hard to follow the propagation of features toward the filter operations (i.e., otimes symbols), which would easily be solved using dashed lines or different colors. Finally, broader information on related works focusing on feature aggregation and propagation would be interesting as they are easy to find in other application fields.
On a different note, Eq. 2 should be double-checked as the O_i indices for the FAE module seem to be wrong, considering i=1,2,3. Furthermore, Fig. 2 should also be modified as concatenation and summation operators in the FPE and FAE have the same oplus symbol, which is confusing.
As possible future work, it would be interesting to show and discuss failed examples so that further advancements can be accurately designed for the proposed methodology in the segmentation of this life-threatening illness.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors present a novel approach with several interesting strategies that can consistently improve the state-of-the-art on several datasets. Furthermore, they show extensive experimentation through quantitative and qualitative results as well as an ablation study that helps fully appreciate the proposed solutions.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
As per the previous review, the authors present a novel method backed up by extensive experiments. Moreover, in their rebuttal the authors understood the raised issues and explained how they are to be fixed, thus remaining as a strong accept from my point of view.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Some details of proposed method may be missing. The method section should be re-organized and presented in a more formal way. The improvements yielded by the proposed approach may be marginal, compared to U-Net shape architecture. Some recent studies should be included for comparison.
Author Feedback
To Reviewer 1
- Technical Details Clarification:
- p in FPE in Fig.2: Yes, it should indeed be p4.
- The process of merging inputs (‘C1’, ‘C2’, ‘C3’) in CU layer involves integrating multi-hierarchical features using a concatenation operator rather than element-wise summation or direct summation to provide enhanced segmentation by incorporating additional information. The inspiration for this concatenate operation comes from DeepLabV3 and PSPNet, both of which utilize concatenation of multi-hierarchical features to achieve rich and robust feature representations. Subsequently, the merged features are fed into a CU to reduce the number of channels. Finally, these reduced features are combined with the output of the previous FAE using a summation operator.
- Reference feature generation: By selectively enhancing useful information and filtering out irrelevant data, these reference features (denoted as X) assist in identifying optimal features at the current level (denoted as Y). For example, (‘P1’, ‘P2’, ‘P3’) in FPE are interpolated by stepwise downsampling strategy, and then the output from three other level branch is the reference features X.
- Typo: The final output is indeed O1.
- Explanation for MSA: It benefits from an additional supervision signal, as observed in PraNet, CaraNet, and etc by aiding in forming a coarse location of the polyp and contributing to improved accuracy and performance.
To Reviewer 2
- Novelty: U-Net and its variants have established themselves as widely used baselines in both medical image analysis and general computer vision tasks. Our proposed method introduces innovative modules for improving feature propagation and feature aggregation, which can be easily incorporated into U-Net-like networks. More importantly, our method can resolve the common challenges of feature propagation and aggregation with its generality and scalability as a task and domain solution. For example, a novel approach to transmit a real multi-scale feature aggregated from multi-level branch.
- Performance Improvement: Through extensive evaluations on five datasets, we demonstrate the effectiveness of our method, particularly in challenging datasets such as CVC-ColonDB and ETIS, where it surpasses previous state-of-the-art models by a significant margin (3%) in terms of mIoU and mDice in Table.1.
- Compared methods: We couldn’t compare our method with all the methods on the leaderboard as they were designed for different datasets than ours. For example, FCB-SwinV2 Transformer achieved the top score on CVC-ClinicDB’s leaderboard, but was trained and evaluated only on that dataset, while our method follows PraNet’s settings and was trained on part of Kvasir and CVC-ClinicDB dataset and evaluated on multiple datasets. Our approach aims to evaluate the model’s learning and transferability on various datasets, rather than being limited to one dataset. Our experimental setting aligns with numerous works published in MICCAI, such as MSNet, SANet, LDNet, and SSFormer, all of which our method outperforms. The best-performing method, DuAT, achieved similar results to ours on the same experimental setting. Its results on the five datasets in terms of mDice were 0.948, 0.924, 0.819, 0.822, and 0.901, while our results were 0.931, 0.928, 0.837, 0.822, and 0.905. Our method outperforms the state-of-the-art methods on four datasets, highlighting the significance of our approach.
To Reviewer 3 1.Writing: The writing and figured will be improved in the revised version (“full stages” will be replaced by “all encoder-extracted feature maps”) as suggested by reviewer.
- As for Eq.2, It’s right, It means, O1 = FAE(C1,C2,C3,O2),O2=FAE(C2,C3,O3),O3=FAE(C3,O4), corresponding to Fig2’s three FAE module.
- The Fig.2 may be incurring a little confusing, we will re-draw and making it clearly, however, the Oplus operation is right, they perform element-wise summation in both FPE and FAE.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
All reviewers agreed on the acceptance after rebuttal.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After rebuttal, all reviewers strongly support for the acceptance of this paper. Along with my reading of the paper and rebuttal, a decision of accept is recommended according to the overall quality of the paper. However, I encourage the authors to revise the paper per the reviewers’ suggestion in the official version which helps to enhance the impact of the paper on the MICCAI society.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Based on the rebuttal, the authors have addressed the main concerns from the reviewers. Overall, hhis paper proposes a novel feature enhancement network to improve feature propagation and aggregation for polyp segmentation, and the experimental results show the effectiveness of the proposed model. Thus, we have decided to accept this paper.