List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yutian Shen, Ye Lu, Xiao Jia, Fan Bai, Max Q.-H. Meng
Abstract
Colonoscopy images from different centres usually exhibit appearance variations, making the models trained on one domain unable to generalize well to another. To tackle this issue, we propose a novel Task-relevant Feature Replenishment based Network (TRFR-Net) for cross-centre polyp segmentation via retrieving task-relevant knowledge for sufficient discrimination capability with style variations alleviated. Specifically, we first design a domain-invariant feature decomposition (DIFD) module placed after each encoding block to extract domain-shared information for segmentation. Then we develop a task-relevant feature replenishment (TRFR) module to distill informative context from the residual features of each DIFD module and dynamically aggregate these task-relevant parts, providing extra information for generalized segmentation learning. To further bridge the domain gap leveraging structural similarity, we devise a Polyp-aware Adversarial Learning (PPAL) module to align prediction feature distribution, where more emphasis is imposed on the polyp-related alignment. Experimental results on three public datasets demonstrate the effectiveness of our proposed algorithm. The code is available at: https://github.com/CathyS1996/TRFRNet.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16440-8_57
SharedIt: https://rdcu.be/cVRwI
Link to the code repository
https://github.com/CathyS1996/TRFRNet
Link to the dataset(s)
All datasets we used are public datasets as indicated in reference
Reviews
Review #1
- Please describe the contribution of the paper
A method to segment Colonscopy polyp images is presented. In particular, the paper focuses on the domain shift problem when data belongs to different centres. The paper proposes a domain invariant feature decomposition (DIFD) module that aims to reduce style variations, then a task-relevant feature replenishment module tries to disentangle informative context from the residual domain specific features of DIFD, and finally, an adversarial learning strategy is applied to bridge the domain gap by aligning features in output space. The architecture is an encoder-decoder framework. The DIFD modules are placed after each encoding block. It decomposes the features into domain-invariant portion and side-out domain-specific portion. The domain-specific ones serve as input for TRFR module, which uses them to produce task-relevant features. Both domain-invariant features and task-relevant features are combined to predict four segmentation maps through a decoder network. Finally, features in the
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper faces a very important problem in the medical imaging field, the domain-shift problem. The idea to join domain-invariant features and task-specific features is interesting. The paper is well written and simple to follow. Extensive quantitative comparisons are conducted, and all the methods are trained using the same data splits.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It is not clear how attention is used in the DIFD module. In section 2.1: the paper states that h() and g() are the attention functions, but no details are provided about them. An overview of the DIFD module is provided in Fig.2 but there is not a clear explanation of the design of the module in the paper.
- Sec 2.3 How the w_bg is calculated? What the “significance of predicted background region alignment” represents?
- What is the real contribution of adversarial and TCLoss? The lamda parameters used are very lower. Why? Have these two losses different orders of magnitude w.r.t. the segmentation loss?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The hyperparameters are clearly listed. Although is not clear how the model is evaluated. The paper states that the maximum training epoch is set to 150. Is the model evaluated at the end of the 150 epoch? Or maybe some strategy is used to select the best epoch in which the model is evaluated?
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Some aspects need to be clarified as stated in the weakness section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is easy to follow, and tackles a real problem in medical image field. The validity of this approach is supported by the improved performance w.r.t. the other methods. Although some aspects need to be clarified.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This paper propose an UDA method that effectively combines multiple techniques, including style transfer, adversarial learning and self-attention, for polyp segmentation. Extensive experiments demonstrate the superiority of the method over prior works.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is generally well-written and easy-to-follow;
- The paper explore multiple techniques in computer vision and effectively combine them into a single framework; also, each proposed module are modified to tailor the segmentation task;
- Extensive experiments on multiple datasets demonstrate superior performance of the proposed methods;
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Lack of qualitative analysis of each module: while each component (e.g. DIFD, TRFR) has clear objective, whether in the end they realize the goal or not is hard to tell from the current presentation;
- Some design decisions are lack of careful justification (TCLoss, PAAL and TRFR).
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Sufficient implementation details are provided in the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The formulation of TCLoss is not intuitive; it is understandable that I(p+) should have lower entropy than I(p_di), but TCLoss essentially encourages the network to be uncertain on I(p_di); how is that benefit the overall method and what if we only penalize I(p+)?
- For PAAL module, why pixel-level adversarial learning is applied rather than image-level? Domain is usually defined on the image-level, how does the network can differentiate the domain of a pixel?
- The TRFR module is very similar to a feature pyramid network (FPN). Does TRFR really replenish the task-relevant feature, or is just simply aggregating feature from different scales? Please discuss the different between TRFR and FPN and consider citing relevant works.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper explore and exploit ideas from multiple CV tasks and achieve considerable improvement on the UDA tasks; on the other hand, some designs lack in-depth justification and discussion, making the overall story not sound and logically coherent.
- Number of papers in your stack
8
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors successfully address the comment 1-3 raised in this review. However, according to the rebuttal of comment 2, the design of PAAL module is largely based on ref[16], which is not properly cited in the original paper. Overall, for one hand, most of the proposed designs are based on prior works; for another hand, the authors do adapt those techniques properly to fit in the interested task. So I still consider this paper as a borderline/weak accept.
Review #3
- Please describe the contribution of the paper
The authors proposed a Task-relevant Feature Re-plenishment based Network (TRFR-Net) to tackle existing problems in UDA, for eliminating domain shifts in multi-centre colonoscopy images while retaining sufficient discrimination capability.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well organized and clearly written. Tables and some of the figures are well presented. The authors present a new method for unsupervised domain adaption which is really a challenging problem.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Figure 2 needs to be redrawn again. In the current form, It is difficult to follow. For instance, From the reader’s point of view, it is hard to understand where X(s) is being used. The authors also need to mention which is their final output (prediction masks)? From the Figure, it is unclear.
- The author uses SE-block. CBAM has already shown improvement over squeeze and excitation block. Is there any specific region behind choosing the block? An ablation on this would be helpful.
- From the reader’s point of view, the authors confuse domain adaptation and domain generalization. Please make it clear.
- The authors only test on still images, consisting of tiny images samples. Currently there are many public video polyp segmentation datasets available with large number of samples and are public. Please validate your results on larger datasets such as ASU-Mayo, CVC-videoClinicDB, and SUN datasets).
- The authors could site some relevant important domain generalization and recent multi-centre work in the literature.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Although the author does not provide source code, they mention about the hyperparamters. But i do not think the method is reproducible at current form.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The authors should redraw Figure 2 and make it easier for the audience. Through literature search is missing currently. The clinical motivation of the paper could be improved. I feel that CBAM can further improve your model performance. Please give it a try.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The experiment is limited to still iamges that consist very few images. There are more few publically available video polyp segmentation dataset and also still image that have decent number of images (for example, Neopolyp (BKAI)) and few more. Experimental results on the larger dataset may help to justify the current improvement of the method and usability of the models in the clinic.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Domain adaptation is an intresting problem for the machine learning community and the authors agreed to address some of my raised concerns. Therefore, I think publishing this research would be beneficial for the community.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper faces a very important problem in the medical imaging field, the domain-shift problem. The idea to join domain-invariant features and task-specific features is interesting. The paper is well written and simple to follow. Extensive quantitative comparisons are conducted, and all the methods are trained using the same data splits. From the reader’s point of view, the authors confuse domain adaptation and domain generalization. Lack of qualitative analysis of each module: while each component (e.g. DIFD, TRFR) has clear objective, whether in the end they realize the goal or not is hard to tell from the current presentation;
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
9
Author Feedback
We thank all reviewers for their thoughtful comments. We categorize major concerns followed by responses in the following paragraphs.
AC & R3: Domain adaptation/generalization. It is a DA problem because unlabeled data from target domain is available in our training. Clinical motivation: Domain shift exists in multi-center colonoscopy images, and the ultimate goal is to improve segmentation of target sites. Pixel-wise annotations are costly, but unlabeled images from target sites are easy to obtain.
AC & R2: Qualitative analysis. We show in Fig. 2 that DIFD generates f_di with more focused attention while TRFR generates f_trr with broader attention, reffering to that DIFD extracts domain-invariant features and TRFR distills task-relevant information. Prediction map p_+ using f_di and f_trr is more confident than p_di that only uses f_di. Upon acceptance, a figure clarifying objective of each module will be added.
R1: Design of DIFD. DIFD is a normalization module to reduce style variations. In individual samples, channel attention h() & g() are used to scale the mean and std of each channel, as in Eq (1) & (2). As in Fig. 2, we use two FC layers to wrap h() and g().
R1: Weight of background pixels. w_bg is assigned as 0.5 to attach less emphasis on feature adaptation of background pixels (that of polyp pixels is 1). Thanks for problem finding, we will complement it upon acceptance.
R1: Weights of adversarial loss & TCLoss. Adv loss is to train the discriminator. Lambda_adv is low as we want the learning speed of discriminator lower than that of segmentation model (generator). TCLoss is to train TRFR module to advance disentanglement of task-relevant features. Lambda_TC is low as it is less significant than segmentation loss.
R1: Evaluation. We evaluate all models at the best epoch that achieves highest overall performance in validation (by sum of all metrics). Experimentally, performance varies little when model converges.
R2: Design of TCLoss. Thanks for pointing out, we only penalize I(p+) indeed. TCLoss is only backpropagated to update params in TRFR and will not affect params in other parts. It calculates entropy of p_di without influencing p_di. Upon acceptance, we will proofread our paper to avoid confusing sentences and release the codes for better clarity and reproducibility of our work.
R2: Design of PAAL. Our PAAL utilizes pixel-wise adversarial learning to realize output space adaptation (ref[16]). Then tailored for our task, we modify the loss with a polyp significance mask M_p to enhance adaptation of polyp-related region.
R2: Difference from FPN. TRFR replenishes task-relevant information from the residual domain-specific features by proposing the TCLoss, compared with FPN that simply assembles multi-level features.
R3: Illustration of Fig. 2. The data flow in blue is source domain data, whose input is x^s with label. While that in purple is target domain data, whose input is x^t without label. The final output for segmentation inference is the prediction map p_+. Upon acceptance, we will redraw the figure for easy understanding.
R3: Adoption of SE-Block. Thanks for putting forward CBAM, we’ve explored it for potential improvement. In our model, we use channel attention to disentangle useful components at each semantic level, so SE is adopted rather than CBAM that also learns spatial attention. And convolutional blocks following SE extract the spatial information at multi levels, as in Fig. 2.
R3: Datasets. Our model is designed for segmentation in images, so we only tested on still images. We also conduct experiments on the suggested CVC-clinicDB (C) and ETIS (E) in both directions. State-of-the-art segmentation results are achieved with improvement of 13.65% in Dice and 6.23% in mIoU when C->E and improvement of 5.93% in Dice and 2.47% in mIoU when E->C, compared with “source only”. In the future, we will extend our model to video sequences and validate on video database like ASU-Mayo DB.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
-
Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
-
After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
-
What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This article presents a method for segmentation of colonoscopy polyp images. It focuses on the domain shift problem to deal with multicenter datasets. The idea of jointly taking into account domain-invariant features and task-specific features is interesting. The rebuttal clarifies some important points. Even if some questions remain, I think that this work deserves to be published at MICCAI. My proposal is therefore “Acceptance”.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The author answer the question well in the rebuttal. 1)The paper solute the domain-shift problem by joining domain-invariant features and task-specific features ,which is interesting. 2)Author elucidates the generation of domain dataset and analyse the outcome metrics
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4