Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Yazhou Zhu, Shidong Wang, Tong Xin, Haofeng Zhang

Abstract

Automated segmentation of large volumes of medical images is often plagued by the limited availability of fully annotated data and the diversity of organ surface properties resulting from the use of different acquisition protocols for different patients. In this paper, we introduce a more promising few-shot learning-based method named Region-enhanced Prototypical Transformer (RPT) to mitigate the effects of large intra-class diversity/bias. First, a subdivision strategy is introduced to produce a collection of regional prototypes from the foreground of the support prototype. Second, a self-selection mechanism is proposed to incorporate into the Bias-alleviated Transformer (BaT) block to suppress or remove interferences present in the query prototype and regional support prototypes. By stacking BaT blocks, the proposed RPT can iteratively optimize the generated regional prototypes and finally produce rectified and more accurate global prototypes for Few-Shot Medical Image Segmentation (FSMS). Extensive experiments are conducted on three publicly available medical image datasets, and the obtained results show consistent improvements compared to state-of-the-art FSMS methods.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43901-8_26

SharedIt: https://rdcu.be/dnwDa

Link to the code repository

https://github.com/YazhouZhu19/RPT

Link to the dataset(s)

Abd-MRI: https://chaos.grand-challenge.org/

Abd-CT: https://www.synapse.org/#!Synapse:syn3193805/wiki/217780

Card-MRI: http://www.sdspeople.fudan.edu.cn/zhuangxiahai/0/mscmrseg/

Reviews

Review #1

  • Please describe the contribution of the paper

    The authors proposed to combine subdividing images with search and filter modules to help reduce the effects of heterogeneity between patient scans and lead to better prototypes for prototypical learning in medical image segmentation.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The authors propose a method which appears to be at least somewhat novel and the proposal of the solution is well motivated to tackle the problem of heterogeneity of scans across different patients.
    2. The authors prove out their method across multiple modalities (CT and MRI) and show SotA results. The experiments and thorough and extensive.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The method is not extremely novel. The introduction of search and filter modules to help reduce bias is not new. The subdividing of regions to deal with heterogeneity is not new. However, their combination appears to be novel and is well suited to handle the significant intra-class variability that can be seen across patients and scanning types.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Good, public code.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    Some of the language is a bit over the top and could be toned down. For example: “endowed with” and “ingeniously designed”.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The method contains enough novelty and further, the solution seems well motivated by the problem (heterogeneity across patients can lead to high intra-class variance and thus difficulty in segmentation).

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    5

  • [Post rebuttal] Please justify your decision

    I am in agreement with R4 “Appropriate methodology, extensive experiments with comparison and ablation study” and I said something similar. There is not a ton here in terms of true novelty, mostly just an incremental approach, but I think there is just enough to get over the bar and justify publication.



Review #3

  • Please describe the contribution of the paper

    This paper presents the Region-enhanced Prototypical Transformer (RPT), a few-shot learning-based method to address the issue of large intra-class diversity/bias in automated segmentation of medical images. The RPT method leverages a subdivision strategy to generate regional prototypes from the foreground of the support prototype, along with a self-selection mechanism to suppress interferences present in the query prototype and regional support prototypes. The RPT method employs Bias-alleviated Transformer (BaT) blocks in a stacked manner to optimize the generated regional prototypes and ultimately produce more accurate global prototypes for Few-Shot Medical Image Segmentation (FSMS). The experiments on three publicly available medical image datasets show consistent improvements compared to state-of-the-art FSMS methods.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    This paper presents the Region-enhanced Prototypical Transformer (RPT), a few-shot learning-based method to address the issue of large intra-class diversity/bias in automated segmentation of medical images. The RPT method leverages a subdivision strategy to generate regional prototypes from the foreground of the support prototype, along with a self-selection mechanism to suppress interferences present in the query prototype and regional support prototypes. The RPT method employs Bias-alleviated Transformer (BaT) blocks in a stacked manner to optimize the generated regional prototypes and ultimately produce more accurate global prototypes for Few-Shot Medical Image Segmentation (FSMS). The experiments on three publicly available medical image datasets show consistent improvements compared to state-of-the-art FSMS methods.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    However, there are some limitations to consider. The authors used a weight-shared ResNet-101 as a backbone for feature extraction, which was pre-trained on the MS-COCO dataset. It is not clear why Resnet-50 or a model pre-trained on Imagenet was not chosen instead, as they are more commonly used in computer vision tasks. Additionally, the prototype approach used in this paper is a common technique in few-shot learning. In Fig. 2, MHA stands for Multi-Head Attention? Should be highlighted. The description of Bias-alleviated Transformer (BaT) blocks is not included in the paper, and it is unclear how it works for few-shot segmentation tasks. Further explanations and descriptions would be beneficial to enhance the paper’s clarity

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Yes

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    This paper presents the Region-enhanced Prototypical Transformer (RPT), a few-shot learning-based method to address the issue of large intra-class diversity/bias in automated segmentation of medical images. The RPT method leverages a subdivision strategy to generate regional prototypes from the foreground of the support prototype, along with a self-selection mechanism to suppress interferences present in the query prototype and regional support prototypes. The RPT method employs Bias-alleviated Transformer (BaT) blocks in a stacked manner to optimize the generated regional prototypes and ultimately produce more accurate global prototypes for Few-Shot Medical Image Segmentation (FSMS). The experiments on three publicly available medical image datasets show consistent improvements compared to state-of-the-art FSMS methods. However, there are some limitations to consider. The authors used a weight-shared ResNet-101 as a backbone for feature extraction, which was pre-trained on the MS-COCO dataset. It is not clear why Resnet-50 or a model pre-trained on Imagenet was not chosen instead, as they are more commonly used in computer vision tasks. Additionally, the prototype approach used in this paper is a common technique in few-shot learning. In Fig. 2, MHA stands for Multi-Head Attention? Should be highlighted. The description of Bias-alleviated Transformer (BaT) blocks is not included in the paper, and it is unclear how it works for few-shot segmentation tasks. Further explanations and descriptions would be beneficial to enhance the paper’s clarity

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    See comment

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #4

  • Please describe the contribution of the paper

    This paper proposes a region-enhanced prototypical transformer to mitigate the effects of large intra-class diversity by 1) subdivision strategy for regional prototype formation; and 2) self-selection mechanism with bias-alleviated transformer (BAT) blocks.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    1) This paper is well-structured, well-illustrated with detailed experiments and visualization.

    2) The prototypical generation mechanism and the selection mechanism in transformer block are suitable and reasonable under the few-shot segmentation setting.

    3) Three datasets are are validated, which is good and necessary.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1) The novelty of the proposed modules is limited as the idea of local/subregional prototype has been proposed (e.g., [1][2][3]).

    2) The presentation of math formulas is kind-of unclear and needs improvement.

    Ref [1] Yu, Qinji, et al. “A location-sensitive local prototype network for few-shot medical image segmentation.” 2021 IEEE 18th international symposium on biomedical imaging (ISBI). IEEE, 2021.

    Ref [2] Ouyang, Cheng, et al. “Self-supervised learning for few-shot medical image segmentation.” IEEE Transactions on Medical Imaging 41.7 (2022): 1837-1848.

    Ref [3] Liu, Jinlu, and Yongqiang Qin. “Prototype refinement network for few-shot segmentation.” arXiv preprint arXiv:2002.03579 (2020).

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Code available with README intro.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    1) Please include a math table to indicate all the involved math denotation and vector shape into the supplementary material for clarity.

    2) Why the foreground region is partitioned into N_f regions where N_f=10 by default? What if we adopt an adaptive manner to change the number of regions according to the size of the foreground target?

    3) Can the author provide a comparison with a fixed threshold tau in formulation of the coarse query foreground prototype?

    4) In the  selection of prototype in RPT, we use the simple affinity map by dot production to act as weights for \hat{P}_q. Can we try other techniques such as learning such affinity given \hat{P}_s, \hat{P}_q?

    5) There exists a trade-off between the boundary loss and the dice loss. What is the reason of setting such a hyper-parameter? Please also discuss its effect with experiments.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Appropriate methodology, extensive experiments with comparison and ablation study

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper has received mixed scores, with reviewers raising important concerns. On the positive side, reviewers acknowledge the efforts of the authors to evaluate the work on three different datasets, and agree that the prototypical generation mechanism is somehow well-motivated. Nevertheless, reviewers question the actual novelty and request further clarifications in important components of the methodology and design choices (e.g., unclear formulas; use of ResNet-101 as a backbone for feature extraction pre-trained on the MS-COCO, contrary to the standard literature; description of Bias-alleviated Transformer (BaT) blocks; strategy for foreground region partition; or comparison with a fixed threshold \tau in formulation of the coarse query foreground prototype). Authors are encouraged to consider the constructive feedback and recommendations provided by the reviewers, and address all these concerns in their rebuttal.



Author Feedback

Q1: Unclear formulas(MR&R1&R4).Thanks, we will modify unsuitable descriptions, and a detailed math table involved denotation and vector shape will be included in appendix. Q2: ResNet101 pretrained on MS-COCO(MR&R3).Since most of the compared methods[2][4][12][13][15][20] using ResNet101 in their papers, we did the same for fair comparison. Results with ResNet50 pre-trained on ImageNet: (Setting1(mean): Abd_MRI:81.09, Abd-CT:75.83; Setting2: Abd-MRI:77.98; Abd-CT:69.49). Although the result shows a little lower, it is still sota and outperforms other baselines. There are two reasons for the degradation: 1) ResNet101 is better than ResNet50 for feature extraction; 2) MSCOCO is a segmentation targeted dataset, so the pre-trained network is better for segmentation task than classification targeted dataset, such as ImageNet. Q3: Common technique(R3&R4).Our method focuses on alleviating diversity/bias with a Region-enhanced Prototypical Transformer between support and query prototypes, which has never been studied in previous FSMS methods. Our contributions are included in two novel modules: Stacked Bias-alleviated Transformer blocks is proposed to mitigate the effects of large intra-class variations; a subdivision strategy is proposed to generate multiple regional prototypes, which can be further iteratively optimized by the RPT to produce the optimal prototype. In addition, prototype based network is still the SOTA framework for FSMS task, so we adopt it for achieving better performance.We will discuss difference between our method and Ref[1,2,3] suggested by R4. Q4: Description of BaT and MHA block(MR&R3).BaT is a part of RPT, so we describe it after line 6 in Section 2.4. BaT has two main components: (a) S&F module described in Eq.(3) and Eq.(4); (b) a stack of MHA module and MLP module which is formulated in Eq.(5). In Fig.2, MHA stands for multi-head self-attention attention which is written from line 24 of Section 2.4. In BaT block, S&F module suppresses the incompatible regional prototypes with apparent affinity map and designed selection threshold \xi, the rest part of prototypes will be activated and reconstructed as rectified prototypes. Then, subsequent MHA and MLP blocks can enhance its representation ability. In RPT, support prototype \hat{P}_s and query prototype \hat{P}_q are iteratively updated by three stacked BaT and two QPG for generating final optimized prototype P_s. Q5: Strategy for foreground partition(MR&R4).Setting N=10 is an empirical solution. If N is too small, a limited number of foreground prototypes will hinder accurate removal of disturbing parts from the foreground. When N is too big, it reaches dozens or even hundreds of foreground prototypes, and leads to an unnecessary increase in computational workload. Thanks for your suggestion, an adaptive N may be much better, and we will follow your advice to conduct our next research. Q6: Fixed tau(MR&R4). We conduct experiments with fixed threshold tau(tau=-7.5,-8.0,-8.5,-9.0,-10.0) in QPC module. Results in setting2 are:(tau=-7.5: Abd_MRI:75.87, Abd-CT:68.36; tau=-8.0: Abd_MRI:75.06, Abd_CT:68.17; tau=-8.5: Abd_MRI:75.52, Abd_CT:68.81; tau=-9.0: Abd_MRI:76.28, Abd_CT:69.01; tau=-10.0: Abd_MRI:76.74, Abd_CT:69.36), which are worse than our results. Q7: Learn affinity from \hat{P}_s, \hat{P}_q(R4).Learning affinity map from \hat{P}_s and \hat{P}_q sounds better than directly using similarity computation although it may introduce more parameters. It is a great suggestion, and we will follow your advice to conduct our next research. Q8: Setting of hyper-parameter in loss(R4).Our hyper-parameter setting follows that in Eq.(6) of paper[7], which is the empirical setting for using combined dice loss and boundary loss. In addition, we also set fixed coefficients \eta as 0.5, and achieve: (Setting1: Abd_MR:81.78, Abd_CT:77.10; Setting2: Abd_MR:78.32, Abd_CT:70.63), which show that empirical setting with a varying hyper-param can achieve much better performance.



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    This work received mixed scores during the review process, with major concerns related to the novelty and further clarifications on design choices. While some unclear points have been clarified in the rebuttal, authors failed to clearly position their work in terms of novelty. In particular, the claimed novel modules are detailed again, but the differences with existing works are not highlighted, which is left as a promise. The main argument for these additions is based on the good performance on other tasks, but there are no helpful insights which properly motivate their use. Comments after the rebuttal do not seem very enthusiastic about this work, which makes me wonder the potential interest that it may bring during its presentation at the conference. Given these points, I recommend the rejection of this submission.



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    Upon reviewing the reviews and the authors’ rebuttal, I believe they have addressed the majority of the issues effectively. While the overall structure is similar to existing techniques, they have proposed specific modules and demonstrated their usefulness in the results.



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The rebuttal unfortunately did not address my concerns. The author does not clearly explain the novelty of the paper in the rebuttal. The conclusion is not convincing. There is no reasonable theory to support technological innovation,but only results. At the same time, the typesetting of rebuttal is so crowded, and the reading impression is very poor.



back to top