List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yu-Jen Chen, Xinrong Hu, Yiyu Shi, Tsung-Yi Ho
Abstract
Magnetic resonance imaging (MRI) is commonly used for brain tumor segmentation, which is critical for patient evaluation and treatment planning. To reduce the labor and expertise required for labeling, weakly-supervised semantic segmentation (WSSS) methods with class activation mapping (CAM) have been proposed. However, existing CAM methods suffer from low resolution due to strided convolution and pooling layers, resulting in inaccurate predictions. In this study, we propose a novel CAM method, Attentive Multiple-Exit CAM (AME-CAM), that extracts activation maps from multiple resolutions to hierarchically aggregate and improve prediction accuracy. We evaluate our method on the BraTS 2021 dataset and show that it outperforms state-of-the-art methods.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43907-0_17
SharedIt: https://rdcu.be/dnwcf
Link to the code repository
https://github.com/windstormer/AME-CAM
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper, authors propose a novel Class Activation Map (CAM), called Attentive Multiple-Exit CAM (AME-CAM) to improve the segmentation of brain MR images. The proposed approach extracts activation maps from different exits of the network to capture information from multiple resolutions. Then, an attention model is used to hierarchically aggregate these activation maps, learning pixel-wise weighted sums. The proposed method is evaluated on the BraTS2021 database.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The problem addressed is difficult and the results obtained are interesting. The original contribution resides in the use of an attention network to aggregate different activation maps from the outputs of internal classifiers.
The mathematical background of the proposed method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper is well written, but the proposed method lacks details, hardly a page of the paper is used to describe it.
Also, the flowchart of the proposed method is a bit hard to understand. For example, the operator (a dot in the middle of a circle) is not commented.
It would have been interesting to show examples of obtained foreground and background images.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Authors provide a link for the release of their code
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
This paper, authors propose a novel Class Activation Map (CAM), called Attentive Multiple-Exit CAM (AME-CAM) to improve the segmentation of brain MR images. The proposed approach extracts activation maps from different exits of the network to capture information from multiple resolutions. Then, an attention model is used to hierarchically aggregate these activation maps, learning pixel-wise weighted sums. The proposed method is evaluated on the BraTS2021 database.
The paper is well written, but the proposed method lacks details, hardly a page of the paper is used to describe it.
Also, the flowchart of the proposed method is a bit hard to understand. For example, the operator (a dot in the middle of a circle) is not commented.
It would have been interesting to show examples of obtained foreground and background images.
Strengths The problem addressed is difficult and the results obtained are interesting. The original contribution resides in the use of an attention network to aggregate different activation maps from the outputs of internal classifiers.
Weaknesses The paper is well written, but the proposed method lacks details, hardly a page of the paper is used to describe it.
Also, the flowchart of the proposed method is a bit hard to understand. For example, the operator (a dot in the middle of a circle) is not commented.
It would have been interesting to show examples of obtained foreground and background images.
The paper would benefit from the application of the proposed approach to other clinical database.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Obtained results are interesting. However, the proposed approach is rather weak. Also, the paper lacks details.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
Among weakly-supervised semantic segmentation (WSSS) methods, existing CAM methods suffer from low resolution due to strided convolution and pooling layers. In this paper, the authors propose a new CAM method, Attentive Multiple-Exit CAM (AME-CAM), which extracts activation maps from multiple resolutions and then uses an attention model to aggregate activation maps at different resolutions.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- In this paper, the authors propose a CAM method to address the problems that exist in existing CAM methods, and the results on the BraTS 2021 dataset are significantly better than the other methods mentioned by the authors, which is of some practical value in the field of medical image analysis.
- In the qualitative analysis of the results, the authors added a visual rendering of the comparison model, which allows for an intuitive and clear analysis of the problem and the merits of their own model.
- The experimental design in this paper is relatively reasonable. In addition to the comparison experiments with weakly supervised segmentation methods, there are also comparisons with unsupervised and supervised methods. These experimental comparisons serve as interesting references for potential performance upper and lower bounds for all CAM methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The motivation of the paper is not very clear, and a detailed statement of the research motivation is lacking.
- For solving the problem of low-resolution class activation maps, the proposed method is limited in its innovation.
- In the Methods section, the authors do not give an overall introduction to the model.
- The adequacy and validity of the experiments are not sufficient. First, there are relatively few models to compare, and most of the models compared are before 2022, with only one model in 2022. Second, only two datasets were used in the experiments, and the introduction of the datasets and the setup of the experimental parameters were not detailed enough. Finally, in the supplementary material, HD95 does not perform well on the TCGA-LGG dataset.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors did not specifically introduce the experimental environment and parameters and did not provide the code, which is not very convenient for other researchers to access.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- The authors are encouraged to add more datasets for experiments to ensure better generalization ability of the model.
- In the Methods section, in addition to the analysis and introduction of the proposed modules, the authors can add an overview of the full model to make it easier for the reader to understand the full model.
- In the ablation experiments section, the authors only record the numerical results of relevant experiments, which can further validate the effectiveness of the module from a visual effects perspective.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The paper lacks a detailed statement of the motivation for the study.
- There is limited innovation in the approach presented in the paper.
- The adequacy and validity of the experiments is not sufficient.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proposes a CAM-based approach for weakly supervised segmentation of t brain tumors in MRI.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A novel method based on using multiple CAMs, and aggregating them to generate the segmentation in a weakly supervised framework
- Large dataset (BraTS 2021)
- Reasonable segmentation results especially for FLAIR sequence
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- There are perturbation based algorithms for weakly-supervised segmentation, which is missing from literature review and comparison to results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper is reproducible
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The paper introduces a new algorithm for weakly supervised segmentation. The paper is generally well written. Perhaps using MRI sequences as unput cannels would improve the results, rather than feeding them into separate models.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
A novel method with reasonable results, for weakly supervised segmentation of brain tumors.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
In this paper, authors propose a novel Class Activation Map (CAM), called Attentive Multiple-Exit CAM (AME-CAM) to improve the segmentation of brain MR images. The proposed method is evaluated on the BraTS2021 database. The experimental results show that it outperforms state-of-the-art methods. The three reviewers also affirmed the merits of this paper. The issues include adding more datasets, adding an overview of the full model, adding literature review and comparison to results, adding examples of obtained foreground and background images, and some other details mentioned by reviewers. Please address these concerns in the final version.
Author Feedback
Reviewer #1
- Q1: The proposed method lacks details; hardly a page of the paper is used to describe it. A1: We believe that all the details, including the loss, hyperparameters, and the operation of the model, are introduced in Section 2 and 3.2.
- Q2: The flowchart of the proposed method is a bit hard to understand. For example, the operator (a dot in the middle of a circle) is not commented. A2: The operator (a dot in the middle of a circle) denotes the pixel-wise weighted sum, and the operator (a cross in the circle) denotes the pixel-wise multiplication. We will add a description in the caption of Fig. 1.
- Q3: It would have been interesting to show examples of obtained foreground and background images. A3: Unfortunately, due to space constraints, we only embedded an example of the input (I) and the obtained foreground mask M_f in Fig. 1. If possible, we will add more examples in the supplementary materials.
- Q4: The paper would benefit from the application of the proposed approach to other clinical databases. A4: Due to the space limit, most MICCAI papers use one or two dataasets. We reported results on one large dataset, BraTS, in the main manuscript, and another small dataset, TCGA-LGG, in the supplementary materials. We will explore more tasks in the future.
Reviewer #2
- Q1: The motivation of the paper is not very clear, and a detailed statement of the research motivation is lacking. A1: The motivation is explained in Section 1: Most deep learning approaches for segmentation require fully or partially labeled training datasets, which can be time-consuming and expensive to annotate. To address this issue, recent research has focused on developing segmentation frameworks that require little or no segmentation labels. Our work puts forward a framework that achieves better performance than SOTA using only class labels.
- Q2: For solving the problem of low-resolution class activation maps, the proposed method is limited in its innovation. A2: Different from existing works, our efficient hierarchical method aggregates the intermediate activation map to generate a high-resolution CAM. In addition, our ablation study (Table 2) also demonstrates the significance of this novel technique.
- Q3: In the Methods section, the authors do not give an overall introduction to the model. A3: Due to space constraints, the overall introduction of the method is provided in the last paragraph of Section 1.
- Q4: The authors did not specifically introduce the experimental environment and parameters and did not provide the code, which is not very convenient for other researchers to access. A4: We have introduced the experimental environment and the hyperparameters in Section 3.2. Moreover, we have also released our code, which can be accessed through the link provided in Section 1.
- Q5: The authors are encouraged to add more datasets for experiments to ensure better generalization ability of the model. A5: Due to the space limit, most MICCAI papers use one or two dataasets. We reported results on one large dataset, BraTS, in the main manuscript, and another small dataset, TCGA-LGG, in the supplementary materials. We will explore more tasks in the future.
- Q6: There are relatively few models to compare, and most of the models compared are from before 2022, with only one model from 2022. A6: In this paper we have compared with a total of six popular WSSS methods, plus one unsupervised and two fully supervised baselines. In fact, two of the WSSS methods are from 2022, Swin-MIL (published in MICCAI’22) in Table 1, and, C2AM (published in CVPR’22), in Table 2. C2AM is presented in the ablation study as it is a refinement to the Avg. ME method. Beyond these two, we are not aware of any other WSSS method published in 2022 using class labels for medical image segmentation.
Reviewer #3
Thank you for your support.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
In this paper, authors propose Attentive Multiple-Exit CAM (AME-CAM) for brain tumor segmentation in magnetic resonance imaging (MRI). This method aggregate multiple resolution activation maps to enhance the resolution of the segmentation mask and learns the pixel-wise weighted sum of the activation maps by a novel contrastive learning method。 The authors provided accurate and detailed answers to most of the reviewers’ questions. However, I believe that some of the concerns raised by reviewer 2 have not been fully addressed. The motivation of the paper is not very clear. The authors’ answer is too general and lacks a direct connection to the innovation presented in this paper. In the Methods section, the authors do not give an overall introduction to the model. The authors provided an explanation in the penultimate paragraph of section 1, however, I believe it still does not provide readers with a clear understanding of the entire model. In conclusion, the authors’ description of the novelty and motivation of this paper is not sufficiently clear. Therefore, I believe this article falls below the acceptance threshold.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
An interesting method with reasonable results, for weakly supervised segmentation of brain tumours. The rebuttal addressed most of the comments in a satisfying way, in my opinion.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The present manuscript introduces an innovative AME-CAM technique aimed at enhancing the segmentation of brain MR images. The approach developed by the authors entails the extraction of activation maps from various network exits to glean information at multiple resolutions. Subsequently, an attention model is employed to hierarchically consolidate these activation maps, thereby learning pixel-wise weighted sums. The proposed methodology is put to the test using the BraTS2021 database. The authors’ responses satisfactorily addressed the majority of the concerns raised. This work, in my opinion, holds significant value and may act as a stepping stone for further investigations within the medical realm. Thus, I advocate for the acceptance of this paper. In case of acceptance, I would strongly suggest the authors incorporate discussions brought up during the rebuttal process in the final version of the manuscript.