List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Junyong Shen, Yan Hu, Xiaoqing Zhang, Zhongxi Qiu, Tingming Deng, Yanwu Xu, Jiang Liu
Abstract
Common lesion detection networks typically use lesion features for classification and localization. However, many lesions are classified only by lesion features without considering the relation with global context features, which raises the misclassification problem. In this paper, we propose an Interaction-Oriented Feature Decomposition (IOFD) network to improve the detection performance on context-dependent lesions. Specifically, we decompose features output from a backbone into global context features and lesion features that are optimized independently. Then, we design two novel modules to improve the lesion classification accuracy. A Global Context Embedding (GCE) module is designed to extract global context features. A Global Context Cross Attention (GCCA) module without additional parameters is designed to model the interaction between global context features and lesion features. Besides, considering the different features required by classification and localization tasks, we further adopt a task decoupling strategy. IOFD is easy to train and end-to-end in terms of training and inference. The experimental results for datasets in two modalities outperform state-of-the-art algorithms, which demonstrates the effectiveness and generality of IOFD. The source code is available at https://github.com/mklz-sjy/IOFD
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16437-8_31
SharedIt: https://rdcu.be/cVRth
Link to the code repository
https://github.com/mklz-sjy/IOFD
Link to the dataset(s)
https://figshare.com/articles/dataset/brain_tumor_dataset/1512427
Reviews
Review #1
- Please describe the contribution of the paper
This paper introduces the whole image context embedding to enhance the classification branch of the Faster RCNN solution for the lesion detection task. The main idea is to encode the whole image as a fixed size embedding and leverage the self-attention mechanism to interact with the RoI align features before the final classifier. Evaluated on two different datasets, the proposed approach can consistently improve performance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1, The global context embedding module receives two forms of supervision during training: one is standard whole image classification and the other is a feature adaptation branch for later interaction with the localized detection heads of the original Faster RCNN design. This design enforces a good embedding at the global level.
2, Self-attention module is leveraged for the interaction between global context and local features. This interaction directly enhances the local features and proved to be beneficial for the performance. Combined with the Global embedding, the whole design is novel.
3, Experiments are well designed and conducted thoroughly. They provide good support for each proposed new component.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Despite Novel design and promising results, I do have the following concerns:
1, how is the model trained on the OCT dataset? This dataset is very small and only has 32 cases. Given the complexity of the proposed model, I am assuming each slice in each case is fed into the model for training and testing. If so, the positive sample should be highly sparse, how is this issue solved? If not, the dataset is too small to be significant.
2, Since the brain tumor dataset is public and exists for more than 5 years, I am wondering how the proposed solution compared to another published work specific to the same task? Even though many other solutions are listed in Table 2, they all are general solutions for natural image modalities, which is less persuasive.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Positive if code and the private dataset is released.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please address my concerns.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The whole idea is novel and each module design makes sense.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This paper aims at improving the classification accuracy in lesion detection problems. In some problems, classifying different lesion types is closely related to the relative location and size of the lesion in the image. Therefore, the authors propose to use attention-weighted global features to do classification, instead of ROI features. They designed the Global Context Embedding module and the Global Context Cross Attention module for this goal. On a private OCT dataset and a public MRI dataset, 6% and 3% improvement were observed in the classification accuracy, compared to Faster RCNN.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The authors pointed out that many types of lesions are distinguished by the position and the relative proportion of the lesion to the tissue,
- The idea of using attention-weighted global features to do classification is novel and intuitive.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The writing of the paper can be improved. There are some word mistakes and some unclear parts, see detailed comments below.
- The Global Context Auxiliary module seems to only support one lesion type per image. What if there are two lesions of different types in one image?
- The OCT dataset is very small, only 23 cases.
- “the recall is equal to the precision, which means the number of detections is equal to the number of ground truths. Therefore, GCE can effectively avoid missed detections.” I cannot understand the logic of this sentence. If GCE can avoid missed detections, it should have high recall.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors have checked all questions in the Reproducibility Checklist as Yes. However, no code or data will be released according to the paper. No statistical significance was reported either.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- “We adopt a 3×3 convolutional layer (Conv3) on input features of the module to avoid the optimization effect of the localization branch.” What is optimization effect?
- “The proposed IOFD network superiors the state-of-the-arts algorithms” –> “The proposed IOFD network surpasses the state-of-the-art algorithms”, “two modal datasets” –> “datasets in two modalities”, “valid set” –> “validation set”.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method has some novelty. However, the paper writing needs improvement. There are inconsistency between the Reproducibility Checklist and the paper. The proposed algorithm should be compared with some existing lesion detection / classification methods, instead of only general object detection baselines.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
5
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors added new comparison results and explained how multiple lesions of different types in one image can be detected. They also promised to release codes and improve writing. I also appreciate the novelty of the network design. However, I still concern that the OCT dataset is too small to claim the improvement is significant. Besides, the logic of this sentence is apparently incorrect: “the recall is equal to the precision, which means the number of detections is equal to the number of ground truths. Therefore, GCE can effectively avoid missed detections.” Therefore, I am neutral to the final rating of the paper.
Review #3
- Please describe the contribution of the paper
This paper proposes a new framework to fuse global context features into local lesion features for better lesion detection. Specifically, a global context embedding (GCE) module and a global context cross attention (GCCA) module are introduced to combine global and local features for classification. Experiments are conducted on an inhouse OCT dataset and a public brain MRI dataset. Results indicate the proposed method achieves state-of-the-art performance. Ablation study proves the effectiveness of the GCE and GCCA modules.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This paper introduced two new modules, global context embedding module and global context cross attention module, to enhance lesion features for better lesion detection. The introduced modules are well motivated and simple.
- Experiments on two datasets show the effectiveness of the proposed method. Both GCE and GCCA modules are validated through an ablation study.
- This paper is well-written and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- What is the IoU threshold for reporting precision and recall? What are the APs for IoU=0.50 and 0.75 like?
- The proposed method only modifies the features for classification branch. Why not use the enhanced features for box regression branch?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper is well-written with clear description of technical details. It should be easy to reproduce.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Section 2.2, “Firstly, We adopt …” -> “First, we adopt …”
- Section 3.2, “In particular, the recall is equal to the precision, which means the number of detections is equal to the number of ground truths.” What does this imply?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method introduced two new modules with good motivation and simple formulation. It is also validated on two lesion datasets. Despite some minor concerns, this paper deserves to be accepted.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors’ feedback addressed my concerns. I maintain my rating as this paper proposed a novel method with well-established motivation and good experimental results.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper introduces the whole image context embedding to enhance the classification branch of the Faster RCNN solution for the lesion detection task. It receives two positive reviews and one negative reviews. In the rebuttal, the author should clarify the major concerns raised by reviewer 2, including the strategy for addressing two lesions of different types in one image, the performance on the large dataset and limited comparison, etc.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Author Feedback
We sincerely thank the reviewers for their high-quality reviews and constructive comments. We are happy to learn that all reviewers appreciate our motivation and novelty. Below we provide point-to-point responses to the comments, which will be integrated into the final version.
[Q] Numbers of OCT dataset(R#1/2). [A] There are 5063 OCT images for training and testing. The OCT dataset contains 23 cases and each case has more than one image.
[Q]Two lesions of different types in one image (R#META/2). [A] Our algorithm can localize different types of lesions, no matter multiple identical lesions or multiple different types of lesions in one image. As shown in Fig. 2, our IOFD obtains many bounding box features L, which have their corresponding enhanced features F for corresponding lesion types. The type can be the same or different even in the same image. Specifically, for an image with multiple types of lesions, our GCE loss only needs to be changed to multi-label loss.
[Q]Why not use enhanced features for the box regression branch (R#3). [A]There are two reasons. One is our motivation to improve classification performance. Our motivation is targeting a severe misclassification problem caused by not considering the relationship between the region and context features, even though the region-based networks successfully locate the position of lesions. We do not directly improve the box regression branch in this paper, as the main purpose of our IOFD is to improve classification performance. The other one is the consideration of feature misalignment. As the features are misaligned between classification and localization [21], and classification is more dependent on the global context features, we adopt the strategy that enhanced features for classification and lesion features for regression to avoid feature misalignment.
[Q]Reproducibility of our method(R#2) [A] We will release our code and trained mode after our paper is accepted.
[Q] Lesion detection method for comparison(R#META/1/2). [A] We list the following two representative lesion detection papers for comparison with corresponding results. One is from a published work specific to the same task (2020) in the public MRI dataset. The other is from existing lesion detection from MICCAI (2019). We reproduced their methods on datasets in two modalities. The results are as follows Method / mAP / Acc / Recall / Precision
AMD(OCT) MSB[1] / 0.7112 / 0.8233 / 0.7546 / 0.7765 Yakub[2] / 0.7195 / 0.8392 / 0.7509 / 0.7640 IOFD(ours) / 0.7548 / 0.9030 / 0.7963 / 0.7963
Brain Tumor(MRI) MSB[1] / 0.6954 / 0.8942 / 0.7589 / 0.7774 Yakub[2] / 0.7658 / 0.9253 / 0.7916 / 0.8015 IOFD(ours) / 0.7930 / 0.9548 / 0.8242 / 0.8242
The results show that our IOFD outperforms these lesion detection methods. Particularly, our method improves Acc by about 2.95% and mAP by about 2.72% in the public MRI dataset, which indicates global features we extracted are beneficial for lesion classification. Our method successfully reduces the misclassification rate and therefore achieves better mAP. [1]Detection and classification of brain tumor in MRI images using deep convolutional network. ICACCS. IEEE, 2020. [2]Attentive CT Lesion Detection Using Deep Pyramid Inference with Multi-Scale Booster. MICCAI. Springer, Cham, 2019.
[Q] Some unclear parts(R#2/3) [A]1.“GCE can avoid missed detections” means GCE can help to detect the lesion as many as possible and can avoid lesion missing detection. 2.”We adopt a 3×3 convolutional layer…” We use a 3×3 convolution to encode the features to alleviate the localization branch’s emphasis on the original features. The backbone features are shared by localization and classification. FPN of the location branch interacts with the multiple layers of the backbone, but the classification branch is from the end of the backbone.
- The IoU threshold for reporting precision and recall is 0.5. Finally, we will further revise the paper following our native English friend.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
In the rebuttal, the authors added new comparison results and explained how multiple lesions of different types in one image can be detected. The reviewer also admit the novelty fo this paper. Therefore, I would suggest to accept this paper. In the final version, the author should proof read the whole paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Based on the feedback of the authors and the combined comments of the reviewers, we have decided to accept this paper. In the rebuttal, the authors add new comparisons and explain how multiple different types of lesions can be detected in a single image. The reviewers also acknowledge the novelty of this paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After rebuttal, all three reviewers are positive on evaluating this paper as a reasonably novel work with good/solid performance. I would agree with three reviewers.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6