Authors

Han Li, Long Chen, Hu Han, S. Kevin Zhou

Abstract

Universal Lesion Detection (ULD) in computed tomography plays an essential role in computer-aided diagnosis. Promising ULD results have been reported by multi-slice-input detection approaches which model 3D context from multiple adjacent CT slices, but such methods still experience difficulty in obtaining a global representation among different slices and within each individual slice since they only use convolution-based fusion operations. In this paper, we propose a novel Slice Attention Transformer (SATr) block which can be easily plugged into convolution-based ULD backbones to form hybrid network structures. Such newly formed hybrid backbones can better model long-distance feature dependency via the cascaded self-attention modules in the Transformer block while still holding a strong power of modeling local features with the convolutional operations in the original backbone. Experiments with five state-of-the-art methods show that the proposed SATr block can provide an almost free boost to lesion detection accuracy without extra hyperparameters or unique network designs.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16437-8_16

SharedIt: https://rdcu.be/cVRs2

Link to the code repository

https://github.com/MIRACLE-Center/A3D_SATr

Link to the dataset(s)

nihcc.app.box.com/v/DeepLesion

Reviews

Review #1

Please describe the contribution of the paper

This paper presents an approach to integrate inter-slice information for universal lesion detection. Results show consistent improvements over various baseline approaches.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The idea of the paper is simple but solid. This paper presents a natural way to combine transformers into the multi-slice fusion problem.
2. The paper is well-written and easy to follow. The figures are illustrative.
3. The results are promising. The authors compare various baselines and show their innovation is a “plug-and-play” module with consistent improvements over the baseline methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The equations in the method section is messy. List the whole workflow as equations is unnecessary and not essentially a mathematical contribution. A better way is just show an algorithm flow or pseudo codes, which would be much better and clearer. My recommendation is only keep the most important innovation as mathematical equations.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Good reproducibility if code can be provided.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

Please refer to my weakness part.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This is a solid paper with minor issues. The idea is not too novel but is solid and so are the experimental results.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper propose a novel Slice Attention Transformer (SATr) block that could be easily integrated with various CNN models. Experiments show that the proposed method improves the detection performance on Universal Lesion Detection (ULD) task under the settings of both full or less training data.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. A novel transformer block for multi-slice-input ULD backbone is proposed. Compared to the naïve version, the proposed method has clear motivation: (1) enhancing the key-slice feature, as the key-slice is where the supervision applied; (2) using the adjacent slices as query and key to further strengthen the key-slice representation; Ablation study also clearly demonstrates the contribution of each modification;
2. Extensive evaluation on multiple networks to show the effectiveness of the proposed SATr block.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The final design of SATr block, while has strong motivation, still needs more justification; for example, why we should use adjacent slices as query and key? can we simply use key-slice feature as query and all-slice feature as value?
2. The analysis of model size and flops are not provided; as the proposed block will introduce additional computation overhead, it is important to see the trade-off between accuracy improvements and computational requirements;
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors claim that code will be published. Also, sufficient implementation is provided in the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1. The writing needs improvement and there are a few typo/grammar errors, e.g. Page 4 “overfittin”, Page 5 “Kernal”, Sec 3.2. full training results should be in Table 1;
2. Please provide more justification on the design of SATr block as discussed in the above weakness section; and/or provide ablation result on, e.g. using key-slice as query and all-slice as key and value;
3. Provide analysis on model size and flops to see the computational overhead introduced by the SATr block;
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper propose a novel way to enhance the key-slice feature on ULD task, by using the self-attention block and modifying the query, key and value component in a reasonable way. However, the final design needs more justification. On the other hand, the paper lack of analysis on the introduced additional computation overhead and the performance improvement is not significant.
Number of papers in your stack

6
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

This paper proposes a novel Slice Attention Transformer block (SATr) for universal lesion detection. The proposed SATr can extract features from both individual and multiple slices. It can also be integrated into multiple network structures. Experiments are conducted on DeepLesion. Promising results are achieved.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Strength: 1.This paper is well written and organized. It is easy to follow and read. 2.The proposed SATr is easy to understand and implement. It is easy to transfer to other network structures. 3.The comparison in Table 1 between the vanilla network and the version with SATr shows that the proposed SATr does help with the lesion detection.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Weakness: 1.For the ablation study in Table 3, the authors ablate the few components. How about different hyper-parameters in the SATr? Would it be sensitive to the hyper-parameters. 2.I think the CAM figure is interesting. Could the authors kindly offer some comments or constructive explanation based on their understanding please?
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Can be reproduced
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

Overall, I think this is a very good paper. If the authors could offer some useful feedback, that would be lovely.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

See the strength
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

All reviewers agree that this is a solid and interesting paper. The authors introduce a transformer block designed for lesion detection tasks that can act as a plug-and-play addition to different architectures. Experiments were thorough and well done.

Authors are encouraged to address reviewer comments as much as possible, especially request for more clarity and explanation of rationale.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

1

Author Feedback

We would like to express our sincere gratitude to the reviewers, Area Chairs, and Program chairs for their valuable comments. In the following, we provide point-to-point responses to the main comments from reviewers.

1.More justifications on the design of SATr block are needed. We have tried a lot of reasonable q-k-v combinations, and most of them yield weak performances. For example, using key-slice as query and all-slice as key and value only bring 0.1~0.5% improvement in FPPI=0.5 over baseline models, The strong the baseline model is, the weak the improvement it can bring. Our CAM experiments show that using key-slice as the query can easily bring more false-positive predictions. These results were not reported due to the paper limitation, and we have added these results and more analysis to the final version of our paper.

2.Classication activation maps. The CAM we used is designed for detection methods, we will add more analyses of our CAM results in the final version of our paper.

3.The analysis of model size, and flops, are not provided. we have added these analyses to the final version of our paper.

4.Typo/grammar errors. We have corrected the typos/grammar pointed out in the comment and go through our manuscript again very carefully.

An algorithm flow or pseudo-code is needed. We will add an algorithm flow in our final version.

back to top

SATr: Slice Attention with Transformer for Universal Lesion Detection