Authors

Rui Xu, Yong Luo, Bo Du, Kaiming Kuang, Jiancheng Yang

Abstract

Convolutional neural networks (CNNs) have been demonstrated to be highly effective in the field of pulmonary nodule detection. However, existing CNN based pulmonary nodule detection methods lack the ability to capture long-range dependencies, which is vital for global information extraction. In computer vision tasks, non-local operations have been widely utilized, but the computational cost could be very high for 3D computed tomography (CT) images. To address this issue, we propose a long short slice-aware network (LSSANet) for the detection of pulmonary nodules. In particular, we develop a new non-local mechanism termed long short slice grouping (LSSG), which splits the compact non-local embeddings into a short-distance slice grouped one and a long-distance slice grouped counterpart. This not only reduces the computational burden, but also keeps long-range dependencies among any elements across slices and in the whole feature map. The proposed LSSG is easy-to-use and can be plugged into many pulmonary nodule detection networks. To verify the performance of LSSANet, we compare with several recently proposed and competitive detection approaches based on 2D/3D CNN. Promising evaluation results on the large-scale PN9 dataset demonstrate the effectiveness of our method. Code is at https://github.com/Ruixxxx/LSSANet.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_63

SharedIt: https://rdcu.be/cVD7j

Link to the code repository

https://github.com/Ruixxxx/LSSANet

Link to the dataset(s)

https://mmcheng.net/sanet/

Reviews

Review #2

Please describe the contribution of the paper

The paper proposed a long short slice-aware network for pulmonary nodule detection, which have the ability to capture long-range dependencies. This is a relatively painful problem in this field, and it is very valuable.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The method is relatively new.
2. The logic is clearer.
3. The experiment is relatively sufficient.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The description of the formula in the paper is not clear enough.
2. Full writing and abbreviations in the paper are confusing and should be consistent. If the abbreviation has been abbreviated in the front, please keep the abbreviation in the back, instead of using the full letter and the abbreviation back and forth.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The paper is very valuable, but requires more information from the authors to illustrate reproducibility. Such as code, models, etc.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1. Full writing and abbreviations in the paper are confusing and should be consistent. If the abbreviation has been abbreviated in the front, please keep the abbreviation in the back, instead of using the full letter and the abbreviation back and forth.
2. There are English spelling errors in the paper, please correct them carefully.
3. The method proposed in this paper is based on the pre-training model of the current best method as the benchmark model. It cannot be clearly stated whether it is the advantage of the method in this paper or the overfitting of the original method.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The research of this paper is very valuable and is a hot topic in the field. Although the experimental results are relatively good, the method used in this paper is based on the pre-trained model of the current state-of-the-art method. Papers will not be accepted unless the authors provide code or models to illustrate the reproducibility of the method.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper proposes a LSSANet, which has the ability to capture long-range dependencies for pulmonary nodule detection. This network not only reduces the computational burden, but also keeps long-range dependencies among CT slices. Experimental results show that this network has convincing performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. Effective design and comprehensive experiments: authors compare their method with several proposed methods, and LSSANet achieves convincing performance.
2. Clear writing: it is easy to follow the writing.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. Some typos: Section 2.2 line 6: ‘5 our proposed LSSG blocks are integrated into the second and third stages of the ResNet50 encoder’. Table 1, LSSANet does not achieve the best performance, but authors still bold 89.87, which may mislead readers. Please examine typos carefully.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The reproducibility of this paper is good.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

Generally speaking, this paper does not have many weaknesses. The motivation and paper writing are all good. It would be better if the authors could be more careful about typos.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The motivation and paper writing are all good.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
As the two reviewers pointed out, this work is well motivated, the method has decent novelty, and authors performed comprehensive evaluations and comparisons. Overall the work is well presented, although it will be best if the code can be released to reveal some detailed settings. I have some further comments and questions in addition to those raised by reviewers:
- I don’t quite get the idea of “long-distance” will help “the elimination of continuous pipe-like structure”. I understand that slice-wise nodules and blood vesssels can appear similar, but I think with a few slices’ information they can be distinguished (this of course depends on slick thickness). Since this is a critical motivation for the long-distance part, a few qualitative examples will be helpful to illustrate the contribution from long-distance grouping (to accompany Table.2)
- “LSSG blocks are integrated into the second and third stages of the ResNet50 encoder”, why these two? will other stages help?
- “By doing so, the compact non-local operation can capture richer long range dependencies among any positions and any channels”, this may need some result support / ablation.
- How to pick the optimal G paramter depending on physical space? How sensitive is the proposed method to G?
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

2

Author Feedback

We sincerely thank all reviewers for their valuable and constructive feedback. All concerns (such as adding more results and correcting typos) will be duly addressed in the final version. We will carefully organize and examine the final version of the paper following the comments. Furthermore, we would like to elaborate on some major concerns as follows:

Long-distance slice grouping (LSG) explores “long-distance” context information due to its capability of capturing the dependencies among slices far away from each other. Unlike short-distance slice grouping (SSG), which only focuses on distinguishing slice-wise nodules, LSG pays more attention on discriminating the nodules from continuous pipe-like structures (such as blood vessels) beyond a few neighboring slices, and these structures are common false positives. We will provide visualization of some pipe-like structures, which have high probabilities to be detected as false positives by the original SANet.

We integrate the LSSG blocks into the second and third stages of the ResNet50 encoder following SANet, which is a special case of our LSSANet when all the blocks added into the encoder are SSG blocks. This is to facilitate the fair comparison with SANet. Adding more LSSG blocks to other stages may not help improve the detection performance. This is because features of the first stage are low-level features, which are hard to capture the global information, while high-level features of the last/fourth stage may already contain much global information. In addition, non-local operations in the first stage may incur very high computational cost due to the large feature map size.

The compact non-local operation merges channel into position to obtain vectors for the next similarity computation, and the resulting similarity naturally explores richer long-range dependencies. We add an ablation study to compare the network equipped with the compact non-local operation (CNL-LSSG) with the one utilizing the original non-local operation (NL-LSSG). The results demonstrate that CNL-LSSG outperforms NL-LSSG by around 1% in terms of FROC score.

Our LSSG is implemented to capture the relationship between any positions and any channels in one group. Larger G leads to fewer consecutive short-distance slices or sparser long-distance slices in one group. Fewer short-distance slices may be appropriate for small nodule detection, but degrade the large nodule detection performance. Sparser long-distance slices may deal better with eliminating long pipe-like structures, but negatively affect the removing of short pipe-like structure. This is vice versa for smaller G. In this paper, we set 4 as the number of groups following SANet. We are conducting experiments on the sensitivity of our method w.r.t. different number of groups G, and will add the results in the final version.

We split the 8,796 CT images of the PN9 dataset into 6,037 for training, 670 for validation, and 2,089 for testing. There is no overlapping between the different subsets, and thus we believe that the advantage of our method does not come from the overfitting of the original method. To illustrate the reproducibility of our method, we release our model in the following link: https://drive.google.com/file/d/1DOXO_8d2sSdtoKNo94uieFMyQLdQuiUx/view?usp=sharing.

back to top

LSSANet: A Long Short Slice-Aware Network for Pulmonary Nodule Detection