Authors

Chen-Han Tsai, Yu-Shao Peng

Abstract

Lung nodules can be an alarming precursor to potential lung cancer. Missed nodule detections during chest radiograph analysis remains a common challenge among thoracic radiologists. In this work, we present a multi-task lung nodule detection algorithm for chest radiograph analysis. Unlike past approaches, our algorithm predicts a global-level label indicating nodule presence along with local-level labels predicting nodule locations using a Dual Head Network (DHN). We demonstrate the favorable nodule detection performance that our multi-task formulation yields in comparison to conventional methods. In addition, we introduce a novel Dual Head Augmentation (DHA) strategy tailored for DHN, and we demonstrate its significance in further enhancing global and local nodule predictions.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_67

SharedIt: https://rdcu.be/cVD7n

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper proposes an improved architecture for nodule detection. The implementation of multi-task in the architecture improves both performance on classification and detection. The dual head structure is reflected in both the overall architecture and a data augmentation approach.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper has been well laid out, clearly presented, and with improved results. Ablation study has been conducted to study the effect of the proposed modifications to the original Faster R-CNN approach.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

In multi-task training, there is an assumption that the training data contains images that may or may not have nodules. For those without nodules, the second task on nodule detection will not contribute to the model training. If most images do not contain nodules, what influence will it has on the overall model performance?
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Reproducibility of the paper is high.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

It will be interesting to have a discussion on extending the proposed DHN and DHA to another stream of object detectors, namely the YOLO family of models.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper is well presented, organized, with enough details for reproduction. The proposed contributions are supported by experimental results. Ablation study is conducted to study the effects of proposed improvements. A nice paper for reading.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

A multi task pulmonary nodule detection algorithm for chest X-ray analysis is proposed.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The authors use a double headed network to predict the presence of a global label indicating the presence of nodules, and use a local label to predict the location of nodules，In addition, the authors propose a new double headed enhancement (DHA) strategy, which proves the importance of DHN in further improving the prediction of global and local nodules.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The method selected in this paper is resnet-18. The author should prove why resnet with 18 layers is selected in the experimental part. In addition, there are few methods compared in the experimental part.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Although the author does not give the source code, the network model is not complex and has certain reproducibility, but because all the experimental parameters are not fully explained, the reproduction results may be biased. Of course, it’s best to hope that the author publishes the source code.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

It is suggested to add more experimental parts and describe the implementation details of the model as clearly as possible.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

In this paper, multi task detection of thoracic pulmonary nodules is a good work. Dual head network is used for more effective detection on the model. However, the detailed description of the method is not clear enough, and the experimental part is not perfect.
Number of papers in your stack

5
What is the ranking of this paper in your review stack?

3
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

The authors propose a multi-task lung nodule detection algorithm to reduce the false-negative rate in this manuscript. The dual-head network (DHN) is proposed to predict the global-level label of nodule presences as image-level prediction and the local-level label of precise locations. A dual-head augmentation is proposed for DHN training. The experiments on the NIH dataset demonstrate the effectiveness of the proposed model.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The paper is logically organized and very easy to read.
2. The paper proposed multi-task lung nodule detection to help the false negative detection in chest radiograph analysis. By learning global case-level prediction and local detection simultaneously, both tasks show performance improvements.
3. Dual-path augmentation is novel as the global and local paths may need to adopt different augmentation strategies.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The reason for properties of pulmonary nodules need adopting deformable convolutions is unclear
2. How to use the two results is unclear.
3. The experiments comparison with state-of-the-art on pulmonary detection performance is not shown.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The source code will not be made available. The paper provides enough details to reproduce the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Major:
1. ‘Considering the properties of the pulmonary nodule, we propose to use deformable convolutions [6] and the gIOU loss’ – this needs more details of what properties of the pulmonary nodule lead to the use of deformable convolutions. The authors mentioned: ‘Applying deformable convolutions allows more dynamic focus on particular regions in the image where the size of the nodule might be small .’ We know that FPN can help leverage the nodule of various sizes, but how can the deformable convolution help detect the nodules of small sizes?
2. The dual-path network contains global classification and local location prediction with a confidence score. Is it possible for the framework predicts several bounding boxes from the local path but with the negative prediction from the global path, or positive predictions from the global path but no bbox generated from the local path? How to deal with this? Generally for the nodule detection, we have a prediction of the lesion and the corresponding location. What is the exact output of this network, and how to use the result of this paper to guide clinical practice?
3. In the experiments, this paper compared the CONAF [19]. Why are only the classification metrics shown for CONAF? Additionally, in the introduction, both references [13] and [19] are discussed, so what about the performance compared to Li et al. [13]?
4. It is reasonable that the author compares the performance with similar attempts in the experiments by comparing the CONAF [19] in Table 1. However, by reviewing the purpose of this paper – ‘identify pulmonary nodules on chest radiographs’, should this paper also compare the nodule detection performance with the state-of-the-art methods [1, 20, 26] (global) or [10,12,15,17,23] (local) cited in the introduction?
Minor:
1. A typo in “default setup ‘utilizs’ a Smooth L1 Loss “ of the last paragraph in section 2 on page 3.
2. It is recommended that Table 1 be placed near the corresponding text on page 7.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper provides a novel solution to improve the detection accuracy by adopting the case-level classification network with the lesion-level detection network. Instead of using lesion-level detection results, the authors provide an alternative way to implement global classification results. The performance also shows the improvement in both the classification result and detection results. However, there are some explanation needed for the design of this network and the design of the experiments.
Number of papers in your stack

7
What is the ranking of this paper in your review stack?

3
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

This paper received three consistent positive reviews or accepts and AC initially checked the quality of this submission. The overall feedback are positive for acceptance but there are a lot of detailed questions and clarity issues to be further addressed for the final version. This paper is recommended for provisional accept.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

3

Author Feedback

Reviewer #1

Thank you for the generous feedback regarding our paper. The idea of extending DHN and DHA for single-stage object detectors would certainly be of great interest for future works.

In the event that most images do not contain nodules, it would still be possible to train the DHN. The images without nodules still incur a loss from the local head during training if false positives are predicted. The trained model would most likely perform favorably at suppressing false positives rather than the intended nodule detection.

Reviewer #2

Thank you for your positive rating and detailed review.

The backbone of our DHN was selected empirically. Specifically, we conducted a series of experiments comparing various CNN architectures from the ResNet family, and we observed slightly better performance as model size increased. However, the training time increased significantly for larger models. Hence, we selected the smallest model from the ResNet family, the ResNet-18, to serve as a lower bound for DHN performance throughout our experiments.

As suggested, we will update the paper with more details on the selection of the DHN architecture and the experiments we have conducted in the camera-ready version.

Reviewer #3

Thank you for the positive recognition and valuable comments. We will update the camera-ready manuscript with more detailed explanations regarding our algorithm and the experiments performed.

The reviewer is correct in that the FPN helps leverage nodules of various sizes. It does so by encoding multi-resolution features so that the ROI Head can select the optimal feature map to crop. However, the feature map that feeds into the FPN has traditionally been extracted by feature extractors utilizing standard convolutions (with fixed strides). Using deformable convolutions allows the backbone CNN to learn adjustable receptive fields to sample the feature maps. The adjustable receptive field provides the model with an advantage to leverage “in context” (granular details within the nodule) or “out of context” (ancillary) features that benefit the predictions (see Figure 2). Together with the FPN, incorporating deformable convolutions into the DHN allows the extraction of more expressive features highlighting nodule presence.

We’re glad the reviewer pointed out how to combine the global classification and local location prediction. As we developed the DHN, it soon became apparent that a “clinically relevant” model requires both heads to be coherent with one another regarding predictions. Throughout our experiments, we observe that both heads agree on 80% of the predictions, and the local head is usually more sensitive than the global head. There are several cases to consider and we intend to leave this topic as future work.

We only showed the classification metrics for CONAF since its classifier head prediction was the only output that can be directly compared. Although CONAF has a dual-head architecture (i.e., the one for classification and the second for localization), the localization map from the second head is too coarse (resolution 28x28) to extract a precise bounding box even if upsampled. In the original paper, the localization predictions were considered true positives if they overlapped greater than 25% of the ground truth. In our paper, true positives were considered only for an IOU greater than 40% between ground truth and predictions. In contrast, our metrics are much more strict and require precise predictions of nodule locations which an upsampled score map is unable to provide.

Prior to developing the DHN and DHA algorithm, we experimented with several global and local methods. The results for the best performing candidates in the global and local methods will be added to the camera-ready version.

back to top

Multi-Task Lung Nodule Detection in Chest Radiographs with a Dual Head Network