List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yihua Sun, Qingsong Yao, Yuanyuan Lyu, Jianji Wang, Yi Xiao, Hongen Liao, S. Kevin Zhou
Abstract
Digital chest tomosynthesis (DCT) is a technique to produce sectional 3D images of a human chest for pulmonary disease screening, with 2D X-ray projections taken within an extremely limited range of angles. However, under the limited angle scenario, DCT contains strong artifacts caused by the presence of ribs, jamming the imaging quality of the lung area. Recently, great progress has been achieved for rib suppression in a single X-ray image, to reveal a clearer lung texture. We firstly extend the rib suppression problem to the 3D case at the software level. We propose a Tomosynthesis RIb SuPpression and Lung Enhancement Network (TRIPLE-Net) to model the 3D rib component and provide a rib-free DCT. TRIPLE-Net takes the advantages from both 2D and 3D domains, which models the ribs in DCT with the exact FBP procedure and 3D depth information, respectively. The experiments on simulated datasets and clinical data have shown the effectiveness of TRIPLE-Net to preserve lung details as well as improve the imaging quality of pulmonary diseases. Finally, an expert user study confirms our findings. Our code is available at https://github.com/sunyh1/Rib-Suppression-in-Digital-Chest-Tomosynthesis.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_66
SharedIt: https://rdcu.be/cVD7m
Link to the code repository
https://github.com/sunyh1/Rib-Suppression-in-Digital-Chest-Tomosynthesis
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a rib suppression and lung enhancement network (TRIPLE-Net) for chest tomosynthesis. This is the first work on rib suppression using deep learning in tomosyntheses.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is very well-written and easy to follow. The physics of DCT is clearly explained followed by the logic of the design. Good job!
- The method is a dual-domain method that first learns in the projection domain, then learns in the reconstruction DCT domain.
- The experimental results on simulation data show clear quantitative and visual improvements.
- Additional reader study is performed to validate the method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The network design aims to predict the I_delta. What if the network predicts the I_rs and then reconstructs and then refined with another 3D network? That would also be a rational design.
- Table 1, is there any statistical difference in the improvements?
- One of the major concerns the reviewer has is the real data study. The authors only collected 5 real data in total for the reader study, which should be expanded.
- Following the above comment, it is clearly seen in Table 2 that the proposed method does not really outperform the baselines on average for the clinical real data set. This means the method cannot generalize well to the real-world dataset, unlike the 3rd and 4th columns where it performs reasonably well on the simulation dataset since there is no domain shift. To alleviate the domain shift and to use this method, it is required to construct/acquire real data with and without rib which is impossible. So how can this method be deployed in real-world scenarios still remains unclear.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please see above
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well-written, the method is interesting, and the application is important. However, the major concern lies in how can this method be deployed in real-world scenarios with real data with and without rib, and how to achieve better performance on the real clinical dataset.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors address my comments.
Review #2
- Please describe the contribution of the paper
The paper presents a novel deep learning-based rib suppression approach in DCT by modeling the rib artifacts. The method is based on three subnetworks to model 2D and 3D rib components, and merging the results to make the final rib suppressed DCT prediction. The proposed TRIPLE-Net is validated against DCT dataset simulated from CT as well as clinical dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The idea of modeling rib artifacts in DCT with information from both 2D and 3D domains is novel and interesting. The evaluations with the simulated data as well as the clinical study also demonstrated the feasibility of this idea.
-
The paper is fairly well written and the exposition of the results is good.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The main concern is the actual evaluation on the clinical dataset : small datasize.
-
Details on the networks and training configurations are missing.
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Without the network and training specifics, the work is not straightforward to reproduce.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
This is an interesting study. However, more analysis is required, especially on the network’s robustness to varying sizes of patient anatomy and disease types.
-
The paper mentions M2D and M3D are trained separately before F; But it would be interesting to see how the joint training of the three networks works.
-
It looks like the proposed TRIPLE-Net is almost on the same level as M3D (PSNR), both Doctors A and B rated M3D predicted images higher. The authors should add an explanation. Perhaps, a domain adaptation approach would be feasible with a larger clinical dataset.
-
To make it more reliable, the simulated DCTs should be quality checked.
-
The paper doesn’t report the per scan execution time of the proposed model.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Interesting idea, writing and organization of the paper
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The paper proposed TRIPLE-Net with three subnetworks (projection-net, volume-net, aggregation-net) that can model Rib in Digital Chest Tomosynthesis as a linear suppression. It can be trained in hard constraints from 2D and 3D domains. Therefore, such downstream tasks as pulmonary analysis will be beneficial. The improvement over RSGAN is to have a Filtered Back Projection that resembles the 2D images to 3D volume and captures more accurate rib modeling. A user study that involves two doctors is carried out, which shows that the proposed model gives a slightly better rib disentanglement.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed method leverages 3D information with the FBP operator to remove the rib artifact from DCT images consistently. The paper is well written and backed up with a clear description of mathematical image formation. Experiments were carried out qualitatively and quantitatively. A user study shows that two doctors appreciate higher TRIPLE-Net ratings than its ablations and other state-of-the-art methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Data preparation is unclear: “in-paint rib mask with surrounding tissues” lowers the confidence of 2D projection. There is no tissue inside the 3D rib regions, and if someones want to eliminate rib artifacts, they need to set the maximum transparency of those regions. Hence, final pixels compositing will not count the presence of ribs. Filling the 3D rib regions with interpolation from surrounding tissues will cause inaccurate tissue appearance, leading to unreliable analysis.
The role of the merging module (aggregation-net) needs further discussion. One counter-example is that the difference of two inputs to the \mathcal{F} should be minimal, which results in V^M_{\alpha, \Delta} being the average of these two inputs. It leads to having the \mathcal{F} model will be over-parameterized.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility is feasible so that a skillful graduate student can replicate it.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The CT data is orthographic on sagittal and coronal orientations but entirely perspective on axial planes. However, once the X-ray beam source and detector rotate a complete circle around the object, the integration compensates both directions and turns the collected voxels equidistant on axial planes. Please clarify whether the projection and simulated datasets were made on an orthographic or perspective scene? Experiments on simulated 3D CT datasets need to explicitly clarify whether the full HU scale is used and normalized to (0, 1), or a specific window is applied to the data. This matter will affect the evaluation metrics L1, L2, and the intensity range of 2D projections.
On the other hand, L1, L2 norms, and PSNRs metrics are somehow related. Perhaps, other interesting metrics would be performing the maximum-intensity projection (MIP) on 3D rib segmented data compared with the 2D rib segmentation artifacts in Dice Score.
Assuming that the proposed method can separate the rib artifacts in the image domain, the bone shadows still presented, given the results in Figures 3 and 4. It poses another question: Is the artifact well-observed if one performs gradient transform on the rib-suppression images?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
In conclusion, revealing the nodules covered by rib shadows on a single or limited view of XR (such as DCT) is another highly ill-posed problem. The proposed solution addressed it in a way that separates the rib artifacts. There have some unclear points that can be minorly clarified to improve the current exposition of the paper.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Given the rebuttal addressed most of my comments within the limit of 4000 characters, I would like to keep my evaluation as it is. It would be better if the author and show visually the comparison between tissue interpolation and maximum transparency of the rib regions (perhaps in a longer version or a journal).
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All reviewers find the paper well-written with reasonably design and decent novelty. Major comments that will need to be addressed in the rebuttal include 1) the small datasize of real images, as well as that on real clinical images, the performance gain against baselines does not seem significant; 2) the initition of the current method settings against other alternative ways; and 3) more details/discussion of results, e.g. bone replacements and shadows
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
We sincerely thank the reviewers for their valuable suggestions and comments.
Q(R1,R2): Small clinical dataset(CD); TRIPLE-Net doesn’t outperform M3D in CD in Table 2. A: 1) DCT is a novel technique with significant advantages, but not yet widely deployed in clinics. Thus, there is a lack of public, large-sized clinical dataset. One of the current limitations of DCT is exactly the presence of rib artifacts, which hampers the imaging quality and motivates us to design TRIPLE-Net.
2) Given the insufficiency of clinical data, the user study can not demonstrate the sheer superiority of TRIPLE-Net in Table 2. But it did demonstrate that DL-based rib suppression methods can improve the DCT imaging quality for better lung detail from the clinical perspective, compared with FBP. Besides, we extended the rib suppression problem from 2D to 3D scenarios like DCT, and doctors all agree that 3D methods outperform 2D methods.
3) As R2 said, “domain adaptation would be feasible with a larger clinical data”. We are willing to do so and are cooperating with hospitals for more data.
Q(R1): Predict I_\Delta or I_rs? A: CNNs tend to learn low-frequency information first. Directly predicting I_rs may lose lung details. Predicting I_\Delta is an easier task for the networks, and predicting I_rs = I - I_\Delta^P can preserve original lung textures with higher quality (same for V_\Delta vs. V_rs).
Q(R1,R2): p-value in Table.1? TRIPLE-Net is at the same level as M3D(PSNR)? A: PSNR does not increase linearly, and the p-value for PSNR against M3D in paired t-test is <0.027. TRIPLE-Net outperforms other methods in all other metrics with p-value<0.001.
Q(R2): Networks and training configurations? A: The work is easy to reproduce(R1,R3). We will open the source code and configurations if the paper is accepted.
Q(R2): Network’s robustness to varying lung size and disease types? A: We evaluate the robustness on the test set to varying disease types ({LI; RF; MR; NR}, while MR and NR are not in train set) and lung size, which are split by intervals [0, 2.8, 3.4, 4.1, 4.7, 5.4, 6.1, \infty] dm^3, resulting in 7 bins (b1-b7). Metric: L1(e-4); \alpha=30;
Method b1 b2 b3 b4 b5 b6 b7 & LI RF MR NR RSGAN 82 82 84 82 83 92 95 & 82 77 86 90 –M2D– 72 71 74 72 74 81 85 & 72 67 75 81 –M3D– 65 65 66 63 64 70 72 & 63 62 67 69 TRIPLE 46 45 47 45 46 50 52 & 45 44 47 50 TRIPLE-Net outperforms other methods with all p-value(s)<0.001.
Q(R2): Training jointly(JOT) vs. separately(SEP)? A: JOT needs online FBP. Training M2D/M3D/F/JOT needs 25/392/388/4456 ms/step. SEP is more efficient since we have ground truth for 2D&3D.
Q(R2): Quality check of the simulated DCT. A: It’s reliable with high quality, confirmed by the doctor.
Q(R3): In-paint vs. set maximum transparency of rib mask when simulating I_rs. A: If maximum transparency is set, visually there is an abrupt intensity jump around the rib area in I_rs. In-painting the rib mask leads to a more realistic I_rs simulation with consistent intensity in the lung area, as reported in RSGAN[12].
Q(R3): Is F over-parameterized than taking an average(AVE) of “the 2 inputs”(2I) to F? A: It is certainly not. L1(e-4) for [F, AVE] with \alpha=30/15 is [47/69, 60/83], and F outperforms AVE. R3 said: “The difference of 2I should be minimal”, but it is not equivalent that 2I converge to the ground truth in the same way. We observe essential difference between them. M2D+FBP and M3D incorporate different prior knowledge and have different advantages (Section 2.2, 3.2). So F is necessary.
Q(R3): The simulated datasets were made on an orthographic or perspective scene? A: The simulation matches the clinical DCT setup, which is in a limited angle cone-beam (perspective scene) geometry (Section 2.1, Fig.1).
Q(R3): Preprocessing for HU values? A: HUs are converted to attenuation coefficients f(E), as [a]. I and L1/L2 are calculated in the scale of f(E). [a] DeepDRR–A Catalyst for Machine Learning in Fluoroscopy-guided Procedures
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Reviewers found the rebuttal to be reasonable in addressing their questions.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors did a good job of addressing the comments raised by the reviewers. The unanimous decision by the reviewers is for acceptance of the work. The authors should include the important details provided in their rebuttal in the final version of the paper or in the supplemental material.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper address a challenging question of suppressing ribs in Digital Chest Tomosynthesis (DCT). As the proposed approach is novel, clinical data can only be derived through simulations on such images. The authors relay on two readers who assess the different methods to perform the task. TRIPLE-Net, the method proposed by the authors, seem to perform well in the sense of improving the DCT imaging quality. That is, TRIPLE-Net seem to provide better clinically meaningful lung details, compared with the other alternatives.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
10