Authors

Wenjie Liu, Hu Ding

Abstract

The Computed Tomography (CT) for diagnosis of lesions in human internal organs is one of the most fundamental topics in medical imaging. Low-dose CT, which offers reduced radiation exposure, is preferred over standard-dose CT, and therefore its reconstruction approaches have been extensively studied. However, current low-dose CT reconstruction techniques mainly rely on model-based methods or deep-learning-based techniques, which often ignore the coherence and smoothness for sequential CT slices. To address this issue, we propose a novel approach using generative adversarial networks (GANs) with enhanced local coherence. The proposed method can capture the local coherence of adjacent images by optical flow, which yields significant improvements in the precision and stability of the constructed images. We evaluate our proposed method on real datasets and the experimental results suggest that it can outperform existing state-of-the-art reconstruction approaches significantly.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43999-5_50

SharedIt: https://rdcu.be/dnww3

Link to the code repository

https://github.com/lwjie595/GANLC

Link to the dataset(s)

https://drive.google.com/drive/folders/1gKytBtkTtGxBLRcNInx2OLty4Gie3pCX

https://opendatalab.com/RIDER_Lung_CT

Reviews

Review #1

Please describe the contribution of the paper

Authors propose a novel method for low-dose CT reconstruction using generative adversarial networks (GANs) with local coherence, namely GAN-LC. GAN-LC takes into account the coherence and smoothness of sequential CT slices through optical flow. Based on traditional GANs, GAN-LC adds a component called optical flow estimator, which captures the local coherence by the network architecture of FlowNet. Authors validate the method on two types of dataset. By varying parameters of projections and noise, GAN-LC can outperform several existing algorithms in most cases in terms of PSNR and SSIM.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Clarity : The paper provides a detailed description of the proposed approach and its implementation, including the architecture of GAN-LC model. It also clearly states the assumptions. Novelty : GAN-LC considers the neighborhood correlations among the 2D slices. Authors apply the idea of optical flow to CT reconstruction to capture neighbor slices coherence, which has not been investigated before. Extendability : GAN-LC can be applied to a wide range of applications, such as segmentation problems, 4D reconstruction, etc.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Limited applicability : GAN-LC relies on the assumption of local coherence between consecutive CT images, which may not always hold true in practice due to a variety of factors. These factors may include variations in the position of the body, differences in the thickness of the CT slices, motion artifacts caused by patient movement or breathing, and variations in tissue density and composition due to disease or injury. Limited validity : GAN-LC cannot always outperform all the compared algorithms in all cases, especially in the case of fewer projections and louder noise. Limited comparison to state-of-the-art : Authors compare GAN-LC to several existing algorithms, but it may be possible that other methods not included in the comparison may perform better.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Authors provide detailed information on the datasets, including data sources. The proposed method is clearly described. All relevant results are reported. However, it may still be challenging to reproduce the experiments without the source code.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Authors provided the network architecture for the optical flow estimator, but there is no information about architectures of the generator and the discriminator. The training parameters are not mentioned except the number of epochs. For future work, GAN-LC needs improvements to completely outperform other algorithms, especially in the case of fewer projection angles and more noise. More state-of-the-art algorithms need to be considered for comparison to validate the proposed approach.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper integrates the idea of optical flow into CT reconstruction, which makes sense based on several assumptions. However, the proposed approach cannot always perform better than the compared algorithms. While the idea of GAN-LC is interesting, there are still some weaknesses in terms of performance.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

The authors propose a novel approach for low-dose CT reconstruction using generative adversarial networks with local coherence. The method takes advantage of the inherent continuity of lung, allowing local coherence to capture small deformations and structural differences between consecutive CT slices using optical flow.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The incorporation of local coherence enhances the model’s ability to capture small deformations and structural differences between consecutive CT slices through optical flow, considering the inherent continuity of the human body.
- Clinical feasibility: The paper’s proposed method for low-dose CT reconstruction has the potential to reduce radiation exposure to patients while maintaining high-quality imaging.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Insufficient explanation of local coherence: While the concept of local coherence is central to the novelty of the proposed method, the paper could provide a more detailed and intuitive explanation of how local coherence is incorporated into the GAN framework, and how it was useful comparing to non local coherence. This would help readers better understand the unique contribution of the proposed GAN-LC method.

Clinical validation: Although the paper demonstrates the clinical feasibility of the proposed method, it would benefit from a more in-depth analysis of its impact on clinical decision-making. For instance, the authors could have assessed the method’s performance in detecting specific lesions or its influence on radiologists’ diagnostic accuracy.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Confirmed that reproducibility checklist is completed and accurate.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Local coherence explanation: Please provide a more detailed explanation of the local coherence concept, its importance, and how it is incorporated into the GAN framework. This will help readers understand the main novelty and contribution of your work better.
Clinical impact: Since the focus of MICCAI is increasingly on clinical translation, please consider discussing the clinical implications and potential real-world impact of your method, including how it might affect clinical workflows or patient outcomes.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The experimental design is robust. Image quality evaluation is strong on multiple dataset. All demonstrated potential for clinical feasibility are its major strengths. However, lack of clinical translation based evaluation and the under explained concept of local coherence are its main weaknesses. Given the potential impact on real-world clinical workflows and the novelty of the approach, I recommend acceptance, but encourage authors to address these concerns in the revision.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

This paper presents a generative adversarial network (GAN) with local coherence module for low-dose CT reconstruction. The proposed local coherence module utilizes optical flow to capture the temporal coherence between adjacent CT slices.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

• This work incorporates the local coherence to improve CT image reconstruction. • The proposed deep learning model, GAN-LC, shows improvements comparing to existing methods.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

• The formulation of the paper is confusing. For example, the framework of the GAN-LC (figure2) is not informative. According to figure 2, both ‘optical flow’ and ‘generator’ modules have two input. However, in the context, the inputs of each module are not clearly explained. • The experiment design needs to be improved. For example, with the Mayor-Clinic dataset, the authors use data from 9 patients for training and remaining 1 patient for testing. This has a potential risk for overfitting. • For the evaluation part, cross-validation needs to be performed to reduce the risk of overfitting. • Ablation studies need to be performed in order to confirm the combination of local coherence and GAN really brings the improvement.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

This work is reproduciable.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

• Please make sure the figures are informative for the audience. • When designing experiments, please think thoughtfully.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

3
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This work tries to improve CT reconstruction in the context of low-dose CT. However, the description of the proposed method is not clear. Besides, the experimental design needs to be improved. Moreover, the ablation studies are needed to justify the design of the proposed method.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

Current techniques for low-dose CT reconstruction mainly depend on either model-based methods or deep learning-based techniques, which tend to disregard the coherence and smoothness between sequential CT slices. To resolve this challenge, a novel approach utilizing generative adversarial networks (GANs) with improved local coherence is proposed.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

Namely, the neighborhood correlations among the 2D slices are often neglected, which could potentially impact the reconstruction performance in actual use. This insight may contribute to the further development of related research topics.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. How does local coherence affect image artifacts? It is suggested that more visualizations of the results of local coherence be displayed.
2. Changes in brightness can be captured by optical flow networks, but more details about the design of these networks have not been displayed (including in the appendix).
3. The author mentions surpassing the most advanced methods, however, the baseline experiments shown are from 2021. It is suggested that more recent methods be compared.
Please rate the clarity and organization of this paper

Satisfactory
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The supplementary material does not seem to provide enough experiments to support the conclusions mentioned in the manuscript, and also the details of the network structure are missing.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
1. The impact of local coherence on image artifacts is not clear. It is recommended that the results of local coherence be visualized more extensively to better understand this relationship.
2. Optical flow networks have the capability to detect changes in brightness, but the design of these networks is not well documented, including in the appendix. It would be beneficial to have more information on this topic.
3. The author claims to have surpassed the state-of-the-art methods, however, the baseline experiments presented are from 2021. It is recommended that a comparison with more recent methods be included.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The optical flow network mentioned in the manuscript is fascinating for this research topic, and more details are shown that would be beneficial for this work to be discussed more.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The majority of reviewers lean towards the acceptance of the paper. The meta reviewer carefully reads the reviewers’ feedback and agrees with their scores. The authors should address the mentioned issues and provide a more comprehensive evaluation of their proposed method’s performance. For example, conducting an ablation study and providing clarity in the formulation and the applicability of the method.

Author Feedback

We are very grateful for the helpful comments. Below we respond to the reviewers’ questions. The applicability of our method and clarity in the formulation are explained for Reviewer 3 and 4, respectively. Also, we will conduct an ablation study and more comprehensive evaluation of our proposed method in the final version.

Reviewer 1: —More information about the network architecture Thanks for the comment, and we will provide more details about the architecture and the training parameters in the final version.

—“For future work, GAN-LC needs improvements …” More experiments for the case of fewer projection angles are shown in the supplement. Also, we will compare our method with some more recent works in the final version, such as “DuDoTrans: Dual-Domain Transformer for Sparse-View CT Reconstruction” that was published in 2022.

Reviewer 2: —“The impact of local coherence on image artifacts …”
“Local coherence” indicates the correlation degree between adjacent images of a tissue, which can be captured by the optical flow. The local coherence can efficiently help the model to detect large changes (e.g., artifacts). We will add more explanations and visualization in our paper.

—“Optical flow networks have the capability to detect changes …” The optical flow network architecture is mainly derived from FlowNet [8]. After warping one image into another, the brightness difference between the two images is used as the loss for the optical flow network training, which can detect the changes in brightness. More details on this topic will be added to our paper.

—“ The author claims to have surpassed the state-of-the-art methods …” Please refer to our response to the second question of Reviewer 1.

Reviewer 3: —“Local coherence explanation …” Please refer to our response to Reviewer 2.

—“Clinical impact …” Thanks for this suggestion. The reconstructed CT images are used to assist doctors for diagnosing diseases (e.g., detecting specific lesions). We are currently collaborating with the medical imaging center of our local hospital, which has more than 10 thousands patients every year. In future, our method will be evaluated on these real-world CT images in clinic.

Reviewer 4: —“The formulation of the paper is confusing …” We briefly explained the inputs of optical flow’ and generator ’ modules in section 3 (the last paragraph in page 4) and Equation (6) in page 5. Some details are also shown in the Optical flow estimator’ subsection and Generator’ subsection in page 5. We will add more information about our formulation to our paper and thanks for this question.

—“The experiment design needs to be improved …” Thanks for this question. To evaluate the performance of our model, we also consider another dataset RIDER for testing on the model trained from the “Mayor-Clinic” dataset; the RIDER dataset contains 32 patients with 15419 slices. We believe this experiment can reduce the risk of overfitting to some extent. We will also add more experiments to our paper later.

—“For the evaluation part, cross-validation …” We agree with the reviewer that cross-validation is helpful to reduce the risk of overfitting, and we will add this part to our experiments.

—“Ablation studies need to be performed …” Thanks for this suggestion, and we will consider the ablation studies to improve our experiments. We have conducted some preliminary experiments this week. We consider the experiments on GAN without local coherence, and it performed worse than GAN-LC. For example, when N_v=200, N_d=400, sigma=0, the PSNR of ablation study is 36.876 and SSIM is 0.892. Also, when N_v=150, N_d=300, the PSNR is 35.970 and SSIM is 0.885. On the other hand, the results of GAN-LC in our paper are 39.548 and 0.950 for the first parameter setting. For the second case, the results of GAN-LC are 36.542 and 0.899. We will add more detailed ablation studies to our final version.

back to top

Solving Low-Dose CT Reconstruction via GAN with Local Coherence