Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Jie Jing, Tao Wang, Hui Yu, Zexin Lu, Yi Zhang

Abstract

The low-dose computed tomography (LDCT) denoising field is primarily dominated by supervised learning-based approaches, which necessitate the accurate pairing of LDCT and corresponding clean reference images (NDCT). However, obtaining real-world paired data is not feasible. Obtaining unpaired LDCT and NDCT data, on the other hand, is easy to do. Unsupervised methods have become increasingly popular for LDCT denoising. One commonly used method is CycleGAN,but it is both memory-intensive and challenging to train without the risk of collapse. To address these limitations, we propose a novel unsupervised method based on boosted contrastive learning (BCL), which requires only a single generator. Additionally, due to limitations in computing capability and memory size, most existing methods only focus on single slice, resulting in unstable results between slices. Our BCL-based method overcomes this problem. Our method astonishingly lows computing resources used and makes modifications to the original contrastive learning method, including weight optimization for positive-negative pairs and constraint of difference invariants. Our experiments demonstrate that our method outperforms existing state-of-the-art supervised and unsupervised methods in both qualitative and quantitative measures. Importantly, our framework does not require paired training data and is more adaptable for clinical application.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43907-0_23

SharedIt: https://rdcu.be/dnwcA

Link to the code repository

N/A

Link to the dataset(s)

https://www.aapm.org/grandchallenge/lowdosect/

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents a novel unsupervised method for low-dose CT (LDCT) denoising based on contrastive learning. The primary advantage of the proposed method is that it does not require paired training data, making it more suitable for clinical use. The authors have also made several modifications to the original contrastive learning method, which has led to state-of-the-art denoising results using minimal computing resources. The experimental results demonstrate the method’s superiority over existing supervised and unsupervised methods in both qualitative and quantitative measures.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper has several strengths, including:

    1. Novel unsupervised approach: The paper presents a novel unsupervised method for LDCT denoising, which significantly outperforms most existing unsupervised techniques and even surpasses some fully supervised methods. This demonstrates the potential of unsupervised learning in the field of medical image denoising, providing an alternative to traditional supervised methods that require labeled data.

    2. Computational efficiency: The proposed model is highly computationally efficient compared to common cycleGAN-based methods. This efficiency makes it more practical for real-world clinical applications, where quick processing times are essential for accurate diagnoses and patient care.

    3. Adaptability: The method’s adaptability to various generators allows it to be easily integrated into different clinical settings and imaging devices. This flexibility makes it a more versatile solution for LDCT denoising across a wide range of applications and environments.

    4. Improved inter-slice stability: The paper addresses the inter-slice instability issue often found in supervised methods, achieving stable output quality across slices. This consistency in denoising results is crucial for accurate medical image interpretation, leading to better clinical outcomes.

    5. Modified contrastive learning technique: The authors have made key modifications to the original contrastive learning method, including weight optimization for positive-negative pairs and constraint of difference invariants. These modifications enable the model to achieve state-of-the-art denoising results while utilizing low computing resources, further enhancing its practicality in real-world medical imaging applications.

    In conclusion, this paper presents a promising and novel unsupervised method for LDCT denoising based on contrastive learning. The method’s demonstrated superiority over existing methods, its low computational burden, and its adaptability for clinical use make it an important contribution to the field. Future work exploring self-supervised denoising on sinograms, as suggested by the authors, could lead to further advancements in the field of low-dose CT denoising.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Although this paper presents an effective denoising method for CT images, there are some weaknesses that need to be considered. These include:

    1. There are some small grammatical mistakes in the paper, some of them are listed below: In the following sentence, there is a missing space: “v, v^+, v^-$ are anchors, positive and negative pair.”. The corrected sentence should be: “v, v^+, v^-$ are anchors, positive and negative pairs.” In the following sentence, there is a missing word “to”: “The learning rate for optimizer2 for perceptual loss is set to $10^{-5}$ and is halved every 10 epochs.”. The corrected sentence should be: “The learning rate for optimizer2 for perceptual loss is set to $10^{-5}$ and is halved every 10 epochs.” In the following sentence, there is a missing word “the”: “Furthermore, reweighting mechanism has demonstrated its effectiveness in improving our model’s results.”. The corrected sentence should be: “Furthermore, the reweighting mechanism has demonstrated its effectiveness in improving our model’s results.”

    2. Dependence on the choice of generator: The performance of the proposed method is influenced by the choice of generator architecture. This reliance means that the quality of the denoising results can be affected by the selected generator, potentially requiring additional experimentation or fine-tuning to optimize performance across different imaging scenarios.

    3. Potential for overfitting: Although the unsupervised approach does not rely on labeled data, the model may still be prone to overfitting if the generator and discriminator are trained on a limited dataset or a dataset that is not representative of real-world LDCT images. This risk could negatively impact the model’s performance when applied to new and unseen data, limiting its overall effectiveness in practical applications.

    Overall, these weaknesses do not significantly detract from the main contributions of the paper but addressing them could help strengthen the validity and impact of the proposed model.

  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The paper’s methodology is easy to reproduce. Despite the absence of the code in GitHub, the authors have supplied comprehensive details regarding the model, dataset, and evaluation procedure, which significantly enhances the work’s replicability.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    A more comprehensive literature review and grammar editing will enhance the quality of the manuscript. This paper presents a promising idea and builds upon prior research. To fully showcase its potential, the authors could refine the theoretical approach and experiment with additional datasets.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    7

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is well-written, well-organized, and provides a clear and detailed description of the proposed method and its underlying principles. The use of figures and tables throughout the paper effectively supports the presentation and understanding of the proposed method and its experimental results.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    8

  • [Post rebuttal] Please justify your decision

    The authors responded adequately to the reviewers’ comments and have revised the paper accordingly. I recommend the paper is accepted as it presents an interesting approach and makes a good contribution for the CT imaiging field.



Review #2

  • Please describe the contribution of the paper

    This paper proposed an unsupervised LDCT denoising algorithm using constrastive learning and inter slice consistency between neighboring slice. They have also proposed a weighting scheme to weights the negative and positive pairs for improved learning.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. The paper address a practical problem of unsupervised denoising LDCT images.
    2. Paper is well organized and rationality behind the proposed method is well explained.
    3. The method performed comparably among the methods presented in the paper.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Their is a significant similarity between the method proposed in this paper and a previously proposed method [i] for LDCT denoising. Both the method leveraged the CUT method, however the author did not mentioned the previous study in the paper, and also difference between the previous method and proposed method is also not discussed.
    2. All the technical contribution proposed are leveraged from some other study( as per reference in the paper). They have combined previous methods in a single framework. Which makes the paper least technically significant.
    3. Although the idea of interslice consistency for improvement of denoising performance is good. But I have seen the similar idea in a concurrent work [iv]. Which again add a demerit to the technical contribution of the paper towards MICCAI community.
    4. Many previous SOTA for unsupervised LDCT denoising are not considered. For example, [i],[ii],[iii]. This methods must be included in the paper. Atleast [i] should be there, as both follows a similar constrastive method for LDCT denoising.
    5. No clear mention about the test set and train set split. Please mentioned it in the paper.

    Ref: [i]Jung, Chanyong, et al. “Patch-Wise Deep Metric Learning for Unsupervised Low-Dose CT Denoising.” Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VI. Cham: Springer Nature Switzerland, 2022. [ii] Bera, Sutanu, and Prabir Kumar Biswas. “Self Supervised Low Dose Computed Tomography Image Denoising Using Invertible Network Exploiting Inter Slice Congruence.” Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2023. [iii] Wu, Dufan, et al. “Consensus neural network for medical imaging denoising with only noisy training samples.” Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part IV 22. Springer International Publishing, 2019. [iv] S. Bera and P. K. Biswas, “Axial Consistent Memory GAN with Interslice Consistency Loss for Low Dose Computed Tomography Image Denoising,” in IEEE Transactions on Radiation and Plasma Medical Sciences, doi: 10.1109/TRPMS.2023.3260214.

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    As per the details given in the paper, the paper is reproducible upto a certain extent.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    Provide a clear explanation what are the main technical contribution of the paper, and include all the previous SOTA for LDCT denoising.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    3

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    See the weakness.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The paper offers four significant contributions to the field:

    1. The cutting-edge unsupervised approach, based on contrastive learning, outperforms the majority of unsupervised LDCT denoising techniques and holds a slight edge over fully supervised methods.
    2. The model boasts exceptional computational efficiency compared to prevalent cycleGAN-based methods and exhibits a high degree of adaptability to various generators, making it more fitting for clinical settings.
    3. The technique consistently generates stable output quality across slices, overcoming the inter-slice instability issues commonly observed in supervised methods, thus enhancing the consistency of denoising outcomes for improved clinical applicability.
    4. The creative modifications to the original contrastive learning approach, such as optimizing weights for positive-negative pairs and constraining difference invariants, allow the model to achieve top-tier denoising results with minimal computing resources.
  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The paper showcases several notable strengths, including:

    1. Computational efficiency: The proposed model demonstrates exceptional computational efficiency compared to widely-used cycleGAN-based methods. This efficiency renders it more practical for real-world clinical applications, where rapid processing times are crucial for precise diagnoses and effective patient care.
    2. Innovative unsupervised approach: The paper introduces a groundbreaking unsupervised method for LDCT denoising, which substantially outperforms most current unsupervised techniques and even exceeds certain fully supervised methods. This highlights the potential of unsupervised learning in medical image denoising, offering an alternative to conventional supervised methods that depend on labeled data.
    3. Refined contrastive learning technique: The authors have skillfully modified the original contrastive learning method, incorporating weight optimization for positive-negative pairs and constraint of difference invariants. These modifications allow the model to achieve state-of-the-art denoising results while utilizing minimal computing resources, further boosting its practicality in real-world medical imaging applications.
    4. Versatility: The method’s adaptability to various generators enables easy integration into diverse clinical settings and imaging devices. This flexibility positions it as a more universal solution for LDCT denoising across an extensive range of applications and environments.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Minor grammatical errors: The paper contains some small grammatical mistakes.
    2. It’s better to add metrics result value in corresponding visual result.
    3. Despite the unsupervised approach not relying on labeled data, the model may still be susceptible to overfitting if the generator and discriminator are trained on a limited dataset or a dataset that does not accurately represent real-world LDCT images. This risk could adversely affect the model’s performance when applied to new and unseen data, restricting its overall efficacy in practical applications.
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The authors have furnished in-depth explanations of the model, dataset, and evaluation procedure, significantly facilitating the work’s replicability. Although the code is not currently available, implementation should not pose a challenge.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    Improving the manuscript’s quality can be achieved by conducting a more comprehensive literature review and editing the grammar. The paper presents a promising concept and expands on previous research. To fully highlight its potential, the authors may consider refining the theoretical approach and experimenting with additional datasets.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    7

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    In this paper, the authors introduce a novel unsupervised approach to low-dose CT (LDCT) denoising that leverages contrastive learning. A key advantage of this innovative method is that it eliminates the need for paired training data, making it more clinically viable. The authors have also ingeniously adapted the original contrastive learning technique, resulting in state-of-the-art denoising outcomes while utilizing minimal computational resources. The study’s findings showcase the superiority of this method over both supervised and unsupervised alternatives, as demonstrated by qualitative and quantitative measures. The paper is eloquently composed, well-structured, and offers a precise and comprehensive explanation of the proposed method and its foundational principles.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    8

  • [Post rebuttal] Please justify your decision

    Based on the author’s reply, I recommend receiving it



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper received very diverse reviews, with two strong accept and one reject recommendations. The practical value of studying low dose CT (LDCT) denoising are recognized by all reviews. Two reviewers also recognize the paper’s contribution in improved contrastive learning method and computational cost etc. The area chairs considered the paper and the reviewers’ comments and have some concerns with reviewer#2: (1) On page 2, the authors claimed that the paper proposes a novel unsupervised approach for LDCT denoising, and has three significant contributions, but it is not clearly stated what are the differences between the proposed method and the existing method. How did previous method perform unsupervised learning? And how does this method perform unsupervised learning? Why can this method make use of unpaired data more effectively than pervious method? Because of the positive and negative pair construction, the weighting for different pairs, anything else? (2) As we know, PSNR does not exactly reflect human perception quality. Are the denoising results able to improve clinical diagnosis accuracy of physicians? The authors are suggested to provide a rebuttal to address the reviewers’ concerns and the above questions.



Author Feedback

Dear Reviewers, Area Chairs, and Program Chairs,

We appreciate your careful evaluation of our work and constructive criticism. We have addressed minor formatting and grammatical adjustments as advised. We now respond to the primary concerns raised in your assessment.

  1. Similarity with Interslice Consistency of Concurrent Work [i] Although both approaches use the same name ‘interslice consistency loss’, the methodologies are quite different. The referenced [i] utilizes a memory block to retain and reference features from the previous slice, thereby ensuring output consistency. Conversely, our method employs contrastive learning to regulate feature representation, focusing on differences in features from neighboring slices (provided by the model’s encoder). Furthermore, our model has the advantage of being able to process individual slices independently during inference. This contrasts with method [i], which necessitates the use of sequence data, leading to more constraints.
  2. Consideration of Previous State-of-the-Art Methods Thanks for the kind reminder. All the references mentioned by the reviewers will be cited in the revision.
  3. Innovation Concerns
    Indeed, reference [ii] employs contrastive learning in the domain of low-dose computed tomography (LDCT) denoising, yet our methodology applies it from a fresh viewpoint. Given that obtaining stable output results, such as continuous vessels or trachea in the vertical direction, and maintaining slice correlation is essential for LDCT denoising, we’ve augmented our strategy. Not only do we apply contrastive learning to the residual images within encoder feature representations for unpaired image translation, but more importantly, we also weave it into the feature difference pairs. This application is designed to maintain consistency in changes between matching slices located at the same position (positive pair in contrastive learning), while enlarge the feature difference in different locations or slice pairs (negative pair in contrastive learning). Consequently, this guarantees stability and uniformity in the modifications observed between slices pre and post-processing. Our model has consistently demonstrated its efficiency and effectiveness in the denoising of LDCT images. We hope that our rebuttal addresses the primary concerns. We believe that our work holds substantial value and significance for the community and welcome further feedback. Thank you for your consideration. [i] S. Bera and P. K. Biswas, “Axial Consistent Memory GAN with Interslice Consistency Loss for Low Dose Computed Tomography Image Denoising,” in IEEE Transactions on Radiation and Plasma Medical Sciences, doi: 10.1109/TRPMS.2023.3260214. [ii]Jung, Chanyong, et al. “Patch-Wise Deep Metric Learning for Unsupervised Low-Dose CT Denoising.” Medical Image Computing and Computer Assisted Intervention–MICCAI 2022: 25th International Conference, Singapore, September 18–22, 2022, Proceedings, Part VI. Cham: Springer Nature Switzerland, 2022.



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The paper studied low-dose CT denoising, proposed a generally novel method using unpaired data, and reported good accuracy. The rebuttal explained the paper’s differences with the previous method, and the major difference lies in the building of negative and negative pairs. The previous method used patches with the same anatomical structure as positive pairs, while this paper uses slice features located at the same position as positive pairs. After the rebuttal, I think there is a remaining concern: no comparisons are provided in either the original submission or the rebuttal. So, it is difficult to judge whether such a different positive-negative pair construction method is better than previous ones or not. I would like to recommend acceptance of this paper, but I hope the authors can provide a more in-depth comparison.



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    This work presents a novel unsupervised contrastive learning framework for low-dose CT (LDCT) denoising. I agree the reviewers that the authors have solved the concerns on the methodology novelty.



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The paper is technically novel and applies inter-slice contrastive learning without requiring memory blocks to perform contrastive learning. The authors seem to have addressed some of the key reviewers’ concerns. The paper is well written and methods for unpaired low dose CT denoising would facilitate acceptance and use of low dose CT methods in routine use.



back to top