List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Chanyong Jung, Joonhyung Lee, Sunkyoung You, Jong Chul Ye
Abstract
The acquisition conditions for low-dose and high-dose CT images are usually different, so that the shifts in the CT numbers often occur. Accordingly, unsupervised deep learning-based approaches, which learn the target image distribution, often introduce CT number distortions and result in detrimental effects in diagnostic performance. To address this, here we propose a novel unsupervised learning approach for lowdose CT reconstruction using patch-wise deep metric learning. The key idea is to learn embedding space by pulling the positive pairs of image patches which shares the same anatomical structure, and pushing the negative pairs which have same noise level each other. Thereby, the network is trained to suppress the noise level, while retaining the original global CT number distributions even after the image translation. Experimental results con rm that our deep metric learning plays a critical role in producing high quality denoised images without CT number shift.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_60
SharedIt: https://rdcu.be/cVRT4
Link to the code repository
https://github.com/jcy132/DML_CT
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper proposed a novel unsupervised learning approach for low-dose CT reconstruction using patch-wise deep metric learning. Experiments confirmed that the deep metric learning plays a critical role in producing high quality denoised images without CT number shift.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper proposed a novel method for low-dose CT denoising based on the patch-wise deep metric learning. The algorithm can successfully make the network focus on the anatomic information and neglect the noise features by the push and the pull between the features in the embedding space.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The sample size in this study was limited and the one-time split could not verify the generalization ability of the network well.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper proposed a novel unsupervised learning approach for low-dose CT reconstruction using patch-wise deep metric learning. Experiments confirmed that the deep metric learning plays a critical role in producing high quality denoised images without CT number shift. However, there are some concerns as follows:
- The sample size in this study was limited and the one-time split could not verify the generalization ability of the network well.
- Cross-validation should be conducted for further validation.
- Authors should provide more detailed descriptions of the pipeline in section 2.1. The description should include how the GAN is applied to obtain output images from noisy input images, how features are extracted from these two types of images and whether two generators have the same parameters. Moreover, authors did not present how to combine output images with low frequency images to obtain final results.
- In fig. 3, using high frequency images as the gold standard is beneficial to present the difference images.
- In table 1 and table 2, the title should be written before the table.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The limited sample size and one-time split could not verify the generalization ability of the proposed algorithm.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
- In this work, the author applied a patch-wise deep metric learning method to the hidden embedding space in the mid-layer feature map of GAN’s generator to maintain the structural information and suppress the noise.
- The method achieves better PSNR and SSIM with less CT numbers shift compared with existing unsupervised Low-Dose CT denoising methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Use deep metric learning by setting two hidden embedding space in the generate network before the output layer to make the GAN more stable
- Set the positive pair from same location of noisy input and denoised output to maintain the structural and set the negative pair from different location of same image to suppress the noise level
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The purpose of the proposed metric is not well explained. Why can setting the positive pair and the negative pair be used to maintain the structural information and suppress the noise level? Why is the metric method used in the mid-layers of the generator? Why is the metric loss L2 normalization with a temperature parameter? All these setting is quite similar to contrastive learning instead of metric learning.
- In the ablation study results (supplementary material table 1), how the proposed method addresses CT numbers shift is not demonstrated.
- The motivation of maintaining previous CT number statistics is questionable. The LDCT has worse CT number statistics when compared to NDCT. The proposed method should learn the CT shift and make the output as much close as the NDCT.
- The objective function in metric learning is to pull the patches from the same location together while pushing the patches from neighbors away. The trivial solution is to make network output the same input; that is, doing nothing. This is contrast to denoising.
- The description of the datasets and the experiment implementation is insufficient. Especially for the AAPM challenge, even the quantity and the division of the dataset are not given. How the unsupervised data is constructed is not stated.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The approach is quite difficult to reproduce. On the one hand, it’s not clear to know about the specific setting of the forward processing of the generative adversarial network, such as the convolution blocks. On the other hand, the description of the datasets and the experiment implementation is insufficient. Especially for the AAPM challenge, even the quantity and the division of the dataset are not given.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The approach needs more ablation study to demonstrate the improvement in performance with CT numbers shift.
- The approach needs to be compared to more existing state-of-the-art approaches, which include both supervised and unsupervised methods.
- The motivation of maintaining the original CT number is questionable.
- The motivation of the objective function is also questionable. More experiments should be conducted to show how it works.
- The approach needs more detailed description of the model blocks、datasets and experiment implementation to access better reproducibility.
- The related work needs more content about deep metric learning and the related methods applied to the medical image processing.
- I wonder whether the image of domain Y can be called HDCT, because the model does not use any HDCT image, but only denoised LDCT images.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
There is a lack of explanation of proposed methods used in the study to demonstrate the novelty . There is a lack of more reliable experiments and the ablation study to demonstrate the effectiveness and generalizability, especially the CT numbers shift.
- Number of papers in your stack
8
- What is the ranking of this paper in your review stack?
7
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Thank the authors for their clarifications in the rebuttal. Maintaining the CT number shift seems to be reasonable if the low-dose CT is a practical standard. It might be better to clarify this point in the very beginning or title so that the readers won’t be confused. I am also glad that the authors will add the discussion between the contrastive learning and the metric learning the final version. Therefore, I raised my rating.
Review #3
- Please describe the contribution of the paper
This paper introduces deep learning in a deep feature space. This is combined with an adversarial loss to preserve feature consistency. This improves the denoising performance, while maintaining greater CT number accuracy.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Most denoising papers do not take into account CT numbers, but purely consider PSNR and SSIM. This is an excellent view of the problem.
The method is simple and can be easily reproduced. The results are very promising statistically. The ablation studies provide good explanation for hyper parameter choices.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
This paper proposes learning in a deep feature space as novel. Actually, using deep feature extractors, particularly the VGG extractor, is a very common method in LDCT denoising problems. Reference 22 in this paper has introduced this method.
While several GAN based approaches are discussed here, I am interested to see comparisons with other directly supervised approaches. Perhaps the GAN loss is not sufficient.
The choices for the windows used in Figures 3 and 4 are very unusual.
The authors have used the 2016 AAPM dataset. Actually, this dataset is out of date, and has been superseded by the 2020 Cancer Imaging Archive dataset, which is significantly larger.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reproducibility is sufficient. All implementation and dataset details are provided.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Perform the quantitative comparisons with a larger and more modern dataset.
Justify the choices for the windows, or choose more conventional windows.
Perform more comparisons with non-GAN state-of-the-art methods.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper shows a new perspective on a relatively old problem, and then proposes a novel, yet simple method to deal with it.
The statistical results display this method’s superiority over others, and ablation studies well justify the choices made for hyperparameters.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
My main concern was in the loss and dataset used. The authors have provided good results in the rebuttal, and this is sufficient to address my minor concerns.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper has inconsistent comments from reviewers, and such I would invite the authors for a rebuttal to resolve the conflict. It is expected for the authors to further clarify the generalization of the method, motivation of CT numbers, reproducibility, etc.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
We thank valuable comments from all three reviewers. Reviewer2 asked for the clarification of the motivation to preserve the input CT numbers. As mentioned in the paper, one of the motivations for this study was temporal CT imaging for the evaluation of pathologies of the ears especially for children, who are vulnerable for radiation dose. Furthermore, monitoring of post-operative condition for tympanoplasty is mandatory, where the HU value can contribute to detect the recurrent cholesteatoma and avoid the unnecessary surgery. Therefore, the standard practice is to use low-dose CT. Moreover, the diagnosis is based on the CT numbers with a variation as small as 5HU, so retaining the values from the low-dose standard acquisition while reducing the noise is utmost important for clinical diagnosis. Reviewer2 pointed out the similarity between the proposed deep metric learning and contrastive learning. We agree that the recent research [1], which will be added in the final version as a related work, also uses the pull-push mechanism, to maximize the mutual information (MI) between the patches by the contrastive loss (i.e. infoNCE). However, the denoising is not achieved by maximizing MI, since the preservation of the noise pattern also contributes to the MI maximization. On the other hand, our work focuses on the denoising by the pull-push mechanism using a newly designed metric. Specifically, we designed the negative pairs to have same noise level. Then, pushing the pairs discards the noise features to make the them maximally different. Also, pulling the positive pairs regulates the algorithm to maintain the common feature, which is a structural information. The reviewers are kindly reminded that our loss leads to the different solution with infoNCE, as infoNCE pushes the patches with different noise level and spatial feature for the MI maximization. The experimental result supports our claim, verifying the degraded result of infoNCE. Therefore, we suggest our work as a deep metric learning which generalize the methods based on the pull-push mechanism, to differentiate it with infoNCE in [1]. We will also provide a review of deep metric learning in the final version. Reviewer1,2,3 pointed out the reproducibility and detailed description such as dataset, model structure and loss function. We will release the code to ensure the reproducibility. The repetition for AAPM 3 times results PSNR 38.145+-0.084, SSIM 0.875+-0.0018 which shows stability. For AAPM dataset, we selected 3112 images from the trainset, and evaluate the method by 421 test images. For cross-validation, we randomly select 3112 images 3 times. The PSNR is 38.039+-0.187 and SSIM is 0.874+-0.0034. We used L2 distance with exponential function as a deep metric and the features from the shared generator, since it is verified to have an enough capability to embed the useful features[1]. The final denoised image is obtained by adding the generator output with low-frequency image of input. This detailed information will be added in the final version. Reviewer 2, 3 suggested the additional comparison and ablation studies. For AAPM, supervised method results PSNR 39.48, SSIM 0.898 but outputs overly smoothed images. The recent SoTA by infoNCE[1] results PSNR 37.41, SSIM 0.86, compared with our result above, PSNR 38.145, SSIM 0.875. Following Reviewer2’s comment, we removed pushing and pulling to investigated the consequences. Without pushing, the result is PSNR 36.07, SSIM 0.82. Without pulling, the result is PSNR 36.51, SSIM 0.82. To additionally verify a robustness to CT shift, we used AAPM with 100HU decreased for full-dose, and evaluate the output by original full-dose. Our method results PSNR 38.08, SSIM 0.874, and preserves the input HU. However, cycleGAN results PSNR 29.32, SSIM 0.40 showing vulnerability to the CT number shift. [1] T. Park, eta al, “Contrastive learning for unpaired image-to-image translation,” ECCV, 2020.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal successfully addressed the concerns from reviewers about the motivation and reproducibility. The reviewers have reached consensus of acceptance after the retbuaal.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors propose an unsupervised learning approach for low-dose CT reconstruction using patch-wise deep metric learning. The concerns of the reviewer 3 on the loss function and dataset, the concern of all three reviewers on reproducibility have been successfully addressed in the detailed rebuttal. Therefore, I recommend acceptance of the manuscript.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presented an interesting work about unsupervised low-dose CT denoising through patch-wise deep metric learning. The proposed method produces high-quality denoised images while maintaining the CT number. Reviewer#2 previously raised concerns about the motivation (maintaining CT number), novelty (against contrastive learning), and ablation study, which have successfully addressed in the authors’ response. After Reviewer#2 changed the rating from 3 to 5, the reviewers reached a consensus upon the acceptance of this paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1