List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Changyong Choi, Jiheon Jeong, Sangyoon Lee, Sang Min Lee, Namkug Kim
Abstract
Computed tomography (CT) image can be reconstructed by various types of kernels depending on what anatomical structure is evaluated. Also, even if the same anatomical structure is analyzed, the kernel being used differs depending on whether it is qualitative or quantitative evaluation. Thus, CT images reconstructed with different kernels would be necessary for accurate diagnosis. However, once CT image is reconstructed with a specific kernel, the CT raw data, sinogram is usually removed because of its large capacity and limited storage. To solve this problem, many methods have been proposed by using deep learning approach using generative adversarial networks in image-to-image translation for kernel conversion. Nevertheless, it is still challenging task that translated image should maintain the anatomical structure of source image in medical domain. In this study, we propose CT kernel conversion method using multi-domain image-to-image translation with generator-guided contrastive learning. Our proposed method maintains the anatomical structure of the source image accurately and can be easily utilized into other multi-domain image-to-image translation methods with only changing the discriminator architecture and without adding any additional networks. Experimental results show that our proposed method can translate CT image from sharp into soft kernels and from soft into sharp kernels compared to other image-to-image translation methods. Our code is available at https://github.com/cychoi97/GGCL.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43999-5_33
SharedIt: https://rdcu.be/dnwwM
Link to the code repository
https://github.com/cychoi97/GGCL
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
This paper applies image-to-image translation to generate different CT images of different CT kernels, which benefits from Generator-guided discriminator regularization via contrastive learning.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Applying mage-to-image translation to generate different CT images of different CT kernels accurately is meaningful for clinical diagnosis. It seems reasonable to add patch-based contrastive learning between the intermediate feature map from the generator and the semantic label map from the discriminator to solve (N+1)-way classification. This Generator-guided discriminator regularization produces good results compared with other methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The improvements bought by the contrastive learning in Table 2 seems quite small. The generation of the positive example during the contrastive learning needs more explanations. The benefits of contrastive learning should be clarified. The influence of GAN based generation is not clear, as some fake details may be generated. Please make more discussions. The experimental comparisons are not convincing as the compared methods are not specifically designed for the CT kernel conversion methods. More comparisons to the state-of-the-art CT kernel conversion methods should be given,
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I believe that the obtained results can, in principle, be reproduced. Even though key resources (e.g., code) are unavailable at this point, the key details (e.g., proof sketches, experimental setup) are sufficiently well described for an expert to confidently reproduce the main results, if given access to the missing resources.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please kindly refer to the part 6 for details.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Applying image-to-image translation to generate different CT images of different CT kernels is an interesting and meaningful idea to reduce the exposure to radiation for the patients. However, the experimental results are not significant. The writings should be improved.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proposes a CT kernel conversion method using multi-domain image-to-image translation with generator-guided contrastive learning
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Experimental results show that GGCL can translate stably into any direction for kernel conversion while accurately maintaining the source image’s anatomical structure.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
（1） Why is it difficult for existing generative adversarial networks to maintain the anatomical structure of source images in medical image translation? （2） What are the differences and advantages between the proposed method and existing generative adversarial networks? （3） The author claimed, “Our proposed GGCL can be easily utilized into other multi-domain image-to-image translation with only changing the discriminator architecture and without adding any additional networks.” Are there any experimental results in the article to support this contribution? （4） Please elaborate on the meanings of the symbols in the formula. （5） The visual difference between the different methods in Figure 2 is not obvious.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
n/a
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
（1） Why is it difficult for existing generative adversarial networks to maintain the anatomical structure of source images in medical image translation? （2） What are the differences and advantages between the proposed method and existing generative adversarial networks? （3） The author claimed, “Our proposed GGCL can be easily utilized into other multi-domain image-to-image translation with only changing the discriminator architecture and without adding any additional networks.” Are there any experimental results in the article to support this contribution? （4） Please elaborate on the meanings of the symbols in the formula. （5） The visual difference between the different methods in Figure 2 is not obvious.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
see q9
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This paper proposes an CT kernel conversion method using multi-domain image-to-image translation with generator-guided contrastive learning (GGCL) to maintain the anatomical structure of the source image accurately. And the proposed GGCL can be easily utilized into other multi-domain image-to-image translation methods with only changing the discriminator architecture and without adding any additional networks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Good experimental results and the proposed framework is compared and analyzed quite thoroughly. The experimental evaluation and discussion are adequate.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
W1. The authors should description of related work to clarify the main contribution more clearly.
W2. What is the difference between Generator-guided discriminator regularization (GGDR) and Generator-guided contrastive learning (GGCL)?
W3. Lack of model parameter and FLOPs comparison in the experiment.
W4. The technical details need to be clear. For instance, how to determine the balance parameters of the loss function in the paper?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reasonable.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Complement description of related work.
- Complement comparison experiment of model parameter and FLOPs.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty and main contribution of this paper are absolutely not enough.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The rebuttal addressed my concerns well. Considering other reviews, I tent to change my score to Weak Accept.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Overall, this is a reasonably well written paper with sufficient experiments performed with good comparisons to other baselines. The conceptual idea of generating other reconstructions of CT for improving diagnostics is also very good. As the reviewers pointed out, some key concerns are based on the incremental improvements achieved by the method compared to others. Perhaps the choice of other types of metrics, such as some of the commonly used radiomics metrics (first or 2nd order gray level correlation matrix) may be interesting to see if they bring about any differences between the methods. Also discussion of the rationale to use the contrastive learning particularly highlighting the advantages of the approach with respect to other approaches might strengthen the contributions furthermore. Paper could benefit from improved explanations of the various terminologies to better explain the method.
Questions to address in rebuttal: Please focus on the most important concerns raised by the reviewers including what is the rationale for the approach and how it improves over other methods to clearly explain the contributions, the choice of metrics and perhaps consider a different metric to bring about the differences between the baselines and the proposed approach, and improve the explanation of the method in terms of explaining the terminology.
Author Feedback
Thank you for your concerns and thoughtful feedback.
#1. Main contribution. (#R4) The difference between GGDR and GGCL is only what kind of loss is used. GGDR used cosine distance loss, but GGCL used PatchNCE loss for contrastive learning. (#R3) One of our contributions, “Our proposed GGCL can be easily utilized into other multi-domain image-to-image translation with only changing the discriminator architecture and without any additional networks.”, is based on the sentence that the author of GGDR paper said, “Our framework is simple and easy to apply existing GAN models.” By applying GGDR and GGCL to StarGAN, we showed that they can be applied to multi-domain image-to-image translation as well as unconditional generation. This implies that other multi-domain image-to-image translations can also utilize GGCL sufficiently. (#R2) The benefit of contrastive learning is revealed at the translation from soft into sharp kernels. It is a more difficult task than the translation from sharp into soft kernels because spatial resolution should be increased and noise patterns should be clear, so this benefit can be meaningful.
#2. Comparison between GGCL and other methods. (#R3) Differences are revealed at “architecture improvements” part on section 3.1 in our paper. Our method could maintain the fine detail structure and additional computation costs are less expensive. (#R4) We computed total parameters and FLOPs of the models for the comparison: CycleGAN – parameters: 21.2M, FLOPs: 884G CUT – parameters:11.2M, FLOPs: 442G UNIT – parameters: 26.4M, FLOPs: 1133G AttGAN – parameters: 591.2M, FLOPs: 35.9G StarGAN – parameters: 54.1M, FLOPs: 363.8G StarGAN w/ GGDR – parameters: 68.1M, FLOPs: 386.4G StarGAN w/ GGCL – parameters: 68.1M, FLOPs: 386.4G (#R2) We suspect that state-of-the-art CT kernel conversion methods were conducted with paired data, or with unpaired data but did not offer official code, so there was limited access for the reproducibility.
#3. The difficulty of anatomical structure maintenance of GAN based image translation. (#R2) GAN based generation has limitations that the training process may not be stable, and the results may be inaccurate. (#R3) Especially in medical image translation, with unsupervised manner training, the anatomical structure of the translated images relies on cycle-consistency mainly. If trained unstably, as the translated image would not be accurate, the reversed translated image would be poor as well and the cycle-consistency fails to lead the images with accurate anatomical structure. Other existing GAN based models showed these limitations in Figure 2 and Table 2.
#4. Complement description of related work. (#R4) We apologize for the lack of related work. In the following context, we will update complement related work about CT kernel conversion methods: “In deep learning methods for CT kernel conversion, there were proposed methods using convolutional neural networks, but they were trained in a supervised manner. Recently, there was an attempt to train using adaptive instance normalization (AdaIN) in an unsupervised manner, but this method needed the target images for the test phase and additional architecture for AdaIN.”. We will update the references of this related work as well.
#5. The meanings of the symbols in the formula and the technical details need to be clear. We apologize for the descriptions of the symbols and technical details. (#R3) If accepted, we will elaborate the meanings of the symbols on the final version. (#R4) Basically, we followed the hyperparameters of StarGAN. We experimented which of the numbers of the weight for PatchNCE loss is adequate. It was shown on section 3.3 and table 3 in our paper.
#6. The visual difference between the different methods in Figure 2 is not obvious. (#R3) We agree that Figure 2 cannot be obvious. For better visibility, we showed the difference map between the target images and the translated images in supplementary figure 1.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors mostly addressed the key concerns raised by the reviewers, which the reviewers found acceptable. The authors are strongly encouraged to update the discussion of the paper to clarify the limitations of the approach such as potential inaccurate synthesis with the GAN approach, as well as comparison to other methods.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
All three reviewers now have unanimously given accept for this work and the rebuttal satisfied concerns from previous reviews.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The motivation of this work is interesting, i.e., applying image-to-image translation to generate different CT images of different CT kernels. The concerns raised by the reviewers have been solved. Therefore, I recommend to accept this work.