List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Thomas Pinetz, Erich Kobler, Robert Haase, Katerina Deike-Hofmann, Alexander Radbruch, Alexander Effland
Abstract
Today Gadolinium-based contrast agents (GBCA) are indispensable in magnetic resonance imaging (MRI) for diagnosing various diseases.
However, GBCAs are expensive and may accumulate in patients with potential side effects, thus dose-reduction is recommended.
Still, it is unclear to which extent the GBCA dose can be reduced while preserving the diagnostic value – especially in pathological regions.
To address this issue, we collected brain MRI scans at numerous non-standard GBCA dosages and developed a conditional GAN model for synthesizing corresponding images at fractional dose levels.
Along with the adversarial loss, we advocate a novel content loss function based on the Wasserstein distance of locally paired patch statistics for the faithful preservation of noise.
Our numerical experiments show that conditional GANs are suitable for generating images at different GBCA dose levels and can be used to augment datasets for dose reduction models.
Moreover, our model can be transferred to openly available datasets such as BraTS, where non-standard GBCA dosage images do not exist.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43895-0_57
SharedIt: https://rdcu.be/dnwzp
Link to the code repository
https://github.com/tpinetz/low-dose-gadolinium-mri-synthesis
Link to the dataset(s)
https://www.med.upenn.edu/cbica/brats2020/data.html
Reviews
Review #4
- Please describe the contribution of the paper
The main contribution of this paper is the development of a conditional GAN model for synthesizing contrast-enhanced brain MRI scans at fractional dose levels using a novel noise-preserving content loss function based on the Wasserstein distance. The results show that this method is suitable for generating images at different GBCA dose levels and can be used to augment datasets for virtual contrast models. Additionally, the proposed noise-preserving content loss function enables the faithful generation of noise, which is important for the identification of enhancing pathologies and their usability as additional training data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The development of a novel noise-preserving content loss function based on the Wasserstein distance for the faithful preservation of noise in synthesized contrast-enhanced brain MRI scans. This is interesting because it enables the generation of realistic synthetic images that can be used to augment datasets for virtual contrast models. The use of a conditional GAN model to synthesize contrast-enhanced brain MRI scans at fractional dose levels. This is a novel application of GANs in the field of medical imaging and has the potential to reduce the use of Gadolinium-based contrast agents (GBCA) in MRI scans, which can be expensive and have potential side effects. The demonstration of the clinical feasibility of the proposed method through numerical experiments that show that conditional GANs are suitable for generating images at different GBCA dose levels. This has the potential to improve the diagnosis of various diseases using MRI scans while reducing the use of GBCA. A strong evaluation of the proposed method through a comparison with other content loss functions and an analysis of its performance in synthesizing low-dose contrast-enhanced brain MRI scans and its impact on virtual contrast models.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper provides a brief comparison with other methods for synthesizing contrast-enhanced brain MRI scans at fractional dose levels. However, a more detailed and comprehensive evaluation of the proposed method against other state-of-the-art methods in this field would greatly enhance the paper’s contribution. In addition to the conditional GAN model and noise-preserving content loss function, other factors can have a significant impact on the performance of the proposed method. A deeper analysis of the impact of other GAN architectures or training data on the model’s performance could provide valuable insights into improving the method further. While the numerical experiments presented in the paper use a specific dataset of brain MRI scans, it would be beneficial to evaluate the proposed method on other datasets or in different clinical settings. This could help assess the generalizability and robustness of the proposed method in diverse scenarios. Furthermore, additional experiments on larger datasets could provide further insights into the performance of the method and its potential applications.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No problem in this field.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please see the weakness. The paper could benefit from a more detailed comparison with other methods for synthesizing contrast-enhanced brain MRI scans at fractional dose levels. It is recommended to provide a more comprehensive evaluation of the proposed method against other state-of-the-art methods in this field. While the paper primarily focuses on the synthesis of contrast-enhanced brain MRI scans at fractional dose levels using a conditional GAN model and a novel noise-preserving content loss function, it would be interesting to see an analysis of the impact of other factors, such as the choice of GAN architecture or training data, on the overall performance of the proposed method. Although the paper presents results from numerical experiments using a specific dataset of brain MRI scans, it would be valuable to evaluate the proposed method on other datasets or in other clinical settings to assess its generalizability and robustness.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The problem being addressed in clinical practice is a significant one that has been widely recognized, and the novelty of the approach being taken is encouraging. However, it is important to note that there are still some concerns that need to be addressed in order to fully realize the potential of this intervention. As outlined above, some of these concerns include potential side effects, long-term efficacy, and the need for further research to establish the optimal dosage and treatment duration. Nonetheless, it is clear that this approach has the potential to make a real difference in the lives of patients, and it is therefore essential that we continue to explore its potential and work towards its implementation in clinical practice.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This work develops a noise-preserving conditional GAN model to generate low dose contrast-enhanced brain MR images. The generated images can be used to facilitate research on low-dose imaging protocols so that the side effects of GBCAs can be reduced. Qualitative and quantitative results of the generated images, as well as the predicted standard dose images, are provided in the paper.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This is a great example of using deep learning techniques to address clinical needs – to reduce the side effects from MRI contrast agents.
- The proposed noise-preserving conditional GANs generated more realistic images than L1 loss and the perceptual loss from VGG. Similar ideas can also be used for data synthesis in other applications
- The testing data also includes samples from another site using different scanners and GBCA.
- The results given in the paper are promising.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- In Table 1., L1 loss gave the highest PSNR and SSIM for low-dose synthesis, but for the standard-dose prediction, the proposed noise-preserving approach achieved the highest PSNR and SSIM, is this attributed to some behavior from the virtual contrast model used by the authors for standard-dose prediction? This point should be discussed in the paper.
- From the first paragraph on page 2, the authors’ objective is to create a large database to benchmark and compare different low-dose to standard-dose algorithms. However, based on the results in Table 1, it’s not obvious the proposed noise-preserving approach is the best option as the models trained may get adapted to the generated images but perform sub-optimally for real-world data. For example, the low-dose synthesis model with the low PSNR and SSIM outputs standard-dose predictions of high PSNR and SSIM using a virtual contrast model.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The proposed method seems reproducible; however, no description of the result of central tendency and no information on sensitivity regarding parameter range.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Please address the two points given in the section of “Main Weaknesses”
- Page 7, the 3rd paragraph, it’s not clear why was MAE_{signma} only computed for non-CE pixels.
- Page 4, in the paragraph below the definition of L_{GAN}, the prediction range of D is limited to [0.05, 0.5], the authors may want to discuss the impact of this limitation.
- minor comment: page 9, citation 8, “mri” should be all uppercase.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The noise-preserving approach is novel and can be used for other application with data scarcity.
- Diverse testing data See the section of Main Strengths for more details
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proses a GAN-based model that learns to generate contrast-enhanced MRI images with the desired dose level by being trained with real images with distinct doses. The authors address several challenges to make their model stable, preserving the content of the image.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The idea is novel, working on an important/interesting question in the literature. The main concept of this idea is beneficial for a general topic, representing discrete variables as continuous vectors.
- The authors possess a strong understanding of the domain they are working on through the novel insights presented in this paper.
- The content-preserving loss function is evaluated well by the support of a wide ablation study.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The diagram is a bit misleading. x^hat_LD is not in the diagram, while the first paragraph of the Methodology refers to it. The authors should show in the diagram that y^hat_LD is generated by x^hat_LD - x_NA. Different conditions can be specified in the diagram similar to the text by c and c^hat.
- Authors proposed to embed metadata in the generation process inspired by diffusion models, which is not explained well.
- In section 2.1, paragraph 2, authors emphasize that they do not use local attention layers suggested by ViTGAN (ref[20]), while the relation between the proposed method and ViTGAN is not discussed before.
- As the authors describe in section 2.1, the dose condition for the discriminator is the Gaussian (std=0.05) augmented form of one of the distinct dose values:{10%, 20%, and 33%}. In contrast, the generator is conditioned on another dose level sampl
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors mentioned that the training code won’t be available. However, the architecture is explained in detail. The loss function and training flow are described well. Therefore, it should be reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Authors should revise some of their statements in the text to make the text easier to follow.
- The diagram should be revised to be more understandable and followable with the text.
- The relevance of the proposed method with diffusion models and ViTGAN should be explained and clarified.
- More details about the implementation of the claimed idea about generating content images with desired dose level (interpolated from a continuous space) should be clarified in section 2.1. Also, in the evaluation section, the performance of the model in generating data with doses rather than 0.1, 0.2, and 0.33 should be shown. In case no evidence is presented for that, this claim needs to be eliminated from the text.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The probable impact of the proposed method on GAN models in the literature.
- Addressing challenges in training GAN with proper solutions.
- Nice ablation study on content learning methodology.
- Not evaluating the second claimed contribution of the paper.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This is a nice paper that describes a method for synthesis of low-dose contrast-enhanced MRI images using GANs. Authors have collected their own dataset of brain MR images at different dose levels that can be used to train a GAN that can output images at any dose level. The reviewers raise several issues, most of which can likely be addressed in a camera ready version of the paper.
Strengths
- The authors have collected an interesting data set with MRI images at different gadolinium dose levels for the same patients.
- Predicting the contrast enhancement as a residual map is an interesting idea.
- Interesting addition of a loss term that enforces similar noise patterns in real and synthetic images. This is very strong.
Weaknesses
- The use of GANs for image-to-image synthesis is not necessarily novel.
- Reproducibilty might be limited, as there is no indication that the authors will share their code our unique data set.
Author Feedback
We are thankful for the constructive reviews and we appreciate that all the reviewers acknowledge the novelty and potential of our approach. For the sake of reproducibility, we will release code along with the necessary weights to validate our results on the publicly available BraTS dataset. Thus, all necessary implementation details are contained in this repository.
Reviewer #2: “From the first paragraph on page 2, the authors’ objective is to create a large database to benchmark and compare different low-dose to standard-dose algorithms. However, based on the results in Table 1, it’s not obvious the proposed noise-preserving approach is the best option as the models trained may get adapted to the generated images but perform sub-optimally for real-world data.” Thank you for bringing this to our attention. This was indeed unclear in the original text. We claim that preserving the noise in the synthetic images serves as better training data and the resulting models better generalize towards real data. This fact is nicely demonstrated by the numerical results as the PSNR on the standard dose prediction model was always evaluated on real data points, while the training dataset consisted of purely synthetic LD images. We updated the paper accordingly.
“Page 7, the 3rd paragraph, it’s not clear why was MAE_{sigma} only computed for non-CE pixels.” This is an interesting question, that we are happy to clarify. Since we are working on residual images that only contain contrast-enhancing pixels and noise, the contrast-enhancing pixels do not exhibit zero mean and would hence bias the calculation of the noise standard deviation. Therefore, we excluded the contrast-enhancing regions from the MAE_sigma metric.
Reviewer #3: “The diagram should be revised to be more understandable and followable with the text.” Thank you for the comment. In the camera-ready version of the publication, we extended the diagram to include the inference process.
“Also, in the evaluation section, the performance of the model in generating data with doses rather than 0.1, 0.2, and 0.33 should be shown” We indeed performed that experiment and are happy to include the results in the final version.
Reviewer #4: “While the numerical experiments presented in the paper use a specific dataset of brain MRI scans, it would be beneficial to evaluate the proposed method on other datasets or in different clinical settings. This could help assess the generalizability and robustness of the proposed method in diverse scenarios. Furthermore, additional experiments on larger datasets could provide further insights into the performance of the method and its potential applications.” Thank you for pointing this out. Yes, our approach can be extended to different datasets and (medical) tasks in imaging. However, this is out of scope of this paper and we leave this for future work.