List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Dayang Wang, Srivathsa Pasumarthi, Greg Zaharchuk, Ryan Chamberlain
Abstract
Deep learning (DL) based contrast dose reduction and elimination in MRI imaging is gaining traction, given the detrimental effects of Gadolinium-based Contrast Agents (GBCAs). These DL algorithms are however limited by the availability of high quality low dose datasets. Additionally, different types of GBCAs and pathologies require different dose levels for the DL algorithms to work reliably. In this work, we formulate a novel transformer (Gformer) based iterative modelling approach for the synthesis of images with arbitrary contrast enhancement that corresponds to different dose levels. The proposed Gformer incorporates a sub-sampling based attention mechanism and a rotational shift module that captures the various contrast related features. Quantitative evaluation indicates that the proposed model performs better than other state-of-the-art methods. We further perform quantitative evaluation on downstream tasks such as dose reduction and tumor segmentation to demonstrate the clinical utility.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_9
SharedIt: https://rdcu.be/dnwNa
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a global transformer to learn a dose reduction task in an iterative manner. The objective is to be able to generate contrast-enhanced brain T1 at an arbitrary low dose from an image without contrast and an image acquired at full dose.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The iterative aspect of the approach and the global transformer make the proposed method rather innovative.
- The evaluation does not stop at the use of usual image similarity metrics (e.g. PSNR, SSIM) but also includes task-specific metrics (e.g., contrast enhancement percentage).
- An application of the approach is proposed.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The choices made when designing the global transformer are not always well justified which makes the approach somewhat obscure.
- It is not entirely clear whether both pre-constrast and post-contrast images are needed to generate the low-dose image at test time. If both are necessary, it makes the application (low-dose to full-dose synthesis) a bit pointless and I do not see how the method would be used in practice.
- The test set for the evaluation is pretty small (13 samples), so it is difficult to draw strong conclusions when comparing the proposed approach to the state-of-the-art.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors mostly ticked ‘Yes’ while often it does not correspond to the reality (e.g., the cohort is not well described, no variation estimate is proposed when presenting the results, no statistical analysis is performed, etc).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- The introduction never mentions that the paper targets the brain. This could be made explicit or the authors could cite papers that cover other body parts (e.g., Xu et al., Medical Image Analysis, 2021).
- The authors write that “Currently MRI dose simulation is done using physics-based models[7]”. Such approach could be used for comparison.
- The authors mention several times “imbalanced dataset” but it is not clear what imbalance they are refering to.
- The fact that generating the soft label images amounts to adding the enhanced part of the image to the pre-contrast one, modulating the degree of enhancement, could be made more explicit. Additionnaly, the soft labels could be used as baseline for comparison.
- When generating the soft labels, it is not clear why skull-stripping is necessary. Is it a problem that the non-brain tissues do not perfectly cancel each other out?
- The loss L_total could be completed to be really total (i.e., adding adversarial and perceptual losses).
- The notion of global transformer already exists in the literature (e.g., Liu et al., IEEE TMI, 2020 or Zhang et al., ICCV, 2021). It would be good to detail the similarities and differences.
- A minimum should be said about the cohorts from site A and B: some demographic information (e.g., age, sex distributions), pathology distribution, scanner(s) used.
- What do the authors mean by co-registration? Rigid, affine, non-linear registration? Registration of the low-dose and post-contrat images to the pre-contrast one or registration of all the images to a template?
- How were all the hyperparameters optimised (i.e., using what data)?
- I am not sure what the “simple linear scaling” corresponds to.
- No variation estimate (std, CI) is provided, which makes the comparison between the different approaches difficult (impossible to determine whether one significantly outperforms the others).
- The paragraph “Low-dose to full-dose synthesis” could be clearer. What I understood is that the authors generate two synthetic T1CE: T1CE-synth from {T1, T1CE-real-ldose} and T1CE-synth-sim from {T1, T1CE-synth-ldose} and compare them both to the real T1CE. If this is correct, the notations could be more explicit.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method appears innovative, the fact that the evaluation does not stop at the use of image similarity metrics is valuable, but the applicability of the method should be clarified and the evaluation presents some weaknesses (e.g., cohort not detailed, small test set, no variation estimate).
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors have clarified some of the questions raised, hence the switch from weak reject to weak accept. Several limitations remain though.
Review #2
- Please describe the contribution of the paper
The paper develops a trasformer-based deep learning model to simulate low-dose Gadolinium contrast enhancement (CE) brain MRI scans from datasets that combine standard level CE-MRI and MRI without enhancement. The motivation for this is that there is need to develop methods that can synthesize CE-MRI from low-dose CE-MRI but that there are limited training datasets with low-dose CE-MRI. The paper uses an iterative transformer-based architecture in which the network gradually reduces the amount of enhancement in a CE-MRI to turn in into a regular MRI, with training data also including some low-dose CE-MRI as intermediate targets to train the model. The authors show that simulated low-dose CE-MRI resemble true low-dose CE-MRI more closely than a number of alternative methods, and that off-the-shelf segmentation methods perform comparably on this synthetic data as on real low-dose CE-MRI data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper claims to be the first to use deep learning, rather than physical models to perform low-dose CE-MRI simulation. The technical approach appears solid from the engineering perspective, using a transformer backbone with a number of modifications tailored to the proposed task. The evaluation uses multiple alternative techniques for comparison and evaluates the downstream utility of the synthetic CE-MRI images.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There is some circularity in the main premise of the paper, i.e., using DL to generate synthetic low-dose CE-MRI so that other DL methods can be used to simulate clinical CE-MRI from clinical low-dose CE-MRI. I think that there is a risk that the synthetic low-dose MRI might retain information about the full-dose CE-MRI so that a network trained to infer full-dose CE-MRI from the synthetic low-dose CE-MRI might have an easier problem to solve than a network trained to infer full-dose CE-MRI from real low-dose CE-MRI. It is hard to envision how such synthetic data can truly replace real low-dose CE-MRI for a thorough validation of a low->full CE-MRI prediction technique.
A second motivation for the paper is that “it is crucial for these dose reduction approaches to establish the minimum dose level required for different pathologies”. This is a laudable goal, but it is not clear that the current paper accomplishes this. It can synthesize CE-MRI at different doses, but the evaluation against ground truth is only carried out at the 10% dose for which ground truth is available, and it is not certain that the synthesized data at higher doses is accurate.
All the evaluated methods used the iterative scheme and only the backbone was varied. I think from the point of view of simulation, it would have been a good idea to also consider synthesis methods that directly simulate the low-dose image from full-dose and unenhanced images, which would demonstrate that the iterative approach with the proposed interpolated intermediate images as soft-labels is superior.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper provides a lot of details of the implementation. Some components are only described in passing and vaguely (such as adversarial and perceptual losses with citations to very general papers).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please see comments above.
- When talking about Gad downsides in the intro, I think it is important to mention health risks and contraindications in patients with reduced renal function; there have been many papers on this recently.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think the paper has merits in being a somewhat novel method with a thorough evaluation and unique application, but the motivation for the simulation is not as strong as could have been.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
Gadolinium contrast-enhanced (CE) MRI plays an important role in neurological disorder diagnosis. This paper presents a novel transformer based model for synthesizing low-dose CE MRI from full-dose and pre-contrast MRI, demonstrating improvement over the current state-of-the-art. It also showed comparable comparable performance to real low-dose images in two downstream tasks, including a low- to full-dose synthesis task, and a tumor segmentation task, both using external validation datasets.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper is well written. The introduction provides a good motivation and clear description of the problem. The figures and tables are well presented and greatly enhanced the presentation of the paper. The level of technical details is sufficient for someone with intermediate knowledge of the area to understand the proposed methods.
The methods seem correct, and it is a novel use of the rotational shift to replace the cyclic shift. The evaluation seem comprehensive, e.g., the synthesis performance was evaluated using a subset of the dataset, whereas the performance of downstream tasks were evaluated using external datasets.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
No major weaknesses were found.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The proposed methods were validated using external datasets. However, the two private datasets which were used to train and test the model are not available. It is difficult to assess the reproducibility of the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Table 1 summarizes the results of 13 test cases. Cross-validation is encouraged, to check the robustness of the model.
- More details of the downstream tasks need to be provided. It is not clear what models were used for low- to full-dose synthesis.
- For tumor segmentation, the BraTS2018 winning solution (variational autoencoder) was used, but it is not clear how the ground truth masks were generated or if they follow the same annotation standard of BraTS.
- It is also worth discussing the applicability considerations, e.g., the effects of different contrast agents being used, or the requirements of the input images in terms of required modality, quality, etc.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Novel methods, good presentation, and comprehensive experiments for a conference paper.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper describes a method to simulate low gadolinium dose MRI images from full-dose and pre-contrast MRI images. Authors show that their learning-based approach performs well compared to a physics-based model. The reviewers raise several issues that can be addressed in a rebuttal (see below).
Strengths
- The iterative application of the transformer is innovative.
- The work includes task-specific evaluation criteria.
Weaknesses
- Reviewers indicate some issues with reproducibility and availability of data.
- The motivation for the work is not entirely convincing and a bit circular.
No additional experiments should be run for the rebuttal, but it would be good to use the opportunity to address the questions of the reviewers, and specifically those about:
- What kind of data exactly is required for training (Rev. 1).
- The limited size of the test set (Rev. 1).
- The circularity in the motivation of the paper (Rev. 2).
- The lack of information about performance at higher dose levels than 10% and relation to the goals of the paper (Rev. 2).
- The clinical value of gadolinium reduction (Rev. 2, Rev. 3).
Author Feedback
We sincerely thank all the reviewers for their insightful comments. We are encouraged that they found the work innovative, some of the ideas novel, and the evaluation techniques to be valuable.
R1,2 Circularity, need for pre & post-contrast images: We will respond to this using an example of how we internally used the Gformer model. Using the 126 brain MRI patient studies from site A we trained and validated a dose reduction model. We also had 6 patient studies with spine MRI from the same site, which had pre-, low- & full-dose images. The small size of the spine dataset is clearly not sufficient to train a specific dose reduction model. The same site had 100s of spine cases with pre & full-dose images (standard protocol). Using these pre & full-dose spine images, we synthesized low-dose images using the Gformer model (trained only on brain images) and used these to train/fine-tune a spine dose reduction model. This model, in turn, was validated on the 6 full spine patient studies. This is a typical example where the dose-simulation approach was used to fill the data gaps in a downstream application.
R1 Size of Test Set: One of our evaluation goals was to establish cross-site generalizability because of the different contrast agents used in the two sites. Hence, data from Site A was used for training & testing and data from Site B was used for downstream analysis. Within Site A, we had to make a tradeoff between the number of train vs test cases as transformers need large amounts of training data. Hence, we had to reduce the test cases to 13 patient studies (3120 slices). However, we have done extensive downstream task validation using 159 patient studies.
R1 Cohort Description: Site A (126 patient studies; 55 Females; 48 土 16 years) used a Philips Insignia 3T scanner (TE - 2.97-3.11 secs, TR - 6.41-6.70 secs, FA - 8 deg). Site B (159 patient studies; 78 Females; 52 土 17) used a GE Discovery 3T scanner (TE 2.99-5.17 secs, TR 7.73-12.25 secs, FA 8-20 deg). The clinical indications for both sites included suspected tumor, post-op tumor follow-up and routine brain.
R1 Variation Estimate: The std. dev., values for Table 1:Method: (PSNR/SSIM/RMSE/LPIPS). Post-(2.88/0.03/0.13/0.016). Scaling: (2.22/0.19/0.05/0.015). Rednet: (2.72/0.01/0.055/0.009). Mapnn: (1.64/0.012/0.051/0.012). Restomer: (2.27/0.011/0.165/0.016). SwinIR: (2.25/0.015/0.063/0.015). Gformer Cyc: (2.14/0.016/0.045/0.007). Gformer Rot: (2.02/0.015/0.031/0.005).
R1 Statistical Analysis: For both dose reduction tasks, the performance of the simulated was similar to that of real low-dose with p < 0.0001 (Wilcoxon signed rank test). This information was featured in a table we removed due to lack of space.
R1 Gformer Design Choices: The design of Gformer is similar to [12], which includes 6 blocks and symmetric shortcuts. The 3x3 convolution is designed to learn local information. The global attention aims to further capture the global features of the contrast uptake, such as those present in blood vessels that are globally connected, while the rotational shift targets the irregular shape of the contrast uptake.
R1 Comparison with Relevant Work: The proposed global attention is different from Liu & Zhang’s work. Liu et al. generates different queries in the transformer to grasp the multi-scale global information, while Zhang et al. add extra global tokens for the local token to attend. In contrast, our model captures global information from the subsampled feature maps.
R2 Performance at dose > 10%: We have shown that the dose levels > 10% follow an expected trend, in Figure 5. Using physics-based models [27] to evaluate the accuracy of intermediate doses is not feasible as T1-map estimation is needed or an inaccurate GM/WM population average has to be assumed. The equations are also sensitive to scanner parameters.
R2,3 Clinical Value of Gd Reduction: We have highlighted this in the introduction and cited literature that have detailed the same [1-6].
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
While several limitations remain, the authors have addressed most review comments in their rebuttal and the work is acceptable for MICCAI. All reviewers now recommend acceptance.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have adequately addressed the comments regarding the statistical analysis, experimental comparsion, clinical value, etc. They have noted that the reviewers’ comments will be incorporated into the final version of the paper. Hence, I suggest accepting this paper for MICCAI.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Overall this is a well-written paper with minor concerns from the reviewers. The rebuttal has successfully convinced R1 to increase the review score. It would be a good fit for publication at MICCAI.