List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Eyal Hanania, Ilya Volovik, Lilach Barkat, Israel Cohen, Moti Freiman
Abstract
T1 mapping is a quantitative magnetic resonance imaging
(qMRI) technique that has emerged as a valuable tool in the diagnosis of diffuse myocardial diseases. However, prevailing approaches have
relied heavily on breath-hold sequences to eliminate respiratory motion
artifacts. This limitation hinders accessibility and effectiveness for patients who cannot tolerate breath-holding. Image registration can be
used to enable free-breathing T1 mapping. Yet, inherent intensity differences between the different time points make the registration task challenging. We introduce PCMC-T1, a physically-constrained deep-learning
model for motion correction in free-breathing T1 mapping. We incorporate the signal decay model into the network architecture to encourage
physically-plausible deformations along the longitudinal relaxation axis.
We compared PCMC-T1 to baseline deep-learning-based image registration approaches using a 5-fold experimental setup on a publicly available
dataset of 210 patients. PCMC-T1 demonstrated superior model fitting
quality (R2: 0.955) and achieved the highest clinical impact (clinical
score: 3.93) compared to baseline methods (0.941, 0.946 and 3.34, 3.62
respectively). Anatomical alignment results were comparable (Dice score:
0.9835 vs. 0.984, 0.988).
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_22
SharedIt: https://rdcu.be/dnwLx
Link to the code repository
https://github.com/eyalhana/PCMC-T1
Link to the dataset(s)
https://cardiacmr.hms.harvard.edu/downloads-0
Reviews
Review #1
- Please describe the contribution of the paper
In MRI, the objective of this work is to automatically construct parametric images such as T1 mapping from free breathing acquisition using raw image registration and modelling of the signal decay. Indeed, by default these images are acquired in breathhold, and it is not possible for all the patients. So a free breathing alternative is a good solution.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
There were ever tries to reconstruct these parametric images from free breathing acquisition, but it seems to be the first time that this reconstruction involves both image registration and modelling of the T1 decay. Considering both features allow to reinforce the quality of the parametric images. The results are visually convincing, and I think this method could be used in clinical practice. Moreover, there is a good litterature review in the introduction, as a good introduction of the objective. And finally, there are good evaluation process and discussion.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some part of the method are not well explained, or questionables such as :
- I do not understand what are the synthetic images cited by the authors. For me, there is one synthetic image, i.e. the parametric image. But maybe it is not the parametric image. So I do not understand what is the principle of the registration on several synthetic images. This part of the method is strange or not well explained.
- Why there is no clinical score for two methods ? Moreover, the method look very remarkable, but I am a little bit disappointed by the clinical score. I expected something close to 5. Maybe the authors could discuss about that also.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors stated that “Our code and trained models will be made publicly available upon acceptance”.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
First of all, I suggest the authors to modify the article according to my remarks about the synthetic images, and the missing results in the table. Moreover, in the figure 3, it could be good to highlight the difference in the images by showing them with arrows. Minor comments :
- In the abstract, I suggest to do some sentences in the passive voice. And in the abstract, it is not clear what it is evaluated.
- The figure 1 is not cited in the text.
- A little comment about T2 parametric images could be added in the discussion.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is a very good study, with a clear objective. The authors justify their choice, the method is validated with a good evaluation process. Just a part in the methodology must be clarified
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper proposes a joint framework termed PCMC-T1 for T1-mapping prediction along and dense displacement prediction for breadth motion correction. The major contribution is minimizing the difference between the image generated from the T1 recovery curve with the warped image from the predicted displacement.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
It is interesting to use multi-tasking to jointly predict the T1 map and the displacement caused by breadth motion.
-
The idea to use self-consistency as regularization between T1 recovered image and the motion-corrected image is interesting.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The use of R-squared as the major metric to quantify the segmentation mask conformance is not very convincing since R-squared is mostly used in regression analysis instead of segmentation.
-
The performance of the proposed network is limited. From Table 1, SynthMorph seems to be the better method in general especially with a higher Dice and lower HD. The superiority of the proposed network is also questionable from Fig.3, where SynthMorph generally works better, especially in the first two rows.
-
The effect of the proposed framework is also not very convincing. It seems that “auxiliary” segmentation loss is the major driving force. From Table 1, the method performs relatively very badly without segmentation loss (e.g. the Dice and HD metrics drop drastically).
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Given the author’s statement on reproducibility in the last sentence of the abstract, reproducibility is not a major concern.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The major comments are listed in the main weaknesses section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The major consideration of the recommendation is based on the main weaknesses.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
2
- [Post rebuttal] Please justify your decision
Thanks authors for the feedback but my two biggest conerns are still not addressed.
1) the superiority over Synthmorph: though R-squared and clinical score are worse but the Synthemorph is still better in anatomical aspect (dice, hd) by quite a big margin (~4.5% dice). I will say these two methors are similar but it is hard to conclude the proposed one is better.
2) The authors didn’t really address the issue that by having a trivial segmenation loss, the performance boost from 66.2% to 83.5% in in dice.
Thus, I will keep my original score.
Review #3
- Please describe the contribution of the paper
The paper presents PCMC-T1, a physically-constrained deep-learning model for motion correction in free-breathing T1 mapping. The main contribution is the incorporation of the signal decay model into the network architecture to encourage plausible deformations along the longitudinal relaxation axis. The authors demonstrated the advantages of PCMC-T1 over baseline deep-learning-based image registration approaches using a publicly available free-breathing quantitative T1 MRI dataset of 210 patients. Their method achieved better mean model-fitting R2 compared to baseline methods, with comparable mean dice scores. Additionally, they demonstrated the clinical impact of their method through improved semi-quantitative expert cardiac radiologist review. This model has the potential to broaden the application of quantitative cardiac T1 mapping for patients unable to undergo breath-holding MRI acquisitions by enabling motion-robust accurate T1 parameter estimation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
Novel formulation: The paper presents a novel physically-constrained deep-learning model (PCMC-T1) for motion correction in free-breathing T1 mapping. By incorporating the signal decay model into the network architecture, the authors encourage physically-plausible deformations along the longitudinal relaxation axis, which is an interesting and innovative approach.
-
Original use of data: The PCMC-T1 model uses a publicly available free-breathing quantitative T1 MRI dataset of 210 patients, and the authors demonstrate the added value of their approach over baseline deep-learning-based image registration methods. This original use of data helps to validate the effectiveness of the proposed method.
-
Clinical feasibility: The paper demonstrates the clinical feasibility of the proposed method by conducting a semi-quantitative expert cardiac radiologist review, which showed improved results compared to baseline methods. This evaluation highlights the potential clinical impact of the PCMC-T1 model.
-
Strong evaluation: The authors employed a 5-fold experimental setup to evaluate the performance of their model compared to baseline methods, using several evaluation metrics such as R2, Dice score, and Hausdorff distance. The results show that the PCMC-T1 model achieved the best mean model-fitting R2 and comparable Dice score, suggesting its effectiveness in addressing motion correction in T1 mapping.
-
Broad applicability: The PCMC-T1 model holds the potential to broaden the application of quantitative cardiac T1 mapping to patient populations who are unable to undergo breath-holding MRI acquisitions. By enabling motion-robust accurate T1 parameter estimation, the proposed method could improve the diagnosis of diffuse myocardial diseases in these patients.
-
Extensibility to other qMRI applications: The proposed physically-constrained motion-robust parameter estimation approach can be directly extended to additional quantitative MRI applications, showcasing the versatility and potential of the PCMC-T1 model.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
Single-centred dataset: The study relies on a dataset from a single centre, which could limit the generalisability of the results. Employing datasets from multiple centres could improve the robustness of the model and its applicability to a broader range of situations.
-
Segmentation masks: More details on segmentation masks are needed, such as how many images were segmented and on which type of images the segmentation was performed. This information is crucial for understanding the method’s performance and potential limitations.
-
Hyperparameter optimisation: The paper lacks a proper validation set for hyperparameter optimisation, which could result in an overestimation of the model’s performance. Employing a separate validation set would provide a more accurate assessment of the method’s true performance.
-
Intra-observer analysis: For the clinically-related metric, an intra-observer analysis should be performed to ensure the consistency of the assessment. This would add credibility to the results presented in the paper.
-
Comparison with existing methods: The paper should discuss the differences and added benefits of their approach compared to similar work (e.g., https://doi.org/10.1007/978-3-031-16446-0_28), which would help in demonstrating the novelty and advantages of the proposed method.
-
Clinical significance: The paper should provide more information on the clinical significance of the numerical differences observed in the results, especially considering the qualitative assessment.
-
R2 metric discussion: The paper should discuss the fact that the R2 metric not only penalises motion between T1 weighted images but also the presence of imaging artifacts. This would provide a more comprehensive understanding of the model’s performance in addressing motion correction.
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The manuscript mentions the code will be available in a repository and the checklist supports this statement. Once uploaded, it will guarantee reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
The references [7, 9, 10] do not employ direct physically-informed deep neural networks. Only the reference [11] is a direct inclusion. Please highlight the specific contributions of these papers.
-
I would suggest capitalising the first letters of the acronym PCMC for easier linking.
-
Please further clarify how the synthetic images from the signal relaxation model parameters predictions, and whether the synthetic output of the first encoder-decoder is the same as the synthetic output of the second encoder-decoder. Labelling each stack in Figure 2 would be helpful. The figure should be more explanatory. The second architecture is self-explanatory but the first one could be better elaborated.
-
Please clarify the difference between formula (2) and the formula in Figure 2.
-
Please give more details on the segmentation masks. Were these done in the T1 weighted images? How many images were segmented?
-
Please highlight in bold the best results in table 1.
-
Please discuss the limitations of the employed single-centred dataset.
-
Please state how the cropping was done. Was it manually performed? Would that limit usability?
-
The multi-image registration model with a mutual-information-based loss function (REG-MI) has been presented without prior explanation. Please explain.
-
For the new clinically-related metric, an intra-observer analysis should be performed to ensure the consistency of the assessment. Do the randomly selected 29 cases come from the test set? These should be from this set.
-
The term of 5-fold experimental setup is misleading, it gives the impression a cross-validation assessment was performed, but these results are not present.
-
The optimisation of hyperparameters should have been done with a validation set and not with the test set. Otherwise, the real performance cannot be assessed with the same set employed to improve it.
-
It is counterintuitive that the model enforces a clearer delineation between the blood and the muscle while the Dice coefficient is lower than its counterpart. Were the segmentation masks correctly performed? Please discuss.
-
The proposed approach is similar to this work: https://doi.org/10.1007/978-3-031-16446-0_28. Please discuss the difference and the added benefits.
-
While the analysis conducted does show numerical differences, beside the qualitative assessment, are these significant? The results are not fully convincing. It would have been useful to have the clinical scores in the other 2 shown methods too.
-
The R2 not only penalises the motion in between T1 weighted images but also the presence of imaging artifacts. Please discuss.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I recommend acceptance of this paper due to its innovative approach in using a physically-constrained deep-learning model, PCMC-T1, for motion correction in free-breathing T1 mapping. The paper demonstrates the advantages of the proposed method over baseline deep-learning-based image registration approaches. The method has the potential to improve T1 mapping for patients unable to undergo breath-holding MRI acquisitions, enabling motion-robust T1 parameter estimation. I acknowledge some limitations highlighted in the paper: the use of a single-centred dataset, the lack of clarity in cropping methods, potential issues with segmentation masks affecting the Dice score, and the absence of a validation set for hyperparameter optimisation. These limitations should be addressed to strengthen the paper and provide a more comprehensive understanding of the method’s performance and applicability.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
The authors have provided satisfactory responses to the reviewer’s comments, showing a clear understanding and addressing most concerns raised effectively. They have committed to further improve the clarity and completeness of the paper, promising to incorporate the reviewers’ feedback in the final version. I recommend acceptance of this paper.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers are generally positive about the quality of the work. One reviewer raised concerns about the metric and performance. Please address the concerns in the rebuttal.
Author Feedback
We thank the reviewers and meta-reviewer for providing valuable insights and comments on our manuscript. We have taken careful consideration of the main concerns raised. We will diligently incorporate the reviewers’ comments into the final version of the manuscript. R1 Synthetic images refer to the images generated from the estimated parametric maps (M0, T1) at various inversion times using the MRI signal decay model. These synthetic images are expected to exhibit similar intensity characteristics to their corresponding acquired images, thereby allowing the use of the L2 norm as a registration loss. We intend to clarify this concept in the final version. Considering the precious radiologist time, we focused on selecting methods that demonstrate state-of-the-art (SOTA) results in terms of quantitative metrics such as DICE and HD for clinical evaluation. The purpose of presenting the results of the below SOTA methods is to gain insights into the influence of each component in our model. In the final version, we will include the missing scores and thoroughly discuss the clinical score results. R2 We apologize for any confusion regarding the role of the R-squared metric in our study. In T1 mapping, achieving “correct registration” encompasses multiple aspects. Firstly, the registration should align organ boundaries accurately, ensuring anatomical correctness (evaluated using metrics such as Dice and HD). Additionally, we expect the signal along the inversion time axis to adhere to the signal decay model, making it a regression task assessed by pixel-wise R-squared. As pointed out by the reviewer, synthmorph exhibited slightly better Dice and HD scores than our method. However, our approach outperformed synthmorph in terms of both R-squared and clinical scores. This discrepancy highlights the inherent complexity of the problem we are addressing. To tackle this complex question, we adopted a rigorous evaluation approach that quantifies the various perspectives of the problem: anatomical (Dice, HD), physical (R-squared), and clinical (clinical score). This comprehensive evaluation strategy is illustrated in Figure 3, where synthmorph demonstrates superior dice scores, but our method consistently achieves better R-squared values and improved visual quality. Our approach aims to incorporate these various cues to drive the registration system. While the inclusion of the “auxiliary” segmentation loss serves as a crucial factor in ensuring anatomically plausible registration, the added value of our approach lies in the signal decay model loss that also provides a physically plausible registration. R3: We apologize for inadvertently omitting a significant work that aims to solve a similar challenge (https://doi.org/10.1007/978-3-031-16446-0_28) from our introduction and appreciate your bringing it to our attention. In the final version, we will ensure its inclusion. Specifically, the approach presented in the mentioned work employed a sequential process. Initially, it aimed to separate the changes in intensity resulting from different inversion times from the fixed anatomical structure. However, this process heavily relied on perfect disentanglement of the anatomical structure from the contrast. Additionally, the registration was performed solely between the disentangled anatomical images without considering the adherence of the signal along the inversion time axis to the signal decay model. Consequently, this method cannot guarantee a physically plausible registration in terms of conforming to the signal decay model. In contrast, our approach is a simultaneous method that collectively considers the entire set of images. Consequently, it does not rely on prior errors and can provide a physically plausible registration. Segmentation masks: In our study, we utilized the segmentation masks provided by the dataset, as described in ref. [4]. Due to space limitations, we regrettably omitted these specific details from our paper.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper presents a novel method for an interesting application of heart tissue characterization (free-breathing T1 mapping). The rebuttal also addressed most of the remaining concerns raised by the reviewers (except for R2). I agree with R1 and R3 that this paper merits acceptance given its quality and novelty. Congratulations!
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal has satisfactorially addressed all my main concerns. Overall, this is an very interesting paper with novel formulation to improve T1 mapping image quality. It is worthwhile being discussed in MICCAI.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal addressed the major concerns. Although there remain a couple of points to be further clarified, I think this paper is worthy to be presented in MICCAI. I would recommend the acceptance of this paper.