List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Changchun Yang, Yidong Zhao, Lu Huang, Liming Xia, Qian Tao
Abstract
Quantitative MRI (qMRI) of the heart has become an important clinical tool for examining myocardial tissue properties. Because heart is a moving object, it is usually imaged with electrocardiogram and respiratory gating during acquisition, to “freeze” its motion. In reality, gating is more-often-than-not imperfect given the heart rate variability and non-ideal breath-hold. qMRI of the heart, consequently, is characteristic of varying image contrast as well as residual motion, the latter compromising the quality of quantitative mapping. Motion correction is an important step prior to parametric mapping, however, a long-standing difficulty for registering the dynamic sequence is that the contrast across frames varies wildly: depending on the acquisition scheme some frames can have extremely poor contrast, which fails both traditional optimization-based and modern learning-based registration methods. In this work, we propose a novel framework named DisQ, which Disentangles Quantitative mapping sequences into the latent space of contrast and anatomy, fully unsupervised. The disentangled latent spaces serve for the purpose of generating a series of images with identical contrast, which enables easy and accurate registration of all frames. We applied our DisQ method to the modified Look-Locker inversion recovery (MOLLI) sequence, and demonstrated improved performance of T1 mapping. In addition, we showed the possibility of generating a dynamic series of baseline images with exactly the same shape, strictly registered and perfectly “frozen”. Our proposed DisQ methodology readily extends to other types of cardiac qMRI such as T2 mapping and perfusion.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_28
SharedIt: https://rdcu.be/cVRTa
Link to the code repository
https://github.com/Changchun-Yang/DisQ
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This work proposed a novel disentanglement framework for learning latent space of cardiac Quantitative MRI (qMRI) for contrast and anatomy separately, to benefit the downstream image registration and quantitative mapping.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) The proposed contrast-anatomy disentanglement framework suit well for cardiac qMRI problems as this paper promoted. Cardiac MRIs often has different object motions and contrasts across sequences, causing difficulties of quantitative mapping. Thus a disentangled representation of contrast and anatomy can help unify the baseline images, as demonstrated in experiments using T1 mapping. (2) The writing of this paper is very good and easy to follow. (3) Both qualitative results for disentanglement and quantitative results for baseline comparison and ablation studies showed that the proposed method is promising.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Major concerns: The double-encoders approach is a common practice for disentanglement, however, it often requires additional contrast loss or adversarial loss for different modules to prevent them from learning the same representations. (One recent paper I remember is [1].) In this paper’s case, for example, the contrast module need to be forced not to learn anatomy representation and vice versa, such that they are disentangled. I am a little supervised that the cross-reconstruction alone can achieve disentangled contrast and anatomy. For bootstrapping disentanglement, this work proposed a similarity constraint for anatomy (Eqn (4)) and an information bottleneck constraint for contrast (Eqn(5)). However, it is not clear to me how this can prevent one module from learning representation from another? Maybe the way of combining anatomy and contrast as in Eqn(6)? It would be better that this work can discuss more on how the disentanglement is promoted and achieved. Some metrics of dismantlement will also be helpful besides figure (3).
Minor concerns: It would be better to be more specific on the drawbacks of “Groupwise“ baseline for readers to have a sense on the tradeoff between performance and computational cost. Instead of saying Groupwise “demanded lengthy optimization“, maybe consider pointing out how roughly the computational time or resources increased?
[1] Harada, Shota, et al. “Order-Guided Disentangled Representation Learning for Ulcerative Colitis Classification with Limited Labels.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2021.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I didn’t find obvious problems of reproducibility. Most of hyper-parameters are mentioned, especially for the loss function. Network architecture seems missing but this work intends to release its code thus it should not be a big problem.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
It would be better to add some discussion or metrics to the concerns of disentangelement described above.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Generally this paper is well-written, solving important and specific problem with novel solution and valid experiment results.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The manuscript presents an unsupervised pair-wise registration pipeline with disentangled latent space of contrast and anatomy for a sequence of MOLLI T1-weighted images with inherent varied contrast and residual motion. Specifically, the developed neural network architecture decomposes the input pair into synthetic representations with same contrast, preserving anatomy, and same anatomy, preserving contrast, using U-Nets, for an easier pair-wise registration with Voxelmorph. The performance was evaluated by means of the reconstructed T1 fitting error map in the myocardial region and proven to be higher than the vanilla Voxelmorph approach. The application of disentangled representations for motion correction in T1 maps is novel.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Overall, it is a well written manuscript with supporting validation.
- The application of disentangled representation for straight unsupervised registration in T1-weighted images is appealing.
- The proposed framework has been well explained.
- The method achieved an improved precision compared to the vanilla Voxelmorph.
- The ablation study is sound and validates the improvement of each component in the proposed pipeline.
- The proposed work tackles the clinical need for robust qMRI with a different, useful approach (disentanglement framework) contributing to already existing deep learning-based methods for motion correction in T1 mapping.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Myocardial motion correction in T1 mapping has already been addressed [[1, 2]], but the disentanglement representation in this field is novel.
- The employed models did not present execution time.
- Besides showing a higher precision made by “Groupwise”, no further limitation has been discussed.
- Although the paper properly validated the proof of concept, the amount of data is highly limited, not well described, and conclusions may be limited by the generalisability constraints.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The manuscript mentions the code will be available in a repository and the checklist supports this statement. There are some little details not explained in the paper, but it may be out of the scope of a conference paper. Once uploaded, it will guarantee reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- In the abstract, I would suggest mentioning the proposed method is a pair-wise registration method.
- In the abstract and introduction, the authors affirm that modern learning-based registration methods also fail on sequences with highly varied contrast, whereas the tested vanilla Voxelmorph did reduce the observed precision and existing work on deep learning-based methods for motion correction in T1 mapping [[1, 2]] also presented promising results. Unless proven otherwise in the study, I would suggest changing the tone on the limitations of the learning-based registration methods.
- The units of the plot 1(b) should be shown.
- The literature review did mention the classic motion correction method [22] and a robust PCA-based method [7, 21], but the cited learning-based methods [1, 16, 17, 18] did not address the issue of motion artefacts in myocardial T1 mapping as in [[1, 2]]. I would suggest opting for a more conservative tone as this issue has already been addressed. The useful contribution of the paper is in the disentanglement framework.
- The population of the dataset of MOLLI acquisitions should be explained at least in terms of number of subjects, centres, cardiovascular conditions, number of acquisitions with pre and post-contrast, and the ethical clearance.
- The implementation details should include the amount of computing time at least in a GPU.
- The selection of t = 5 is not properly justified, the results with another selection are not presented in the paper, only mentioned that it was not sensitive. In case the selection of t where the T1-weighted has the lowest contrast still achieved the same performance, it would be a missed benefit of the model that is worth describing.
- In Results, it should be specified that these are the results from the test set.
- Table 1 may have a typo in “Ori”, which was previously described as “Org”.
- Table 1 may be improved by including the execution time in each method. This may prove the efficiency of the proposed method compared to “Groupwise”.
- Further brief clarification on the implementation of “Morph” may be needed. I assumed it was the same Voxelmorph with the input pair as is.
- In Results, I would update “significantly” by “substantially”, as statistical significan was never quantified.
- The presented results showed an improved performance in terms of precision, measured as the fitting parameters variability, but not necessarily accuracy, which may be validated in phantom experiments, as stated by the authors.
- In Qualitative analysis, it should be mentioned it is a case study as one case may not me representative for all the test set.
- In conclusions, to support the statement that the method is generic, further evaluation may be required, i.e., evaluating the performance in pre and post-contrast separately, in an independent dataset, etc.
- It may be worth mentioning future studies will involve development or validation in extended datasets.
- I would suggest ordering the references by their appearance.
[[1]] Arava, D., Masarwy, M., Khawaled, S. and Freiman, M., 2021, November. Deep-Learning based Motion Correction for Myocardial T1 Mapping. In 2021 IEEE International Conference on Microwaves, Antennas, Communications and Electronic Systems (COMCAS) (pp. 55-59). IEEE. [[2]] Gonzales, R.A., Zhang, Q., Papież, B.W., Werys, K., Lukaschuk, E., Popescu, I.A., Burrage, M.K., Shanmuganathan, M., Ferreira, V.M. and Piechnik, S.K., 2021. MOCOnet: Robust Motion Correction of Cardiovascular Magnetic Resonance T1 Mapping Using Convolutional Neural Networks. Frontiers in Cardiovascular Medicine, 8.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The presented work aims to ease the registration complexity in a real clinical need, disentangling both contrast and shape, which is quite novel in myocardial motion correction. My comments will hopefully improve the quality of the paper and clarify relevant aspects.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This paper addresses the issue of improving cardiac quantitative MRI (qMRI) such as T1 mapping by proposing an image disentanglement method, called DisQ (Disentangling Quantitative MRI), to discompose cardiac qMRI images into their anatomical representation and contrast representation in the latent space.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- A novel formulation of cardiac qMRI image disentanglement
- A novel network architecture for cardiac qMRI
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Not appropriately justify the medical motivation of the work
- Unclear presentation of the results
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No elements on paper reproducibility could be found in the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This paper addresses the issue of improving cardiac quantitative MRI (qMRI) such as T1 mapping by proposing an image disentanglement method, called DisQ (Disentangling Quantitative MRI), to discompose cardiac qMRI images into their anatomical representation and contrast representation in the latent space. The idea is new, but the medical motivation is not convincing. Throughout the paper, the authors used the term “cardiac qMRI”, but the shown images contain not only the heart but also other organs or tissues, so that it was very difficult to precisely assess the improvement of cardiac T1 map. For ex., it is difficult to assess real interest brought by the proposed method with respect to the original T1 map (Fig. 4) on the myocardium.
From a medical-imaging point of view, the “contrast” meant by the authors is not really a contrast problem. Instead, it is a signal loss problem. In this sense, such understanding of the problem may call into question the basis of the proposed method.
Another important issue in this kind of methods is the sensitivity to noise. This point was not addressed in the paper.
Some typos:
- by substituted Lanatomy with: substituting
- one common factors: factor
- will be present: presented
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The medical motivation of the work was not appropriately justified
- The presentation of the results was confusing and unclear
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The work is interesting and the method proposed is reasonably novel and valid. Evaluation on the disentanglement and results could be further improved.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Author Feedback
We thank all the reviewers for reviewing the paper and all their insightful and encouraging comments. We address a few issues point-by-point and will modify the manuscript accordingly in the final version.
【Computational time (R1 & R3)】
- The training time for our disentanglement architecture is ~10h on one 3090Ti GPU, and ~300ms for inference on a pair of cross reconstructed images. For the registration network, we use the original Voxelmorph [1] (R3-11), and training time is ~6h and evaluation time is ~400ms for pairwise registration.
- For Groupwise registration by ELASTIX toolbox, the inference time is ~9000s. In comparison, our pipeline only takes ~7s for disentangling plus registering (One MOLLI sequence has 11 images).
【Implementation details】
- R3-9, R4: Typo errors: Thanks for your careful reviewing. We will correct all typos in final version.
- Meta, R1: Metrics of disentanglement: We agree with R1’s insightful remarks on disentanglement. There has not been a unanimous standard for the evaluation of disentanglement: Liu, Xiao, et al. “A Tutorial on Learning Disentangled Representations in the Imaging Domain.” arXiv: 2108.12043 (2021). Limited by length of manuscript, there was no space to present the results of anatomy and contrast. We found that Gumbel softmax and capacity of our architecture to be the two important factors underlying successful disentanglement in cardiac MRI. The results of precise mapping can quantitatively prove the accuracy of the disentanglement.
- R1: Why successful disentangling: Based on our extensive experiments during research phase, Gumbel softmax in anatomy encoder and capacity constraint in contrast encoder together (Information bottleneck theory [24]) facilitate successful disentanglement and cross-reconstruction. As we mentioned in the paper, more studies are warranted for investigation in this direction.
- R3-2,4: Other work about T1 mapping in [[1,2]]: The method used in [[1,2]] first performed motion registration and then parametric fitting. Although their results are improved compared with original fitting, the registration network will not work as well if test images have poor or different contrast: Hoffmann, Malte, et al. “Learning Mri Contrast-Agnostic Registration.” 2021 IEEE ISBI. The focus of our work is to make registration easier from a disentanglement perspective, which then improves mapping quality.
- R3-5: MOLLI dataset: We will add more detailed description (and ethical clearance) on the MOLLI dataset in the revision.
- R3-7,15,16: further evaluation: Thanks for these constructive comments. Since a pair of images can be projected into the same contrast space by disentangling, it is not sensitive to the choice of t. We also empirically validated it but due to space limit we could not add more experiments. More thorough validation of results about t and further evaluation on independent and extended datasets will be added in future releases.
- R4: medical motivation: Quantification of myocardial tissue (T1 as an example) is among the most important applications of qMRI. Our motivation is to find an effective solution, from the disentanglement angle, that can robustly solve the motion correction problem in cardiac qMRI. Our work can easily be extended to other mappings (T2, perfusion).
- R4: results and cardiac T1 map: In MOLLI sequence, the imaging results include the region outside the heart. But our quantitative results show the mean and sd of fitting T1 maps (Table 1) are only within the myocardium region (manually annotated by radiologists).
- R4: contrast: The definitions of contrast and anatomy are our disentangling factors, and a medical image can be modeled as contrast and anatomical representations, which is analogous to style and content of natural images in computer vision.
- R4: noise: Our datasets are realistic clinical data with acquisition noise, and we will test noise-resistance when validating on extended datasets in future work.