Authors

Joseph Kettelkamp, Ludovica Romanin, Davide Piccini, Sarv Priya, Mathews Jacob

Abstract

We propose an unsupervised deep learning algorithm for the motion-compensated reconstruction of 5D cardiac MRI data from 3D radial acquisitions. Ungated free-breathing 5D MRI simplifies the scan planning, improves patient comfort, and offers several clinical benefits over breath-held 2D exams, including isotropic spatial resolution and the ability to reslice the data to arbitrary views. However, the current reconstruction algorithms for 5D MRI take very long computational time, and their outcome is greatly dependent on the uniformity of the binning of the acquired data into different physiological phases. The proposed algorithm is a more data-efficient alternative to current motion-resolved reconstructions. This motion-compensated approach models the data in each cardiac/respiratory bin as Fourier samples of the deformed version of a 3D image template. The deformation maps are modeled by a convolutional neural network driven by the physiological phase information. The deformation maps and the template are then jointly estimated from the measured data. The cardiac and respiratory phases are estimated from 1D navigators using an auto-encoder. The proposed algorithm is validated on 5D bSSFP datasets acquired from two subjects.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43999-5_40

SharedIt: https://rdcu.be/dnwwT

Link to the code repository

https://github.com/joseph-kettelkamp/5D_MRI

Link to the dataset(s)

https://drive.google.com/drive/folders/1E1FvKVhR1ye_xVW-z9_6E65G_R3kFckq?usp=share_link

Reviews

Review #1

Please describe the contribution of the paper

The paper proposes a motion-compensated reconstruction algorithm for 5D MRI, enabling respiratory and cardiac motion-compensated 3D imaging. The approach models images as a deformed version of the image template, with non-linear mapping of low-dimensional latent vectors modeled by a CNN.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. An auto-encoder was used to estimate the latent vector (physiological phase characteristics) from navigators.
2. The paper proposes motion-compensated image recovery, which is an improvement over motion-resolved reconstruction.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. The study only evaluates the proposed algorithm on two MRI data sets.
2. The paper lacks comparison studies and presents only preliminary results.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Two MRI data sets were used for evaluation. The reproducibility is limited.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

As noted, the study is currently limited in its preliminary nature. Future studies should be conducted to further evaluate the proposed algorithm’s performance and compare it with other motion-resolved reconstruction approaches.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

4
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The study is currently limited to preliminary results.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper introduces a deep learning-based motion-compensated reconstruction algorithm for 5D cardiac MRI data from 3D radial acquisitions. The cardiac and respiratory phases are estimated from 1D navigators using an auto-encoder, and the deformation maps modeled by a convolutional neural network (CNN) driven by the phase information. The proposed approach is validated on 5D bFFSP in vivo datasets (n=2). The results demonstrate that the proposed approach is capable of resolving the cardiac motion and offering similar image quality for all the different phases.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed method contains several interesting and novel perspectives, such as estimation of motion maps using a CNN and latent vectors, the estimation of latent vectors from the superior-inferior navigators using an auto-encoder, and the joint recovery of the static image template using the motion compensated image recovery. The in vivo results were well presented, demonstrating its superior performance over the competing motion-resolved approach.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

There are several weaknesses. First, the description of the acquisition scheme is inadequate in Section 2.1. More information on the imaging parameters should be provided. It would be desirable to include figures to illustrate the acquisition scheme. Furthermore, please provide more information on the justification for disabling fat saturation pulses and ramp-up sampling. Second, no ablation study was presented. Third, the comparison should incorporate more competing 5D-MRI methods.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The author stated to meet all reproducibility requirements.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Please see the answer to Questions 6. (Weaknesses)
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This paper proposes an intriguing deep-learning approach for motion-compensated reconstruction of 5D cardiac MRI data. The approach utilizes CNN to learn motion maps, and an auto-encoder to estimate latent vectors from superior-inferior navigators. In my opinion, the paper’s strengths outweigh its weaknesses. As such, I have given it a rating of 5.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

The main contributions of this paper are 1) motion-compensated reconstruction, instead of motion-resolved reconstruction for 5D MRI; 2) Jointly estimation of deformation maps and image template from the measured data; 3) modeling deformation maps by a convolutional neural network driven by the physiological phase information.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The technical part of this paper is very interesting. It consists of several novel aspects: 1) motion-compensated reconstruction, instead of motion-resolved reconstruction for 5D MRI; 2) Jointly estimation of deformation maps and image template from the measured data; 3) modeling deformation maps by a convolutional neural network driven by the physiological phase information.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Validation on more subjects or even patients could make this work more solid.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

While there are no links to code or data provided, the authors promised to provide code and pre-trained models upon acceptance.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Summary:

In this work, the authors proposed an unsupervised deep learning algorithm for motion-compensated reconstruction of 5D cardiac MRI data from 3D radial bSSFP acquisitions. This motion-compensated approach models the data in each cardiac/respiratory bin as Fourier samples of the deformed version of a 3D image template. The deformation maps are modeled by a convolutional neural network driven by the physiological information, i.e. cardiac and respiratory signals. The image template and the deformation maps are jointly estimated, with the cardiac and respiratory signals estimated from 1D navigators using an auto-encoder. The authors further validate their algorithm on two in-vivo data sets. This work is very interesting. The manuscript is very well-written and the evaluations are well-designed. I only have one minor comment:

Page 7, Fig 2: Could you please explain orange and green curves in more detail. Thank you!
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The three main technical contributions make this technique novel. Moreover, the application to 5D cardiac MRI is of high clinical value.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The majority of reviewers recommended the acceptance of the paper. The meta reviewer carefully reads the reviewers’ feedback and agrees with their scores. The authors should address the mentioned issues and provide a more comprehensive evaluation of their proposed method’s performance, such as conducting an ablation study and further evaluation of the method.

Author Feedback

We thank the reviewers for the constructive feedback. We are happy that the reviewers found the motion compensated approach that jointly estimates the image template and the deformation maps, which are modeled by a CNN. We address the main criticisms raised by the reviewers below.

(a) lack of details on the acquisition scheme and the need to remove fat-sat pulses.

We have added more details to the acquisition scheme and a new figure to address this concern. Please see added details.

In vivo acquisitions were performed on a 1.5T clinical MRI scanner (MAGNETOM Sola, Siemens Healthcare, Erlangen, Germany). The free-running research sequence used in this work is a bSSFP sequence, in which all chemically shift-selective fat saturation pulses and ramp-up RF excitations were removed to reduce the specific absorption rate (SAR) and to enable a completely uninterrupted acquisition [8]. K-space data were continuously sampled using a 3D golden angle kooshball phyllotaxis trajectory [7], interleaved with the acquisition of a readout oriented along the superior–inferior (SI) direction for cardiac and respiratory self‐gating [11]. The main sequence parameters are: radio frequency excitation angle of 55 with an axial slab-selective sinc pulse; resolution of 1.1 mm3; FOV of 220 mm3; TE/TR of 1.87/3.78 ms; and readout bandwidth of 898 Hz/pixel. The total fixed scan time was 7:58 minutes.

(b) the comparison with competing 5D-MRI methods

We note that the state-of-the-art 5D MRI method is XD-GRASP. We have compared our approach against XD-GRASP in Figure 3 of the paper. We had labeled this approach as motion-resolved reconstruction, which likely caused the confusion.

(c) lack of ablation studies.

We are currently working on ablation studies to understand the impact of the different components. Because fully sampled reference datasets are not available, we are using a numerical phantom for the ablation studies. We plan to report these results at the MICCAI meeting.

(d) the evaluation is restricted to two MRI datasets

We are currently working on validating the approach on more datasets, which we plan to report at the MICCAI meeting. Please note that these datasets are not available from repositories, and there are IRB restrictions on sharing such patient datasets across sites. We have recently installed the sequences in our local hospital and are in the process of acquiring patient datasets.

back to top

Motion Compensated Unsupervised Deep Learning for 5D MRI