List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Chang Gao, Shu-Fu Shih, J. Paul Finn, Xiaodong Zhong
Abstract
The recent development of deep learning combined with compressed sensing enables fast reconstruction of undersampled MR images and has achieved state-of-the-art performance for Cartesian k-space trajectories. However, non-Cartesian trajectories such as the radial trajectory need to be transformed onto a Cartesian grid in each iteration of the network training, slowing down the training process significantly. Multiple iterations of nonuniform Fourier transformation in the networks offset the advantage of fast inference inherent in deep learning. Current approaches typically either work on image-to-image networks or grid the non-Cartesian trajectories before the network training to avoid the repeated gridding process. However, the image-to-image networks cannot ensure the k-space data consistency in the reconstructed images and the pre-processing of non-Cartesian k-space leads to gridding errors which cannot be compensated by the network training. Inspired by the Transformer network to handle long-range dependencies in sequence transduction tasks, we propose to rearrange the radial spokes to sequential data based on the chronological order of acquisition and use the Transformer network to predict unacquired radial spokes from the acquired data. We propose novel data augmentation methods to generate a large amount of training data from a limited number of subjects. The network can be applied to different anatomical structures. Experimental results show superior performance of the proposed framework compared to state-of-the-art deep neural networks.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_69
SharedIt: https://rdcu.be/cVRUd
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a new deep-network, based on Transformers, to reconstruct radially-sampled K-space data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The network seems to perform well. Using the raw k-space data to predict missing samples is a really interesting approach and the application of Transformers in this context seems very relevant.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The description of the method is, at times, hard to follow. The authors could have used cross-validation to evaluate their results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No data or software provided.
Hard to reproduce the algorithm based on the description.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Given that the authors effectively interpolate data in k-space, it would be interesting to see the accuracy on that domain. How well does this mehtod generalize to further undersampling?
Is the ordering of the data essential for MR reconstruction?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is a very interesting approach with possible extensions in the future.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The main contribution of this paper is the projection of k-space and the application of the transformer in utilizing both the real and imaginary parts of the k-space in prediction in the k-space domain.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of this paper is the application of the transformer network in predicting the skipped radial spokes. The radial spokes were acquired sequentially which makes the transformer network a good fit.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The real and imaginary part was stacked together in a big array for network input. Should they be separated for two networks and combined? If this is the better case, a comparison is needed.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors did not comment on the reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- In Fig. 1b, the spokes were inverse FFT transformed to generate a projection vector. What is the projection vector mean and how does the vector be generated?
- Was the data consistency be implemented via the projection step? Based on Fig. 1e, it’s not clear where the data consistency was implemented.
- In the 3.1 data sets section, it’s not clear whether the radial imaging was acquired in a full sampling scheme or undersampled. Were the training and testing based on fully sampled images and retrospectively undersampled images?
- After using the transformer network to predict the radial spokes, how the final image was reconstructed?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
8
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The recommendation is based on the method description and the results presentation.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
o The paper proposes a method for reconstructing GRASP MRIs by arranging the radial k-space lines as a sequence (in order of acquisition time). This sequence is then fed into a transformer network which outputs the fully-sampled radial k-space as output. Further, the paper explores multiple data augmentation techniques to improve performance. This model obtains superior results compared to previous methods on a small dataset containing scans from multiple anatomies.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The idea of treating the radial acquisition lines as a sequence is novel. It allows the authors to use the transformer network for this problem which is not obvious.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The model was compared against baselines like U-Net and squeeze and excitation network that were not designed for this problem. A better comparison would be to compare against a model like XD-Grasp. Without such comparison, it would not be possible to determine how well the proposed model performs.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors used a private dataset which makes it impossible to reproduce. They do list some of the hyper-parameters used for training the model.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Provide comparisons against relevant baselines, so it would be possible to properly judge the merits of the proposed method.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the presented method is novel, it was compared against previous models like U-Net that were designed for 2D images and not for handling radial scans of MRIs. As such, it is not possible to properly judge the merits of this method.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors propose a Transformers based network for radially-sampled K-space data to predict unacquired radial spokes from acquired ones. There are two very positive reviews and one negative review. There are concerns on the description of details of the method for filling the unacquired data, which can be addressed in the fin version. In my opinion the paper has an interesting idea and the paper can be accepted.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Author Feedback
We thank the reviewers and meta-reviewers for their valuable feedback. We have carefully read the comments and addressed them as follows.
R1.1 Accuracy in k-space domain The NMSE of the testing k-space data is 0.0385±0.0497.
R1.2 how does this method generalize to further undersampling? Further undersampling can be achieved by using a combination of multiple Transformer networks. For example, to achieve an undersampling rate of N, N-1 Transformer networks can be used. This is ongoing work, and we have plans to further test the performance of different accelerations and will report the results in the camera-ready paper.
R1.3 Is the ordering of the data essential for MR reconstruction? We used golden-angle ordering in this work but other orderings can also be considered as long as the angles of the acquired k-space lines can cover the range of 0 to 2*pi in a relatively uniform pattern. The readers can consider this approach as a k-space interpolation approach. The dependencies of the acquired k-space radial lines can be learned to predict the k-space lines that are not acquired and located between the acquired ones.
R3.0 separate real and imaginary parts into two networks and combine? We concatenated the real and imaginary parts into one network because there is a dependency between real and imaginary parts. Separating them into two networks may cause a mismatch caused by the imperfections of two networks.
R3.1 Projection vector As illustrated in Fig. 1c, the projection vector is the projection of an image at a certain angle. According to Radon transform, the projection is the 1D inverse Fourier transform of the k-space line at the same angle as the projection vector.
R3.2 Data consistency The data consistency was implemented by directly combining the acquired k-space lines and the output from the networks. As shown in Fig. 1a step 2, the acquired k-space are represented by black lines and the outputs are represented by blue/green/orange lines. The final “fully-sampled k-space” is the combination of the input (acquired k-space) and the output (predicted k-space) of the network.
R3.3 Dataset The images were acquired over-sampled and the training and testing data were based on retrospectively undersampling. We’ll add the information in 3.1 Dataset section.
R3.4 Image reconstruction We briefly mentioned the reconstruction in the last sentence of 2.1 The Overall Framework section paragraph 2. We used nonuniform Fourier transform function in Bart toolbox [22] and combined the images from different coils using adaptive coil combination [23].
R4 GRASP reconstruction We thank R4 for suggesting comparing with GRASP reconstruction method. We think it’s an excellent idea and this could be an immediate future work.
To address R1 and Meta-R’s comments about the description of the method, we will revise our manuscript to make it easier to understand.
To promote reproducible research, if the policies of our institutions permit, we will consider publishing our network code along with our final paper. Using our private dataset to evaluate the proposed method poses a limitation for reproducible research. One main reason is that we were not able to find a public dataset for radial MRI reconstruction. If the policies of our institutions permit, we will consider publishing our network training and testing data.