List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Qiaoqiao Ding, Xiaoqun Zhang
Abstract
Magnetic resonance imaging (MRI) acceleration is usually achieved by data undersampling, while reconstruction from undersampled data is a challenging ill-posed problem for data-missing and noisy measurements introduce various artifacts. In recent years, deep learning methods have been extensively studied for MRI reconstruction, and most of work treat the reconstruction problem as a denoising problem or replace the regularization subproblem with a deep neural network (DNN) in an optimization unrolling scheme. In this work, we proposed to directly complete the missing and corrupted k-space data by a specially designed interpolation deep neural networks combined with some convolution layers in both frequency and spatial domains. Specifically, for every missing and corrupted frequency, we use a K− nearest neighbors estimation with learnable weights. Then, two convolution neural networks (CNNs) are applied to regularize the data in both k-space and image space. The proposed DNN structures have clear interpretability for solving this undersampling problem. Extensive experiments on MRI reconstruction with diverse sampling patterns and ratios, under noiseless and noise settings demonstrate the accuracy of the proposed method compared to other learning based algorithms, while being computationally efficient for both training and reconstruction processes.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_64
SharedIt: https://rdcu.be/cVRT8
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
In this paper, the authors propose a learnable method for k-space data completion and filtering. The missing Fourier coefficient are interpolated using a weighted summation of its neighbors with adaptive weights. Two CNNS are applied to regularize the data in both k-space and image space. The proposed methodology solves the under sampling problem in MRI reconstruction. The authors claim that the accuracy of the proposed method compared to other learning based algorithms is computationally efficient for both training and reconstruction processes.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
In MRI related studies, the reconstruction of missing/corrupted k-space data has been a challenge. In this work, the authors have suggested a deep learning methodology for MRI image reconstruction. It considers MRI reconstruction as an interpolation problem in k-space. The interpolation scheme suggested in this paper is focusing on adaptive interpolating weights trained with DNN. For benchmarking, they have evaluated the performance of the proposed method with a few available algorithms such as zero-filling method (ZF), TV-regularization-based method, ADMM-Net and the plug-&-play methods. According to their results, it shows that their proposed method has outperformed all those work and it takes much less training and testing time. Further, the proposed method has outperformed all those methods in the presence of noise, too.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
A relatively small training 2D image dataset (300 slices for training and 21 slices for testing) has been utilized to investigate the performance of the proposed method. Its recommended to validate the claims on a larger dataset as concluding that the proposed method outperforms all the other benchmarking methods will not be a valid statement, otherwise. The article needs fixing some grammar mistakes found in a few places. Proof reading is recommended.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Experiments on a different larger dataset is recommended.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The authors have attempted to solve a challenging problem in MRI reconstruction research, which is commendable. As claimed by the authors, it outperforms all the benchmarking algorithms in terms of visual and qualitative evaluations. Further the time less time taken for training and testing is a big achievement. However, the only concern to strengthen the above statements is validating the hypothesis for larger datasets. If this could be addressed, this proposed methodology will be a huge success in this area of research.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Its a novel idea that had been investigated on s small dataset. However, comparisons with some state of the art algorithms have also been performed. This work could be further expanded and the authors claims can be validated accordingly.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The main contribution of this paper is to improve MRI reconstruction via an interpolation strategy. The authors use a k-nn strategy along with two CNNs as means of regularisation. Whilst the paper has a strong motivation the technical description is limited as well as the intuition.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
– The interpolation strategy is simple yet it seems somehow effective.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
– The technical novelty is not well-explained and several notations are missing or need clarification. Therefore, it is hard to appreciate the level of novelty. – Authors highlight in several places new optimisation schemes such as deep unrolling and PnP methods. However, it seems like the authors somehow are confused in the terminology. – The experimental comparison seems limited and a major drawback on the paper is the lack of discussion behind the fundings.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The list seems completed.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
– [Novelty] Whilst the paper has indeed a strong motivation, the novelty is appreciated as incremental. This is due to several reasons.
Firstly, there is no intuition on the interpolation strategy that is somehow connected to a naive zero filling interpolation. That is then refined through a couple of CNNs. Therefore, it is hard to appreciate the level of novelty in this work. Secondly, the paper needs a revision on the technical description. What is the norm for both terms in (4)? Why is there not a weighting parameter in both terms in (4)? Overall, the technical description is limited and no intuition is provided. Also, authors should introduce all notations, for example the Fourier operator in (1).
– [Experimental Setting] the experimental setting is not well-explained which can be perceived as not more than case studies which demonstrates only limited advantages in both qualitative and quantitative terms.
Authors are using ADNI. However, they are not explaining from which ADNI version is taking the data (e.g., ADNI-2, ADNI-GO etc.). Moreover, ADNI provides data in 3D, why did authors not show the applicability in 3D? (at least from the text it seems like they are taking 2D slices). In the medical domain, data is usually given in 3D. Authors need to clarify this part. Baselines comparison, Authors highlight in several places new optimisation schemes such as deep unrolling and PnP methods. However, their technique is not directly related to such techniques or novel in that research line. Authors mention [14] as a PnP technique. However, [14] still builds upon ADMM. Author should notice the difference between deep unrolling and PnP methods. Why not include ISTA-net[*] (another deep unrolling technique with higher performance than ADMM-net see [] ) or a PnP method. The suggested work of [] is advised to revise. Therefore, a more stronger baseline could be good to explore or justify.
[*]Zhang, J., & Ghanem, B. ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. CVPR, 2018 [**] Wei, K., Aviles-Rivero, A., Liang, J., Fu, Y., Schönlieb, C. B., & Huang, H.. Tuning-free plug-and-play proximal algorithm for inverse imaging problems. ICML 2020.
No ablation study in parameters or intuition of the model is provided. For example, with a smaller and bigger K.
All in all, the current paper has good motivation but the technical description and intuition needs to be strongly improved as well as the experimental setting.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
All in all, the current paper has good motivation but the technical description and intuition needs to be strongly improved as well as the experimental setting. Otherwise, the current version is appreciated with an incremental technical novelty and a limited experimental setting.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
The authors proposed a novel deep learning framework to perform MRI reconstruction. This framework was decomposed into three parts: the first part aimed to interpolate the under-sampled k-space using a k-near neighbor method (where the interpolation weights are learned by a deep learning method); the second part aimed to denoise the obtained interpolated k-space using a convolutional neural network; the last part aimed to denoise the reconstructed image (the image obtained from the inverse Fourier transform applied to the denoised interpolated k-space) using a convolutional neural network. The authors evaluated their method on one clinical dataset (brains MR images from ADNI). They compared the performance of their method to those of six state-of-the-art reconstruction algorithms.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The major strength of the paper relied on the novelety of the proposed method. The authors addressed the MRI reconstruction problem by interpolating directly the under-sampled k-space with a deep learning method, which seems to be not frequently done in the MRI reconstruction field. This proposed method provided better image quality endpoints values (PSNR, SSIM) than state-of-the-art methods and appeared to be computationally efficient.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The interpolation technique which is the strength of the paper (and method) should be more detailed. Additionally, the method evaluation is not enough comprehensive to ensure the generalization and reproducibility of the findings of the paper (only one dataset was considered, the dataset size is not composed of 3D volume MRIs but 2D MRI slices, the testing data set is small, no cross-validation was performed, etc).
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Some elements of the paper should be more detailed to ensure the reproducibility of the study. How do you determine the k-nearest neighbors of the missing Fourier coefficients in the k-space? Describe precisely the architecture of the deep learning method used to estimate the interpolation weights. What are the inputs and outputs of this deep learning algorithm? How much data are used for training? What is the computational time (at training and testing) of the network? What is the performance of this interpolation process? The considered dataset is small (notably the testing dataset). Due to this fact, a k-fold cross-validation will be more suitable for the fairness of the evaluation method (instead of use a simple split of the cohort in training and testing subsets). The dataset is only composed of 2D MRI slices. To ensure that the method is suitable for clinical practice the method should be evaluated on 3D MRI volumes. The method was applied and evaluated in only one clinical dataset. Several datasets (in other anatomical sites) should be considered to ensure that the method could be used in clinical practice. Describe precisely the architecture of the two denoising convolutional neuronal networks (how they work?). Please describe the data. What is the MRI scanner used for the data acquisition? How many Tesla? What are the MRI parameters (TE, TR, flip angle, acquisition time, etc)? Please perform statistical tests (such as the Wilcoxon test) to compare the significance of the differences between the proposed method and the state-of-the-art methods.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper is globally well written. Please add in Fig. 3, 4, 5, 6, 7 the true images. That will help the readers to visualize the differences between the true images and the reconstructed images. Define all operators and variables used in equations 1 and 4. In section 3.1 results, the authors wrote “The best results were emphasized in bold and the second to the best results were marked in bold”. Bold appears two times in the sentence which is not matching what we can see in the tables. Please correct the typos. Discuss the limitations of the study? Discuss the performance of all methods in challenging subjects (worst cases, tumor areas, etc)?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper overall is good and clinical relevant. The method proposed by the authors has some novelty. However, they is a lack of comprehensive evaluation of the proposed method to ensure that the reproducibility of the findings of the study (that may be a consequence of the MICCAI paper format).
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
To perform MRI reconstruction, the authors presented an interesting deep learning architecture. This framework was divided into three parts: the first aimed to interpolate the under-sampled k-space using a k-near neighbour method (where the interpolation weights were learned using a deep learning method); the second aimed to denoise the obtained interpolated k-space using a convolutional neural network; and the third aimed to denoise the reconstructed image (the image obtained from the inverse Fourier transform applied to the denoised inter The authors tested their technique on a single clinical dataset (brains MR images from ADNI). They compared their method’s performance to that of six cutting-edge reconstruction methods.
All three reviewers post some positive comments to this work, for example, the proposed method has outperformed all those methods in the presence of noise, too (R1); the interpolation strategy is simple yet it seems somehow effective. (R2); This proposed method provided better image quality endpoints values (R3).
Three reviewers are somewhat confident about their concerns. Based on the review comments and my review of the work, I would say the work is interesting with potential value to investigate, but it requires further justifications in the rebuttal stage:
-
Whilst the paper has indeed a strong motivation, the novelty is appreciated as incremental. Please elaborate.
-
ADNI version needs to be specified.
-
More ablation studies: No ablation study in parameters or intuition of the model is provided. For example, with a smaller and bigger K.
-
More technical details are needed, please refer to reviewer 3’s comments.
-
The article needs fixing some grammar mistakes found in a few places. Proof reading is recommended.
-
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
We thank the reviewers and area chair for the careful evaluations. In the following, we address some major and common issues.
Intuition on the interpolation strategy and novelty in this work. The intuition is based on an adaptive interpolation technique in Fourier domain, which results in a NN model with strong interpretationbility and simplicity, especially compared to the unrolling and P&P framework commonly adopted for inverse imaging problems. The proposed method only involves one pair of FFT and IFFTT, compared to multiple evaluations of forward and backward operators for unrolling and P&P based methods. This results in an efficient scheme for both training and inference. The experiments on two datasets (ADNI and BraTS) showed the effectiveness of this network structure. There is also some ongoing investigation on the theoretical approximation analysis for the general Fourier interpolation problem and the learning procedure of for both low and high frequencies. We believe that this line of research is novel and further understanding is promising for theoretical understanding of Fourier interpolation problem.
Experiments on a different larger dataset. Comparisons to other methods. We further evaluated on a larger data sets BraTS, and add a comparison to ISTANet. It shows that our methods still outperform all the other methods on BraST datasets. Our numerical results show that our method outperform ISTANet with a large margin for both ADNI and BraST. To give some examples: under 2D Gaussian sampling, 1/3,1/4,1/5 sampling rate, noise free, in PSNR ISTANet: 33.71; 29.77; 27.00 vs Proposed: 35.89; 35.31; 32.89. In 10% noise case: ISTANnet： 24.87; 23.78; 21.67 vs Proposed: 28.53; 28.33;27.93 The detail comparison table will be included in supplementary material.
Ablation study, reproducibility and technical details.
An ablation study for the size of neighborhood K and 10-fold cross validation were performed and it show the stability of the results with respect to K and different fold of datasets. These results will be provided in supplementary file and the code will be made public in near future.
Weighting parameter in both terms in (4)? (Reviewer 3)
For the weight parameter in (4), it is standard to put a weight parameter in one term, as for minimization problem, there is only a difference of scaling. Most of technical description are standard, and we also provide a supplementary material for some necessary technical description.
- Details on ADNI data in use.
For ADNI dataset, we used ADNI1: Annual 2 Yr 1.5T after standard preprocessing, which can be directly downloaded from ADNI website. The 2D image data are taken from the same slice of 3D images. More technical detail will be completed in supplementary file.
- 2D slices and 3D MRI volumes. We note that most of methods (such as ADMM-net, ISTANet) for MRI reconstruction are based on 2D slices. Our method is also proposed based 2D slices training. However, we agree that the evaluation of the performance can be performed for 3D volumes. In the supplementary file. we provided a comparison on a whole 3D volume from BraTS, and it shows that the reconstruction quality still outperforms the other methods such as ISTANet. Visually, we observe much less artifacts along the slice direction, therefore we believe that our method can be still directly used for 3D volume reconstruction for clinical practice. To give some examples: under Radial sampling, 1/3,1/4,1/5 sampling rate, noise free, in PSNR ISTANet: 30.83；30.13； 29.08 vs Proposed: 39.10； 35.43；33.95. In 10% noise case: ISTANet: 22.24; 15.07; 14.66 vs Proposed: 25.50; 20.88; 20.40 The detail comparison table will be included in supplementary material.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I can see the authors’ effort for the rebuttal of the work; however, none reviewers changed their scores after the rebuttal that bringing up the final scores to 6/4/6. This is not high-ranked in my deck, but I am inclined to accept the work due to its scientific merits and may be worth communicating in the MICCAI.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The work focused on learnable Fourier interpolation for MRI reconstruction with deep learning. All three reviewers found the idea simple, but effective. The authors have done a good job in addressing concerns on intuition, tests on larger datasets and reproducibility. Therefore, I recommend acceptance of the paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The work proposes a MRI reconstruction method with a k-space interpolation, k-space correction and image space correction pipeline. Though the claimed novelty lies in the simple k-space interpolation scheme, it appears very relevant to other k-space interpolation/reconstruction approaches, such as RAKI and KIKI-net. However, these relevant literature is not mentioned and compared in the work, which makes the contribution and novelty of the work less obvious. In addition, the experiments were conducted on simulated data from DICOM (ignoring real phase data), which could make this problem simplified and results over promising. Experiments on complex-valued raw data (fastMRI) are suggested. The abstract also claims that the DNN structure have clear interpretability, whereas this is not explained in the work. Given these are unclear and not properly validated, I would not recommend this for acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR