List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Samuel W. Remedios, Shuo Han, Yuan Xue, Aaron Carass, Trac D. Tran, Dzung L. Pham, Jerry L. Prince
Abstract
In 2D multi-slice magnetic resonance (MR) acquisition, the through-plane signals are typically of lower resolution than the in-plane signals. While contemporary super-resolution (SR) methods aim to recover the underlying high-resolution volume, the estimated high-frequency information is implicit via end-to-end data-driven training rather than being explicitly stated and sought. To address this, we reframe the SR problem statement in terms of perfect reconstruction filter banks, enabling us to identify and directly estimate the missing information. In this work, we propose a two-stage approach to approximate the completion of a perfect reconstruction filter bank corresponding to the anisotropic acquisition of a particular scan. In stage 1, we estimate the missing filters using gradient descent and in stage 2, we use deep networks to learn the mapping from coarse coefficients to detail coefficients. In addition, the proposed formulation does not rely on external training data, circumventing the need for domain shift correction. Under our approach, SR performance is improved particularly in “slice gap’’ scenarios, likely due to the constrained solution space imposed by the framework.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_58
SharedIt: https://rdcu.be/cVRT1
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper perfect reconstruction filter banks to perform MR super resolution in the out-of-plane axis. The pipeline is split into 2 stages; stage1 randomly samples 1D row/columns from in-plane slices to train analysis and synthesis filter banks (apart from H_0 that’s fixed to the PSF), and stage2 where 2D training pairs of y (output of H_0) and { d_1 to d_(M-1) } are trained with pix2pix.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper is novel in the way that the problem is modelled from a Digital Signal Processing (DSP) perspective instead of traditional Deep Learning model design. Authors have compared their method to traditional upsampling methods (i.e. b-spline) as well as a SOTA deep learning method (SMORE) and have shown an improvement in performance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There aren’t any weaknesses from my mindset
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The method should be reproducible from the information provided in the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper is overall well written, and makes mathematical sense. I only have a few questions hoping the authors can clarify.
Is there a reason why H1 - H_(M-1) and F_0 - F_(M-1) are modelled as 1D filters? Can they be 2D since, in stage2, y and d training pairs are 2D?
NIT: Can the authors clarify in text what is the exact value of M used in experiments? From Fig3, I assume M=5. Similarly with the order of the filters (k, I assume M-1?). How does changing the value of M and k affect performance of the model?
NIT: What is the runtime of the model to create a HR image? and how does this compare to B-Spline or SMOTE?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is overall well written, an interesting take of SR using DSP methods (albeit not pure DSP since detail coefficients “d” are predicted using pix2pix). Authors have shown the method is comparable with SMORE, a SOTA method.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The paper does super-resolution (SR) of thick-sliced MR single-modality images by a training-free method (“self-super-resolution”), which casts recovering the unknown high-resolution (HR) image in the context of perfect reconstruction filter banks. They fit the parameters of the filter bank from 1D rows and columns from in-plane slices using gradient descent, and the detail coefficients using a regression CNN on 2D patches extracted from in-plane slices. Finally, the HR image is recovered by encoding the input image with the first set of filters, regressing the detail coefficients from the encoded image, and then decoding the coefficients with the last set of filters. The method is compared to simple resampling, which it outperforms, as well as another self-super-resolution technique where it performs favorably for larger slice thicknesses.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This is a good paper, both well written and well structured, with a novel methodology and solid validation. I am not very familiar with perfect reconstruction filter banks, but I get the idea of the paper, and the results are promising and convincing. A method that can super-resolve high-quality SR of thick-sliced hospital MRIs - without requiring training data - is valuable, and is what is presented in this work.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
My main concern with this work is its applicability in a more realistic scenario – when slice-thickness is not an integer value. This is something that is not covered in the discussion and conclusions section. As this is a very common scenario in routine clinical data I would like an elaboration (or discussion) on this extension of the work in the paper? From looking at the model, my feeling is that this could be a non-trivial task?
Furthermore, as the model is fitted on an image-to-image basis, using two first-order optimization steps, I would like details on runtime. This is particularly important if large populations of retrospective hospital MR images are to be processed, in a feasible time.
Finally, it is a bit concerning that the results section shows a drop in reconstruction performance in the thick-slice direction, seeing this is the direction of interest. It may very well be that this is because data of this type is missing during training; however, the nature of the problem this paper is tackling is such that this type of data will not be available. Or if it was, could it be included in your model somehow? Please elaborate.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good. The paper presents their results with statistical significance, which I appreciate. The authors state that the code will not be made publicly available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Fig 1: It would be nice to add the analysis/synthesis terms to this figure, and also the encoded/decoder analogy (as this is something that DL readers will appreciate). Maybe as boxes around the relevant parts?
“neural SR methods”, I am not sure I understand what this means, please explain.
Could h_0 be learned from the data, without using SMORE as a pre-processing step, or would this break the theory?
Why is a Gaussian used as part of the low-resolution generating process, and not the estimated slice profile? Would not the latter stay truer to the forward generating process?
As the method shares similarities with earlier work in SR - which casts the recovery of the HR image as an inverse problem - some of this work could be referenced in the literature review.
Fig 4: The Low Resolution scans would be better visualized using nearest neighbor interpolation.
Routine clinical scans are often acquired with a number of MR contrasts. It would be nice with a comment in the Discussion section if the model could be extended to this ‘multi-channel’ scenario.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Methodological novelty with a solid validation, and a well written paper to boot.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
(1) This paper proposed a novel filter bank formwork for super-resolution of anisotropic MR brain images based on filter bank theory. (2) The structure of the network is interpretable. (3) Network training does not require extra training data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) This paper proposed a novel formwork which is supported by theory. (2) Experiments verify the effectiveness of the proposed method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) The method is not adequately described, for example, how does the data flow in the model? How does the data flow in the model? How do we manipulate the filter? How do we train generator? (2) Experimental validation cannot support the benefits of the framework described in the Page2-3. How does the method explicitly synthesize the missing high frequencies? The dynamic capacity for lower-resolution images? The robustness of the method?
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper has the reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
(1) Please introduction the method in more detail. (2) Please conduct more detailed experiments to verify the advantages of the proposed framework with theoretical guarantee.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The writing is not clear, and the experiment is inadequate, so I do not recommend acceptance.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
6
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors discuss super-resolution of thick-sliced MR single-modality images by a training-free method, which casts recovering the unknown high-resolution (HR) image in the context of perfect reconstruction filter banks. The reviewers agree that this is a good paper with some minor weaknesses that must be addressed in the camera-ready version
- please comment on applicability in a more realistic scenario
- the use of 1D filters instead of 2D
- experimental choices regarding M
- runtime
- data flow in the model and details about training
- robustness of the method
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Author Feedback
N/A