List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Amaury Leroy, Marvin Lerousseau, Théophraste Henry, Alexandre Cafaro, Nikos Paragios, Vincent Grégoire, Eric Deutsch
Abstract
For interventional procedures, a real-time mapping between treatment guidance images and planning data is challenging yet essential for successful therapy implementation. Because of time and machine constraints, it involves imaging of different modalities, resolutions and dimensions, along with severe out-of-plane deformations to handle. In this paper, we introduce MSV-RegSyn-Net, a novel, scalable, deep learning-based framework for concurrent slice-to-volume registration and high-resolution modality transfer synthesis. It consists of an end-to-end pipeline made up of (i) a cycle generative adversarial network for multimodal image translation combined with (ii) a multi-slice-to-volume deformable registration network. The concurrent nature of our approach creates mutual benefit for both tasks: image translation is naturally eased by explicit handling of out-of-plane deformations while registration benefits from bringing multimodal signals into the same domain. Our model is fully unsupervised and does not require any ground-truth deformation or segmentation mask. It obtains superior qualitative and quantitative performance for multi-slice MR to 3D CT pelvic imaging compared to state-of-the-art traditional and learning-based methods on both tasks.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_15
SharedIt: https://rdcu.be/cVRSV
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
A multi-modal deformable registration algorithm is proposed, based on a combination of modern deep learning techniques, including both modality synthesis with CycleGANs, and DL-based deformable registration. It is evaluated on a large dataset of radiation therapy cases, with intra-session MRI with large slice spacing being registered to pre-operative CT scans.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Very well written paper, thorough introduction of related work & own contribution, method description, detailed and systematic evaluation on a large data set.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
This is more of a system paper, describing a well working overall system; as expected, the inherent mathematical novelty in the method is limited - however many powerful techniques are combined in a smart way and thoroughly evaluated.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Very good; it starts from the related papers that contain source code and points out the algorithmic differences and additions.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
I have only very minor remarks:
You mention X-Ray 2D-3D registration in the same context as slice-to-volume registration, however it is something fundamentally different, due to the accumulated data along a projection geometry.
You refer to the supplementary material several times I believe; make sure this is really optional to understand the manuscript (possibly rearranging figures & text accordingly).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Well written paper, describing a well working method that demonstrates in a credible manner that the state of the art is improved upon.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The work presents a contribution to image generation by modality transfer accompanied by slice-to-volume deformable registration. Authors introduce end-to-end DL-based approach including CycleGAN-based synthesis, 2-D to 3-D registration and improvement of the generation by MIND-based supervision using the registration output.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Problem important in medical practice
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Unclear usefulness in the real multi-slice to volume registration
- References not related to the problem being solved
- Results are not reproducible
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper is not reproducible, as stated in the reproducibility list.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
I have several major comments:
- It was mentioned that all volumes were resampled to given shape. Is it connected also to the MR volumes? What about the interpolation artifacts when the resolution of MR (or USG/Angio in other applications) is much lower than the related CT volume? It seems that the proposed method is dedicated more to low-quality 3-D to higher-quality 3-D instead of slice-to-volume registration.
- The reported registration time is about 2s. For registering 3-D volumes the reported time is understandable. However, the motivation behind the article is to allow real-time processing during the multi-slice to volume registration. Such a registration should be much faster to be useful during real-time applications.
- The main contribution of the article is the concept of using the generated image to perform the unimodal registration instead of directly performing the multimodal registration. The concept is interesting, however, was already explored in other works related to 2-D or 3-D registration. I suggest to more carefully review the literature and cite appropriate references.
- Even though the concept is interesting, the article is relatively chaotic and the method description is confusing. There are some minor language mistakes that should be corrected (basic grammar mistakes, can be captured by automatic text screening tools).
- The source code is not referenced or prepared to be referenced. The paper is not reproducible.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, the paper is interesting, however, chaotic and hard to follow. The concepts are not presented in clear way and the manuscript requires several passes to fully understand the presented concept.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper proposes MSV-RegSyn-Net, a novel, unsupervised, concurrent framework for modality transfer synthesis and MSV mapping in an end-to-end pipeline. The method is evaluated on two clinical datasets. The evaluation shows the mutual benefits triggered by the joint architecture, leading to better performance than state-of-the-art methods. The model is a methodological concept that is theoretically applicable to images of different modalities and quality, but also of different slice spacing, slice thickness or orientation. This has not yet been evaluated by the authors.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Novel end-to-end approach for adversarial method coupled with registration, similar adversarial methods were previously used as an intermediate step to generate synthetic data.
- The large difference in texture resolution between the two datasets used demonstrates the potential for generalization of the method.
- The parameterization of the basic methods was optimized for fair comparison.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Lack of discussion of the limitations of the proposed method.
- The influence of the slice ratio on the performance of the method was not evaluated.
- The influence of the coverage of the field of view by the 3D CT and 2D SCT slices is not discussed.
- The main innovation is the combination of state-of-the-art building blocks in one method.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- The datasets used are private.
- The code is not available, but is based on publicly available earlier methods.
- The method is generally well described and could be re-implemented to some extent, but the results are unlikely to be reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
In Fig. 1, the naming of the intermediate step as “CT-to-MR translation” is somewhat misleading. Probably, “MR-to-CT translation” is more intuitive with respect to the proposed approach.
-
Fig. 2 shows a specific case where the proposed method fails. Please discuss why the method fails and how often it fails.
-
Eq. 2: “i; f _i \neq 0” should be “ i: f_i \neq 0”.
-
Section 2.2 begins with a description of the creation of a 3D volume for the 2D sCT layers. The information that the intervening layers are filled with 0s and that in fact all layers (and not only those for which 2D sCTs exist) are fed into the pipeline is given at the end. It would be helpful if this information was at the beginning of the paragraph.
-
Related work is a bit dated and not state of the art, e.g. p. 2 “Traditional Deformable Image Registration (DIR) methods like SyN [1], Demons [23,24]…” with reference to methods from the late 90s is probably a bit too traditional ;) .
-
The paper should be self-contained, definitions of losses and measures should be part of the paper and not in the appendix.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think this is a good paper and it should be accepted. I particularly liked the novel end-to-end approach of coupling the adversarial method with registration, showing that this offers significant advantages over separate sequential methods. However, the building blocks themselves are not new, and the novelty is limited to combining them in an end-to-end approach. In addition, some relevant aspects are not discussed in full detail, and the evaluation is limited to some extent, so the paper is more at the proof-of-concept level. It is difficult to foresee how generalizable the method will be. In summary, therefore, I believe that this is a fair paper that should be accepted.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper addresses the interesting problem of 2D/3D registration and has received positive remarks on average - with the exception of one reviewer that criticised the somewhat chaotic organisation and writing. In addition there seems to be limited reproducibility (neither code nor dataset released) and the runtimes are too high for the envisioned realtime application. I think the issues can be addressed during a moderate revision and by releasing source code. I hence recommend acceptance.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
N/A