List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Bailiang Jian, Mohammad Farid Azampour, Francesca De Benetti, Johannes Oberreuter, Christina Bukas, Alexandra S. Gersing, Sarah C. Foreman, Anna-Sophia Dietrich, Jon Rischewski, Jan S. Kirschke, Nassir Navab, Thomas Wendler
Abstract
CT and MRI are two of the most informative modalities in spinal diagnostics and treatment planning. CT is useful when analysing bony structures, while MRI gives information about the soft tissue. Thus, fusing the information of both modalities can be very beneficial. Registration is the first step for this fusion. While the soft tissues around the vertebra are deformable, each vertebral body is constrained to move rigidly. We propose a weakly-supervised deep learning framework that preserves the rigidity and the volume of each vertebra while maximizing the accuracy of the registration. To achieve this goal, we introduce anatomy-aware losses for training the network. We specifically design these losses to depend only on the CT label maps since automatic vertebra segmentation in CT gives more accurate results contrary to MRI. We evaluate our method on an in-house dataset of 167 patients. Our results show that adding the anatomy-aware losses increases the plausibility of the inferred transformation while keeping the accuracy untouched.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_22
SharedIt: https://rdcu.be/cVRS2
Link to the code repository
https://github.com/BailiangJ/spine-ct-mr-registration
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper introduces a weakly supervised anatomy aware method for registration of CT & MRI images of the spine.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well written and organized. The results support the main hypothesis of the paper.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
What is the likelihood this method will extend to other areas of the body, or general CT vs. MRI registration?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
the work appears reproducible (clear description of the methods and also a robust evaluation strategy), although the primary data set is unlikely to be accessible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
What is the likelihood this method will extend to other areas of the body, or general CT vs. MRI registration?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
the proposed method and loss functions appear to outperform previous methods.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Not Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
My assessment remains the same after rebuttal.
Review #3
- Please describe the contribution of the paper
The paper proposed a weakly-supervised deep learning framework that preserves the rigidity and the volume of each vertebra while maximizing the accuracy of the registration. Authors introduced anatomy-aware losses depended only on the CT vertebra segmentation for training the network.Results show that adding the anatomy-aware losses increases the plausibility of the inferred transformation while keeping the accuracy untouched.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1, Authors introduced a new anatomy-aware losses only depended on the CT vertebra segmentation to improve the CT-MRI deformable registration. The idea is interesting to let the warped label be similar to its regid transform. Besides, using only one sgemtation map to calculates loss function is innovative since that segment vertebra from MRI is not as easy as segment that from CT. 2, Another loss that force the deformation fields to be regid in bone region is also a good try.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1, The notations used in this text are confusing. Many characters have no definition. 2, Experiment results are not promising. Fig. 2 and Table.1 do not show significant improvements with compared methods, and it is difficult to find the impacts of different functions from the results. 4, Writtings and layouts should be improved. For example, Table 1 was mentioned before Fig.2 and Fig.3, but it appears in last pages.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It is difficult to reproduce without open source codes.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1, The regidly panelties L_{rigid} includes four different variants: rigid dice loss, rigid field loss, properness condition and orthonormal condition. What’s the basis of selection for your different experiments? 2, The rigid transformations seemed to be calculated every iteration during the training which is inefficient. I prefer to get information about how you solve this. 3, You only showed the visualizations of the warped labels. However, the visualizations of the warped images and the DDFs are much more importent.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The idea is interesting but need more experiments.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The paper presents a deep learning-driven deformable registration method using vertebra contour-based DVF rigidity loss, to preserve the local rigidity of the DVF while allowing global deformation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The introduction of the novel rigidity loss terms.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Although the introduced rigid dice loss and rigid field loss perform better than the baseline method w/o such losses, the relative advantage is small. The ablative studies with properness condition and orthonormal condition also show varying results, with different loss terms having their own benefits. It makes it hard to draw a conclusion from such a work.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
NA
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper will benefit from a more in-depth analysis of what loss term might be the most appropriate for the vertebrae registration. And although the title includes ‘biomechanical’, the actual implementation does not include much biomechanical component by only assuming the rigidity of the vertebrae structures. The work will be more exciting if some true biomechanical modeling can be introduced.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
A good work with comprehensive evaluation. But it can benefit from more methodology development and in-depth analysis.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
In this paper, the authors propose a method for same subject registration of spine CT to MR images accounting that each vertebrae is a rigid structure. It is assumed that each vertebrae can be robustly segmented used and automatic 3rparty method on CT images only. A diffeomorphic voxel-morph is then trained to registrer the CT and MR images with penalization losses to ensure rigidity of the transform.
Two new rigidity losses are presented, and two classical rigidity losses and also incorporated in the framework.
Experiments are run on a in house dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The paper is clear and easy to follow.
-
The proposed method is faster than the optimization based counterpart while having similar quality.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The innovation is mild. The method is a rewritting with a deep network of standard Although 2 new rigidity losses are proposed (L_{rigid field} and L_{rigid dice}), it not clear that these losses have any benefit compared to the classical L_oc loss.
-
I do see why there is 4 different rigidity losses. It only makes the computation heavier.
See details below.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Most criterions in the reproducibility checklist are satisfied. The code will be made available but the xperiment have been done on an in house dataset that will not be released.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
How are the parameter for the comparison method (Starring) chosen (mostly bspline node spacing). Was there a hyperparameter optimization on the validation dataset ?
-
Several rigidity losses are used. How were the weight for these losses chosen (table 4) ? It seems only the global lambda_smooth was optimized and not the weight for each loss.
-
L_pc and L_{rigid dice} are weaker losses than L_oc and L_{rifid field}. They do not enforce rigidity: L_pc only enforce volume preservation and L_{rigid dice} allows non rigid deformation inside the vertebrae as long as the edge are rigid.
It would be reasonnable to think that good results regarding both dice and rigidity can be obtained with only one of the strong loss (L_oc or L_{rigid field}). A careful optimization of individual rigidity weight can make results in table 1 similar to each other. -
As L_{rigid dice} only enforce a weak rigidity property, it would be more relevant to used L_oc instead of L_{rigid dice} as a rigidity metric in the results.
-
The properness condition in Starring et al [26] only enforce det(Jc)~=1. It is not clear what the authors means by “we compute the ideal rotation matrix R of every voxel x ∈ ΩF from the Jacobian of the DDF phy.” What is R ? it is not used in the remaining of the paragraph.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The innovation is mild. It is not clear that the 2 new losses have any benefit compared to the classical L_oc loss.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors were reassuring in their answer about two of my concerns:
- OC will be included as a metric in the CRV
- loss coefficients were optimized each loss separately. (This should be added in the finale version of the manuscript).
The authors were not clear about the node spacing parameter of the Staring method used as baseline. This parameter is probably more important than the losses coefficients. It should be made explicit that this parameter was also optimized.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
In general a good work. I would ask for further clarity on the advantage of the demonstrated improvement, while addressing the last two reviewers’ comments.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
nr
Author Feedback
We thank the reviewers (R1,R2,R3,R4,MR1) for their constructive feedback and acknowledgment of our work “Weakly-supervised Biomechanically-constrained CT/MRI Registration of the Spine” (SpineReg) ’s several strengths: (a) the fact that we only use the segmentation map from one modality to train our registration network with “anatomy-aware losses” (R3,R4), (b) our comprehensive evaluation (R1,R2), and (c) the clarity and organization of the manuscript (R1,R2,R3,R4).
Moreover, the reviewers recognized our contributions to the MICCAI community by proposing innovative losses (R3) and porting a conventional method to deep learning (DL) (R4). By doing so, we show that the CT/MRI registration of the spine can be done efficiently while improving the performance of the conventional methods (R3,R4).
We present evidence to select different loss functions based on the clinical application by a detailed evaluation of their performance. As correctly pointed out by R2, different losses have different benefits for different clinical applications. Extending the discussion in this regard in the camera-ready version (CRV) will also satisfy the request of R2, R3, and R4 for a better interpretation of the results:
- L_pc (PC) is most helpful in bone cement estimation in vertebroplasty or kyphoplasty, in which the volume of vertebrae in the warped CT should be preserved.
- When the CT and MRI need to be flawlessly fused, i.e., the image shapes should match perfectly, like in surgical planning, the best losses are L_oc (OC) or L_rigid dice (RD).
- If the smoothness/feasibility of the transformation plays the most important role, e.g., in differential diagnostics, the L_rigid field (RF) is the best choice.
As R4 notes, OC is a better way to grasp rigidity than RD since it focuses on the shape of the vertebrae. However, the clinical application plays the most relevant role when selecting the losses. We will include OC as a metric in the CRV. In respect to R4’s comment regarding rigidity enforcement in PC, we would like to clarify that since PC minimizes the expansion/contractions inside the vertebrae, it also dictates rigidity. Furthermore, RD decreased PC in our experiments, which means it indeed enforced rigid deformation inside the vertebrae. We will add this to the discussion in the CRV.
To set the parameters of the conventional baseline (R4), we started with the values provided by Starring et al. [26] and fine-tuned them according to the validation results. For fine-tuning, we used the same approach as the DL counterpart, in which we optimized the loss coefficients in a logarithmic space for each loss separately. We adjusted the weights slightly to get comparable performance (R4) when using the combined losses. Our goal was to evaluate the impact of the proposed losses on the overall process rather than find the optimal parameter set or provide a significantly superior method to the conventional baseline (R2,R3).
R4 notes that the computation of RD and RF could be optimized. We improved this for RF during our experiments by computing the average rigid transformation of sampled corresponding points within the vertebrae using SVD.
We believe our method can be extended to any bone (R1), but we did not validate this with data. Moreover, the method could be more anatomy-aware (R2) by considering surrounding anatomies (e.g. intervertebral discs, bands); nevertheless, this substantially increases the training cost.
Based on the reviewers’ input, we will improve the document and make the contribution of SpinReg clearer for the community in the CRV. In particular, we will gladly include images of the DDF and the warped images (R3) in the CRV. Also, we will revise the notation to be more clear and more coherent (R3,R4).
Overall, we agree with the reviewers that the contribution of SpinReg is sound and valuable to the MICCAI community. Further, the quality of the manuscript will improve after taking into account their comments.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The work seems sound and reasonably presented. However, the moderately diverging reviews also indicate that this may be short of exciting ideas or convincingly generalisable results.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Overall, I believe the paper is a solid contribution to MICCAI, the method is sound and nicely addresses the challenges of multi-modal (intra-patient) MR-CT spine registration. The rebuttal alleviated a few concerns of the reviewers and the evaluation can be considered relatively good. The novelty is somewhat limited, but in my opinion sufficient for a conference paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Weakly-supervised Biomechanically-constrained CT/MRI Registration of the Spine
This submission tackles the multimodal registration of joint rigid and non rigid tissues (bones and soft tissue). The novelty resides in proposing new loss terms to preserves rigidity and volume of vertebrae. Experiments are on a private dataset.
Three reviews were thorough, one is unfortunately too light to be fully weighted. The thorough reviews were mostly focusing on technical clarifications but all indirectly indicate a positive value of the proposed approach within the defined context of the registration of spine images. The rebuttal did add further clarification without fuss. R3 has a valid point on the use of “biomechanically-constrained” in the title, which is a bit far stretched as no actual biomechanical modeling is used in the method. This, however, does not change the value of the current paper.
For the above reasons, recommendation is towards Acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7