List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Sylvain Thibeault, Stefan Parent, Samuel Kadoury
Abstract
Minimally invasive spine surgery such as anterior vertebral tethering (AVT), enables the treatment of spinal deformities while seeking to preserve lower back mobility. However the intra-operative positioning and posture of the spine affects surgical outcomes. Forecasting the standing shape from adolescent patients with growing spines remains challenging with many factors influencing corrective spine surgery, but can allow spine surgeons to better prepare and position the spine prior to surgery. We propose a novel intra-operative framework anticipating the standing posture of the spine immediately after surgery from patients with idiopathic scoliosis. The method is based on implicit neural representations, which uses a backbone network to train kernels based on neural splines and estimate network parameters from intra-pose data, by regressing the standing shape on-the-fly using a simple positive definite linear system. To accommodate with the variance in spine appearance, we use a Signed Distance Function for articulated structures (A-SDF) to capture the articulation vectors in a disentangled latent space, using distinct encoding vectors to represent both shape and articulation parameters. The network’s loss function incorporates a term regularizing outputs from a pre-trained population growth trajectory to ensure transformations are smooth with respect to the variations seen on first-erect exams. The model was trained on 735 3D spine models and tested on a separate set of 81 patients using pre- and intra-operative models used as inputs. The neural kernel field framework forecasted standing shape outcomes with a mean average error of 1.6 +/- 0.6mm in vertebral points, and generated shapes with IoU scores of 94.0 compared to follow-up models.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43996-4_8
SharedIt: https://rdcu.be/dnwOI
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a comprehensive & complex framework to intra-operative prediction of standing spine shapes to assess the outcome of minimally invasive spine surgery from patients with idiopathic scoliosis. For that purpose, the framework builds on latest state of the art algorithms to forecast the standing posture of the spine immediately after surgery. In total 735 3D spine models are used for training with 81 additional scans used for testing.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-> Challenging task. The given task, namely prediction of the spine shape, is certainly a challenge and special task which requires a lot of different components and experience. From that perspective, it becomes obvious that the paper builds on a lot of existing components and knowledge to address this particular task. However, these learning are for sure worth to share with the community.
-> Innovativeness. Given the challenge of the task, there is some creativity and translational effort needed to build the proposed framework. As such, the framework contains several interesting components that are put together to yield the task at hand. From that perspective, the paper makes a nice technical contribution to the field applied to a relevant task.
-> Data cohort. Availability to such sizeable cohort is certainly a plus, as such cohort for this application is probably rather rare (and again attributed to the fact that the research group has probably worked already many years on this topic.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-> Clarity. While the paper proposes a nicely designed complex framework, it makes the paper quite dense and packed from a content perspective. Consequently, there are several parts (and even key parts) that are hard to follow or were specific design choices are not well motivated or even unclear. As a result, this makes it hard to read and follow, reduces potential reproducibility and even triggers further key questions, especially on the evaluation (see next point)
-> Evaluation. To my mind the evaluation is not thorough enough and consequently raised more questions than answers. How were the landmarks annotated? Where were those set? Who set those? How were groundtruth models obtained (from the stereo reconstruction? How is the DICE obtained (given that stereo reconstruction and landmarks are available, but no image)? Other methods are given for comparison, but it remains unclear if self implemented or taken elsewhere and how applied (e.g., C-OccNet is also used in the overall final framework)? What does it mean to have it as comparison? This is very confusing and deserves further more explicit explainations.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Moderate reproducibility for various reasons. At first, the clinical task is less common and I assume not many groups have access to a cohort as in this paper. The cohort itself is not publicly available and consequently it will be difficult to reproduce and compare later. The level of detail given in the paper is not sufficient to be able to re-implement. Code will not be made available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- To my mind it would be very helpful if the paper aims to be much more explicit and precise in explaining the work carried out. As an example, in the introduction, it says ‘The proposed framework uses as input the intraoperative 3D model obtained with a multi-view Transformer network which integrates the pre-op model, and is based on neural field representations, capturing the articulation variations in a disentangled latent space intra-operative positioning in the prone position.’ This covers a lot of technical details while it would be much more helpful to clearly say what is clinical set up (multi-view X-rays) and what is the input to the framework (3D model obtained from such multi-views). While a reference to the Figure is made, several key ingredients are not clearly defined to my mind and very difficult to extract. What is the shape code and the shape sample? Without such definitions the figure is much less helpful (and the paper much more difficult to follow). Also the various color codes creating further unclarity as purpose is not clear when reading the paper at this stage. It would also help a lot if you could more explicitly highlight the components and create references to the sections. This would also help explaining the structure of the paper and guiding the reader through it.
Is the vertebral mesh at each level m always having fixed predefined topology? If so, what is it? Why was specific transformer chosen? Any idea on performance (accuracy, success rate)?
It would improve readability a lot if for section 2.2. and 2.3 you could state at the beginning more explicitly what input and purpose of the component is.
What is the runtime of the approach?
Between what (or how) is DICE evaluated? DICE is used as an overlap score for evaluation of segmentation while here stereo reconstruction and annotated landmarks are given. Consequently, it remains completely unclear how this measure is obtained.
Please provide more details on the comparison against other methods (how were the methods used/ how were they applied)? Maybe also consider using rather less methods but then provide more explanation and discussion.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
To my mind the paper deals with a challenging and interesting problem. A creative solution is proposed. This is probably of relevance for the community to learn about the application, the problem and the framework put together. I can imagine that this might trigger interesting further ideation within the community. For that reason, I tend to be more on the positive side although, the paper lacks clarity and thoroughness unfortunately.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
After thoroughly reading other reviewers comments, but also the authors rebuttal, I decided to not change the scoring. Other reviewers seem to judge identified items differently (maybe due to their paper stack), but did not reveal major additional item for me. Given the authors answer, I have confidence that lack items mentioned could be addressed properly. Furthermore, code is supposed to be made public.
Review #2
- Please describe the contribution of the paper
This paper provides a shape forecasting pipeline for use in spine surgery. The objective is to predict the patient’s spinal shape in the standing position while taking in a biplanar set of intraoperative X-rays and a preoperative 3D articulated model as input.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The provided network architecture for articulated SDF is very interesting and seems to be suitable for the application at hand. A relatively large dataset of spine models has been used for training and evaluation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Use of subjective tone in the introduction section. English needs correction and editing. Although already attempted in the introduction, the clinical objective for intraoperative estimation of lumbar positioning needs to be further elaborated with objective clinical criteria. The existing (non-NERF) algorithms for estimating the 3D articulated spinal shape have been referred to in a superficial fashion. Without proper critical analysis of the existing algorithms, the gap in the literature is not evident. The is a reference to the “kernel methods” in the introduction without proper elaboration on the details. Related to the target application of this manuscript, there is an extensive body of literature on 2D-3D registration of intraoperative X-rays to preoperative 3D models (this is precisely the input to the herein algorithm). The authors have failed to refer to the existing work in this realm therefore, have not provided a comparative analysis of the existing non-rigid and semi-rigid registration algorithms for the context of spine surgery. Pre and post operative 3D models used for training the herein algorithms are generated using the EOS system. How realistic is this?
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Given my comments above regarding the clarity of the methods section, I have concerns regarding the reproducibility of this work. Proper attention has to be devoted to explaining the flow of information within the pipeline.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The general structure of the presented framework is rather vague. The authors start by presenting figure 1 as the basis of their algorithm while the majority of their explanations are included in a single figure caption at the beginning of the methods section. This makes it very difficult for the reader to follow the presented pipeline and the associated subtleties.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper needs to be improved in terms of structure and clarity specially in regards to the methods section.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
The authors rebuttal has not helped in clarifying/addressing the original issues with this manuscript. The underlying clinical objective is still not communicated carefully. In comparison to the SOTA methods for 2D-3D registration, the authors have stated the primary novelty of their algorithm to be “predicting the standing post-op model from intraop X-rays, rather than the registration used as an initial step.” while this is a legitimate “technical” basis for the underlying application, such bold claims have not been backed up by sufficient “clinical” metrics within the context of the presented paper. Amongst many realistic factors that can influence the ability of the presented work to forecast a “Postop” spinal shape, only vertebra-level shape estimation has been explored while ignoring other equally important postop factors such as tool occlusion, patient specificity, inter-vertebral movement, imaging conditions, breathing effects etc. When looking at the publication at hand from a application-centric point of view, the same objective can be achieved through level-based registration which undermines the clinical contribution of the paper. Therefore, in it’s current form, the manuscript fails to highlight and address the gap in the field. Additionally, the methodology section still needs re-structuring and clarification.
Review #3
- Please describe the contribution of the paper
The authors propose intra-operative forecasting of a post-operative, anterior vertebral tether corrected, standing posture spines in patients treated for scoliosis. The method uses implicit neural representations, which harness a backbone network to train kernels based on neural splines. Kernels estimate network parameters from intra-pose data to regress the standing shape on-the-fly and accommodate spine appearance variations with a signed distance function. Losses incorporate a regularizer from a pre-trained population growth trajectory, ensuring that output transformations are smooth with respect to the variations seen in first-erect exams. The method was tested on new patients and yielded standing shape predictions a mean average error of 1.6±0.6mm for vertebral landmarks and IoU of 0.94 compared to post-op ground truths.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novelty: The methods included in the forecasting architecture are novel and consider a variety of anatomical configurations and populations. Clinical applicability: the content is highly applicable in a clinical setting and has been throughouly tested in such a setting. Organization: The submission is well written and clearly explained. Rigor: The study performs a number of ablation studies to ensure the kernel and regularization model portions of its architecture provide significant improvements to final standing predictions.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Figure 1 and 2 are disjointed. Better define their relationship to one another to clarify the methods. Some spelling and grammatical issues: Table 1 caption: biplabar “[6]. Each patient in the cohort underwent correct spine surgery” –> corrective.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The methods are highly detailed and replication of the forecasting architecture is made as reproducable as possible. Unfortunately, the dataset cannot be made public.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The results section comprised a small portion of the submission. If possible, include additional graphics or detail here and shorten some of the methods.
Ensure spelling and grammar mistakes are corrected throughout
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This seems like a very competitive submission for the overall conference and was the #1 reviewed paper in my stack.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
My opinion remains unchanged. I still find the paper to be good overall, with moderate weaknesses related to the results section and testing cohort. The authors are limited by page number and requests by other reviewers for a further discussion on methods takes precedence. I agree that the methods of the submission could be made clearer, but brevity for MICCAI manuscripts is required.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper presents an innovative intra-operative forecasting framework for standing spine shape, leveraging articulated neural kernel fields, in the context of scoliosis surgery. Two reviewers provided a positive review of this work. However, there are some important questions raised as well. Therefore the authors should be given a chance to address the major points before this work can be accepted. Specifically, about the weakness points raised by the reviewers summarized below.
- Lack of clarity and detail in methodology and evaluation sections, affecting reproducibility.
- Insufficient contextualization and critical analysis of the proposed solution in the literature.
- Issues with language usage and figures’ representation, detracting from overall readability.
Author Feedback
- Lack of clarity and detail in methodology and evaluation sections.
To improve clarity of the methodology, we specify the framework inputs are the 3D model obtained from multi-view X-rays + intra-op C-arm. The shape code is the latent representation of the spine model; a shape sample is a point taken of the mesh model. These are clarified Sec. 2.1. For evaluation, landmarks were obtained automatically from the generated pre-op 3D model, using the method in Humbert et al. 09. The Dice is based on the overlap between GT and predicted vertebral meshes. The comparative methods were publicly available implementations, all referenced in the paper. We should note the code of the proposed model will be published upon publication. The caption of Fig. 1 will be simplified and detailed explanations moved to the appropriate sub-sections (multiple shape instances->2.2, inference->end of 2.3, regularization->2.3). This will help reduce the density of the caption.
- Insufficient contextualization and critical analysis of the solution in the literature.
Prediction of the post-op 3D articulated shape has actually not been explored in the literature. Various clinical studies have measured correlations between clinical parameters and outcomes, but none produced standing full shapes based on the intra-op pose. And while generative methods have used recurrent networks or statistical models, this represents the first using implicit neural fields.
- Issues with language usage and figures’ representation, detracting from overall readability.
A thorough read-through will be made to correct language issues and syntax errors, while Fig. 1 will be improved by removing complex symbols and standardizing color codes, improving figure clarity. Finally, sub-section numbers will be added to each associated module to highlight all the components.
- Clinical objective for intraoperative estimation of lumbar positioning.
Lumbar positioning plays a key role in surgical results, impacting patients standing posture post-surgery. Therefore, by optimizing the intra-op posture, surgeons can help to reduce long-term complications such as lower back pain and improved spinal mobility. The objective clinical criteria is to predict the actual post-op shape within Cobb angles under 5deg.
- “kernel methods” in the introduction without proper elaboration on the details.
Kernel methods (or basis functions) allow to map input data into a different space, where subsequently simpler models can be trained on the new feature space, instead of the original space. We attempt to exploit this with neural fields by learning the mapping between prone and standing postures. We now clarify this in the intro.
- Reference to existing work and comparative analysis to existing non-rigid / semi-rigid registration algorithms for spine surgery.
We agree with R2 that there are several existing works for spine registration; we should emphasize however the primary objective of this work is predicting the standing post-op model from intraop X-rays, rather than the registration used as an initial step. Due to space constraints, comparison of the X-ray registration was not included. We will add the following in the intro: “Several non-rigid and articulated registration methods based on deep learning were proposed for aligning pre-op to intra-op 2D X-rays (Esfandiari et al., 2019; Unberath et al., 2021; Zhao et al. 2023).” Furthermore, the alignment process proposed here yielded a target registration accuracy of 0.9mm, which is a clinical acceptable compared to similar works in the field. Future work will perform a detailed comparative analysis.
- Using EOS system and how realistic is this?
EOS is used in > 400 clinics worldwide, seen as the standard for scoliosis care for pre-op planning and post-op assessment with low-dose X-rays. It should also be noted the model is not dependant to the EOS system, as other 3D reconstruction methods can be used to train and test the model.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper tackles an important and challenging problem in medical imaging - predicting post-operative spine shape in patients undergoing scoliosis treatment. The authors have sufficiently addressed most of the concerns raised by the reviewers in their rebuttal. If the revisions promised are implemented, the paper will be a significant contribution to the field. Taking into account the authors’ responses and the originality and potential impact of the work, I recommend that the paper be accepted.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This manuscript is very close to borderline, however the positive reviews out weight the negative reviews and most of the negative reviews are based on writing, citing of previous work, and not having an expansive discussion section.
I agree with R3 in that while there are weaknesses in the writing (and in the rebuttal related to how to address some of these points) the need for brevity makes it difficult to have a detailed discussion of weaknesses and prior art. A complex manuscript should not be rejected from MICCAI only because there is not enough space to cite all the relevant literature.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
In reviewing the primary meta-reviewer’s comments as well as those of R1-3, the authors in their rebuttal commit to addressing the major revisable points called to attention in the meta-review. The main comments on initial review were around clarity and further description, which seem to be reasonably within bounds of a clarification through rebuttal.
Reviewer 2 provided a review score that was discordant from R1 and 3 that seemed to be primarily driven by disagreement with regards to clinical objective. While the reveiwer’s post rebuttal comments are noted in that R2 felt other elements that contribute to postoperative shape/function of the spine were not appropriately accounted for, my read on the manuscript is that their method is an initial attempt to apply neural kernel field forecasting to a context in which even surgeons do not have the luxury of additional metrics such as intervertebral movement, breathing effects, etc. As such, even when considering clinical objective, the work seems reasonably clinically justified (though of course needing additional work for translation to application). Therefore, I am recommending accept given the other strengths of the paper as already noted in the initial reviews.