List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Guillaume Mestdagh, Stéphane Cotin
Abstract
The nonrigid alignment between a pre-operative biomechanical model and an intra-operative observation is a critical step to track the motion of a soft organ in augmented surgery. While many elastic registration procedures introduce artificial forces into the direct physical model to drive the registration, we propose in this paper a method to reconstruct the surface loading that actually generated the observed deformation. The registration problem is formulated as an optimal control problem where the unknown is the surface force distribution that applies on the organ and the resulting deformation is computed using an hyperelastic model. Advantages of this approach include a greater control over the set of admissible force distributions, in particular the opportunity to choose where forces should apply, thus promoting physically-consistent displacement fields. The optimization problem is solved using a standard adjoint method. We present registration results with experimental phantom data showing that our procedure is competitive in terms of accuracy. In an example of application, we estimate the forces applied by a surgery tool on the organ. Such an estimation is relevant in the context of robotic surgery systems, where robotic arms usually do not allow force measurements, and providing force feedback remains a challenge.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_8
SharedIt: https://rdcu.be/cVRUO
Link to the code repository
https://github.com/gmestdagh/adjoint-elastic-registration
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The manuscript describes a framework for non-rigid image registration and estimation of surface forces that generate the deformations observed in the images. The framework formulates image registration and surface force estimation as an optimal control problem. It utilises the hyperelastic material model to describe tissue constitutive behaviour. The Authors envisage application in force estimation for robotic surgery when direct force measurement (and haptic feedback) are typically not available.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed method attempts to compute physically (biomechanically) plausible deformation field and surface force distribution.
The error measures used (Euclidean distance between the actual positions of selected points and the positions predicted through registration) has a clear geometric interpretation and appears to be clinically relevant (accurate information about target position is crucial in surgery)
The manuscript structure is very clear.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The role of the tetrahedral mesh in the proposed algorithm is unclear. It appears that the computation is done on the continuum/organ (liver in the example analysed in the study) discretiseed using a point cloud rather than the tetrahedral mesh.
Synthetic tests cases for liver: it is unclear how the deformation field resulting from the forces applied to the liver surface is computed. How it was ensured that the computed/predicted deformation field is plausible (i.e. physically correct)?
The reported computational speed/performance (update time of under 2 s using off-the-shelf PC with i7 processor) is impressive, but compatible only with real-time constrains of image-guided surgery. For haptic feedback (estimate foces for haptic is one of the motivations for the study, an update frequency of around 500 Hz would be required).
The registration errors and errors in estimating the surface forces are reported. However, no attempt is made to interpret these errors in the context of the accuracy and robustnessn that would be required for clinical applications.
The propose approach requires patient-specific information about the tissue material properties (Young’s modulus). Elastography is mentioned as a possible way of determining such properties. However, in the context of determining patient-specific properties of soft tissues still remains a subject of active research. It’s accuracy and robustness are hotly debated topic and are from being commonly accepted as amenable to clinical applications.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The relavent form has been filled out by the Authors. Appears to be adequate.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Well written manuscript on important topic of image registration and estimation of forces that causes intraoperative organ deformation. The manuscript would benefit from: (a) More detailed/clear explanation of the methods used (including the role of tetrahedral mesh);
(b) Interpretation of the reported accuracy, robustness, and computational efficiency (computation time) in the context of time constrains of clinical workflows;
(c) Clear statement of how patient-specific material properties of soft tissues required for the proposed framewrok can be reliably obtained using methods (and equipment) available in clinics/clinical settings (or offer perspective of the use in clicnical setting in a reasonable future).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The study addresses clinically relevant problem and the proposed method appears, in general, to be scientifically sound. However, before the manuscript can be accepted for publication, more detailed description of the methods used needs to be provided (see the comments in sections 5 and 8 of this review)) and specific description of how patient-specific material properties required by the proposed method can be reliably obtained in a clinical environment. It appears that this can be achieved by careful revision the manuscript without any need for obtaining additional results.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
The authors propose a method to estimate surface forces needed to load a biomechanical model of the liver, so that the deformed model will fit observed data. An optimization problem is formulated along with an adjoint scheme to resolve it, with surface loads as the optimization variable. This registration approach is evaluated on experimental phantom data from the Sparse Data Challenge. While the evaluation it not complete, the proposed approach currently ranks second in the challenge leaderboard. An additional contribution of the method is that since the applied forces are estimated, the magnitude of these forces could be used as haptic feedback in a robotic system.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- the problematic is well stated
- the optimization problem and adjoint solving are sound and clearly formulated
- introducing “admissible controls” is very relevant, to restrict the location and magnitude of possible load forces
- the force estimation study is clearly a plus; this part may actually be more significant than the registration result itself
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- your results are only compared to a rigid solution and are thus obviously better. Being ranked second on the dataset/challenge website is clearly positive, but without comparison with other non-rigid baseline methods it is not possible to really assess the added benefit of the method
- you should at least compare your method with the previous solutions proposed in your group as in [17, 18]. (yes, references and the challenge dashboard are enough to easily break anonymity; fair enough).
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The optimization problem and ajoint problem are well explained and could thus be reproduced in another simulation context.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
“The box plot shows that significant improvement is achieved by the elastic registration step with respect to the rigid registration result.” -> Well yes, that was expected! It would be more relevant to compare your method with other non-rigid registration methods. Are statistics for other teams available on the website of your dataset/challenge?
-
“As these datasets were used to calibrate the method, results are better for these sets than in average.” -> this is clear bias, but thanks for noticing. Obviously you should avoid estimating errors on already seen datasets, although my understanding here is that no other data is available. -> what do you mean by “calibrate” here, did you tune some parameters with this datasets? Which ones, the admissible forces properties?
- rheological parameters vary significantly with the experiment: (E=1Pa, nu=.4) and (E=20kPa, nu=.45) in experience 1 and 2, respectively. Could you explain this huge difference, especially for the stiffness?
- why limit yourself to a linear elastic model if you can simulate a hyperlastic law? Computation time is not a factor in this study, and the “small deformations” threshold is never clearly known/real.
- “avoid the inverse crime”? Strange sentence in this context, although we can get the point.
- proof read before the last version (“registration registration” in the abstract, “onyl”, …)
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is clear and interesting, and the problem of finding the loads leading to a solution absolutely relevant. The major limiting factor is that the proposed method is not directly compared to other simulations or other loading approaches.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
This paper introduces a non-rigid liver registration method between pre-operative and Intra-operative biomechanical models. The surface of the liver is registered and compared in rigid and non-rigid registration. The TRE approach is utilized for currency measuring.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is appropriately written. The structure of the article is properly placed.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1-No novelty 2- why the ICP is used , what about CPD approach (Coherent Point Drift)? 3- The landmark selection of TRE selection is not described well , However im not sure if TRE is the best measuring tool for your data.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
noting special for this paper for reproducibility
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
1- Dice or Jaccard indices could be included for measuring 2- Jacobian Determinant tool is a powerful tool for measuring registration accuracy. The JD of borders value could describe the expansion and shrink of border points. 3- Hausdorff distance is another more suitable tool for accuracy measurement
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
2
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This method is very simple with no novelty. I suggest to utilize deep learning approaches
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper is about registration of a 3D model to intraoperative images. The key idea is to recover the forces b that generate the observed images. This may be a good idea. R1 and R3 are positive and R4 negative on the paper. Uncited previous work should be considered in the state of the art, eg, Ozgur et al, IJCARS 2018, and references therein. The AC would like to know how is the set of eligible forces (the force prior) B specified. This set is a crucial piece of information to solve the problem and is, in practice, not available a priori, if one considers that the liver is being handled intraoperatively and its boundary conditions are for most hidden (eg, the posterior ligaments). Please use the rebuttal to clearly specify how can B be found practically, including all the required boundary conditions and possible force application places. In addition, discuss the expected results when B is an uninformative prior (ie, forces may be applied anywhere on the model).
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Author Feedback
— Description and novelty of the method
First, we provide details to answer R1’s questions about the method. The organ is represented by a tetrahedral mesh while a point cloud represents intra-operative data (the observed surface), which is noisy and partial. In synthetic test cases as well as during registration, deformation fields are computed from forces by solving an elastic problem using the Finite element method. In both cases, this ensures physically correct deformation fields. The optimization procedure aims to find forces such that the resulting mesh deformation matches the observed data/point cloud.
The novelty of our approach (questioned by R4) is in the use of the optimal control formalism, which allows us to estimate the actual external forces that led to the observed deformed surface. Elastic registration methods based on ICP or CPD create an attractive force between the point cloud and the matched region of the virtual organ. Such ghost forces are unrealistic, as they do not reflect the true cause of deformation. Very often, the observed deformation results from forces applied on other parts of the organ. In our approach, forces are chosen among a set of eligible forces B, that defines where forces can be applied. The set B may be initialized pre-operatively and then updated using intra-operative images.
— Choice of B
Following the AC’s suggestion, we give details about the choice of admissible forces in practice. In the robotic surgery scenario, we assume that the tip of the instrument appears in the camera field of view. An approximate contact zone, segmented from laparoscopic images, is used to define B. Such a strong prior cannot always be determined. In a general registration scenario, B may be more loosely defined. If no contact occurs in the camera field of view, then the visible surface is labeled as a free surface, while forces are allowed by default on the remaining hidden surface. Finally, when B is uninformative, computed forces tend to be regularly distributed over the surface, due to the regularization provided by the adjoint problem.
— Numerical results
To answer R1 and R3’s questions on material parameters:
- We acknowledge (R1) that elastography is not part of the clinical routine.
- However, our registration method does not require a parametrization (Young modulus) of the elastic model. The Young modulus can be seen as a multiplicative parameter in the elastic energy. Therefore, it does no play a role during the registration process; it only plays a role when we try to estimate forces. In the latter case, as mentioned in the paper, the estimation accuracy depends on the knowledge of material parameters.
- For this reason, we chose E=1Pa in experiment 1 and a more realistic value (E=20kPa) in experiment 2.
The challenge used in experiment 1 was designed to prevent “cheating”. For this reason, only 4 out of 112 datasets contain ground truth data (for only 20% of landmarks). Those 4 datasets were used to tune the Poisson ratio and to notice that no penalization was necessary in this experiment. They were not used to define the set of admissible forces. The overall result presented in Table 1 is the outcome of our method applied to the 112 datasets (of which 108 contain no ground truth information).
To answer R4’s comment on error measures, TRE was chosen by the challenge organizers, as well as the landmarks.
As an interpretation of the registration results in a clinical setting (R1), note that our clinical collaborators and literature usually mention an acceptable registration error of 5mm.
As mentioned by R3, comparing our results to rigid results is not the best quality indicator. In the final version, we will compare our results with other teams from the challenge dashboard instead. Available statistics are composed of a table like Table 1 for each participant.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal makes a clear response. R4 overlooked the paper’s contribution. The paper has merit and the AC recommends acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper is well written and addresses an interesting problem. I believe the authors have done a good job addressing the concerns of the reviewers and would recommend accepting the paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Good and interesting work. The main limitation is the lack of relistic or sizable validation data set, thus the value of the work is very much questionable.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR