List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Florian Kordon, Andreas Maier, Benedict Swartman, Maxim Privalov, Jan S. El Barbari, Holger Kunze
Abstract
Careful surgical planning facilitates the precise and safe placement of implants and grafts in reconstructive orthopedics. Current attempts to (semi-)automatic planning separate the extraction of relevant anatomical structures on X-ray images and perform the actual positioning step using geometric post-processing. Such separation requires optimization of a proxy objective different from the actual planning target, limiting generalization to complex image impressions and the positioning accuracy that can be achieved. We address this problem by translating the geometric steps to a continuously differentiable function, enabling end-to-end gradient flow. Combining this companion objective function with the original proxy formulation improves target positioning directly while preserving the geometric relation of the underlying anatomical structures. We name this concept Deep Geometric Supervision. The developed method is evaluated for graft fixation site identification in medial patellofemoral ligament (MPFL) reconstruction surgery on (1) 221 diagnostic and (2) 89 intra-operative knee radiographs. Using the companion objective reduces the median Euclidean Distance error for MPFL insertion site localization from (1) 2.29 mm to 1.58 mm and (2) 8.70 px to 3.44 px, respectively. Furthermore, we empirically show that our method improves spatial generalization for strongly truncated anatomy.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_59
SharedIt: https://rdcu.be/cVRXx
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #2
- Please describe the contribution of the paper
The authurs proposed an automatic landmark detection framework to identify an anatomic landmark point inside the distal femoral head called “the Schoettle Point”. The accurate calculation of this point is of great importance for patellofemoral ligament (MPFL) reconstruction surgery. In the common practice, first relevant anatomical structures, for example, the tangent line to the posterior femur shaft cortex, are detected by optimimising a proxy objective function. Next, the Schoettle Point is calculated based on geometrical properties of the detected structures. This paper proposed a framework called ‘Deep Geometric Supervision’ to merge these two separate steps into one.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed formulation is novel and improved the point detection accuracy measured as median error from 2.29 mm [95% CI=1.84 to 2.82] to 1.58 mm [95% CI =1.15 to 2.09].
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The manuscript is a bit difficult to read for me. I appreciate the effort by authors to explain the framework as generic as possible but this generalisation make the paper difficult to read. Please identify the cost functions in Eq. 1 as it is based T=3. What are MSE and BCE on page 4? what is spatial-to-numerical transform (DSNT) on page 5? Please define them mathematically in separate equations.
-
Figure 1. Please provide explanation in the caption explaining notations. Figures should be understandable on their own without searching in the text.
-
Figure 2, why the frequency of p>=2.5 and p<2.5 does not sum to 100%? Please also explain the vase shape in panel A.
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Some mathematical details on objective loss functions are missing. Otherwise, it should be reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
authours may wish to consider automatic landamark detection using deep convolutional neural networks for this task.
Liu, Wei, et al. “Landmarks detection with anatomical constraints for total hip arthroplasty preoperative measurements.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2020.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is an intersting paper with novel formulation and satisfactory results. However, the method description should be explained better and so I rated the paper as ‘accept’.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors develop a novel method to accurately identify the graft fixation site for MPFL surgery, from both diagnostic and intraop X-rays. Their method combines anatomical landmark extraction together with a model that learns the geometric relationship between landmarks to infer the fixation site.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Appropriate statistical analysis
- Novel method seems to improve results significantly
- Validated on two different datasets
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Lack of appropriate discussion of the results and experiment choices
- Excessive use of math notation makes the paper convoluted and difficult to read in my opinion.
- Some of the methodology could be better explained
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Good details on experiment parameters, but would be better if code was made available to examine.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Section 1 - RQ1, why the choice to examine activation maps as a research question? This seems unrelated to the planning or surgical goals and does not seem relevant. Seems like this is for supplementary material instead a core research question
- General - What is the accuracy of directly segmenting the graft fixation site using a FCN like U-net compared to using the proposed method which uses multiple learning models (Proxy + DGS). If this is already published, cite in discussion section for comparison
- Section 2 - What value does task 3 (semantic segmentation of femur) add to the overall accuracy? Is segmenting the femur necessary for reasonable planning accuracy?
- Section 2.3 - What is meant by the “original segmentation task”? Is this directly segmenting the graft fixation point or is it referring to segmenting the femur as in task 3 of the meta learning?
- Section 2.4 - What exactly did the medical engineer label in the X-rays? The surgeon marked the SP point, did the engineer mark the Blumensaat point and turning points or was this automatically calculated from the femur polygon?
- Section 3 Fig 3 - Why is there a small increase in error for Shaft/Head ratios greater than 2.4?
- Figure 4 - The legend on figure 4b seems to be wrong as from this plot it seems configuration A is the best.
- Section 4 - What is the accuracy when only DGS is minimized? While this might be less interpretable, I would be interested in the accuracy difference as well.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Well written (although sometimes a little convoluted). Novel methodology with strong evidence of improving planning accuracy.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This study jointly optimized anatomical feature extractor and the planning target in an effort to automate the surgical planning for reconstructive orthopedics.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novel method: This is an extension from presumably the authors’ previous work where the planning target was calculated from the optimization of so-called proxy objective without optimizing planning target itself. In the current study, the optimization for planning target and proxy objective are jointed performed, achieving significantly improved planning target accuracy. The proxy object was also improved to accommodate joint optimization.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I fully understand it is not easy to put everything into this paper due to the page limit, but the paper suffers from a lack of explanation. The organization could be improved to help readers better understand the problem to solve and developed methodology. All the papers cited to explain the limitation of the existing methods are from the author’s group. Citing other studies would help better assess the outcome this study achieved.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility can be rated as sufficient
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Among three model variants, the ‘proxy’ model is an improvement of (or equivalent to) authors’ previous work [13]. I’m wondering how much performance improvement was achieved by hard parameter-sharing proposed in this study. It would be great if the authors could more kindly explain the rationale for model variants ‘B’ Proxy + DGS w/o segmentation. Why did the authors’ expect segmentation task may have chance to negatively impact the overall performance? Also, the current way of writing and naming is confusing because model variants ‘A’ Proxy model, inherently include ‘segmentation’ as one of three tasks. In my opinion, “B) Proxy w/o segmentation + DGS and c) Proxy + DGS makes more sense. Is the multiplicative weighting term (lambda) in 2.2 the same as the risk-weighting (lambda) in 2.4? Keeping consistency in terminology would avoid any unnecessary misunderstanding. How was the initial (0.99) and decrease internal (0.01) of the risk-weighting determined? Have the authors performed any sensitivity test for it? The results are impressive. What would be the next step to achieve full automation? How far the authors think is left to adopt the developed tool in real clinical setting?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This study is an important step towards the automized planning, which is robust to various clinical environments. The writing could be improved for better context and clarity.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This manuscript present a novel method to plan Medial Patella Femoral Ligament (MPFL) surgery. It uses a joint landmark extraction and geometric relationship (proxy object functions) to detect important bony features and plan target reconstruction.
All reviewers found this method to be novel, demonstrated to have improved performance over the state of the art on two datasets.
The main weakness was that many reviewers found the paper difficult to follow and lack sufficient explanation of the problem to solve, the method to have overly complex mathematical notation and some details on the object loss functions were missing.
I recommend acceptance as all reviewers found the strength of the paper outweigh the weakness and found the methodology novel and practically useful.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Author Feedback
Dear chair,
We are very thankful for the positive assessment and the reviewer’s agreement on the paper’s strengths, novelty, and practical usefulness. We want to address the main concerns summarized by the MR and discuss how to incorporate the feedback into our manuscript.
As expressed by all reviewers, the article is difficult to read and sometimes too convoluted. We attribute this problem to two aspects: The detailed and generic mathematical description, which partly interrupts the reading flow, and the difficulty of describing all aspects of the methodology in sufficient detail within the prescribed number of pages. As mentioned by R1, we wanted to explain the framework’s underlying methods as generic as possible and provide a sufficient mathematical foundation to apply our methodology to other surgical applications with greatly different geometries. We agree that this makes it slightly more difficult to understand the specific example of MPFL surgery planning. At the same time, especially since we cannot share the code due to conflicts w.r.t. intellectual property, we believe that this level of explanation greatly benefits reproducibility and practical adoption to various related problems. With this objective in mind and considering space constraints, we clarified and streamlined the method’s explanation by drawing more lines to the MPFL example, providing equations for the utilized cost functions (MR, R1), and adding a reference to [1] where the MPFL planning geometry and underlying framework are explained in great detail. Moreover, following the comments of R3 w.r.t. inconsistent terminology, we have aligned all ambiguous descriptors and have refrained from using mathematical expressions wherever they are not conducive to understanding and tend to impede the flow of the text. Furthermore, we followed the thoughtful suggestions by R2 and R3 regarding crucial details on the risk-weighting parameter choice (lambda), a more intuitive naming of the model variants, the phrasing of research question RQ1, and explaining the influence of the segmentation task.
In addition to these overarching aspects, we want to address minor questions and doubts raised by the reviewers. R1 inquired about a more thorough description and inaccuracies in the figure legends and captions, also relating to errors in Fig. 4 Panel (b) spotted by R2. We gladly revise these aspects in the camera-ready version. As pointed out by R2, the planning error increases for shaft/head ratios above 2.4, which is currently not discussed in the paper. Most bones suffer from varying degrees of antecurvation/recurvation, mostly caused by prolonged weight-bearing, destabilization of the bone‘s structure due to old age, or injuries. Such effects are more pronounced in the image the longer the visible aspects of the bone, making it increasingly difficult to fit a meaningful tangent to the bone shaft cortex. Relating to the question of R2, this problem could be avoided by directly estimating the graft fixation site by direct segmentation or optimizing only DGS. However, this comes at the cost of uninterpretable results, which likely fail to appeal to the surgeon. While we agree that such analysis would be very interesting, we have decided not to include such experiments in this paper but leave them for follow-up work. R3 raised concerns about the motivation for the DGS model w/o segmentation. Although segmentation is not strictly necessary for the DGS model, it allows easy registration of the planning result on subsequent live images, where the registration is limited to the bone mask. At the same time, additional tasks in an MTL setting also bring with them the risk of dominating potentially conflicting tasks during joint optimization. Without further adjusting the objective function by heuristic/dynamic task weighting or gradient manipulation, this can significantly influence the accuracy of the planning.
[1] https://dx.doi.org/10.3390/jimaging8040108