List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Xi Fang, Daeseung Kim, Xuanang Xu, Tianshu Kuang, Hannah H. Deng, Joshua C. Barber, Nathan Lampen, Jaime Gateno, Michael A.K. Liebschner, James J. Xia, Pingkun Yan
Abstract
Simulating facial appearance change following bony movement is a critical step in orthognathic surgical planning for patients with jaw deformities. Conventional biomechanics-based methods such as the finite-element method (FEM) are labor intensive and computationally inefficient. Deep learning-based approaches can be promising alternatives due to their high computational efficiency and strong modeling capability. However, the existing deep learning-based method ignores the physical correspondence between facial soft tissue and bony segments and thus is significantly less accurate compared to FEM. In this work, we propose an Attentive Correspondence assisted Movement Transformation network (ACMT-Net) to estimate the facial appearance by transforming the bony movement to facial soft tissue through a point-to-point attentive correspondence matrix. Experimental results on patients with jaw deformity show that our proposed method can achieve comparable facial change prediction accuracy compared with the state-of-the-art FEM-based approach with significantly improved computational efficiency.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_54
SharedIt: https://rdcu.be/cVRXs
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper describes a method for predicting facial appearance after osteotomy. The method uses the pre-operative bone and soft tissue surfaces of the skull and face, as well as the planned revised bony surfaces as input to a pointnet based network.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper solves a new clinical problem using a pointnet, Allowing for quicker and easier computation than labour intensive FEA and without relying on surgical expertise.
- The authors compile a dataset of pre and post op imaging of facial reconstruction patients that is used to train and evaluate the model
- The authors present a novel architecture that uses PointNet derviced features, point to point correspondence with convolutional layers to learn displacements to apply to pre-operative geometries
- Limited data, but still good performance
- The article is well written
- The methods presented use a different formulation, relying on points that is less researched than voxel based methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The methods could be better explained with more detail. o Figure 1 needs more labelling and explanation in the caption o 2.2 convulational layers are used but it is not clear how many filters are used
- The context for the results is missing. How large can errors be for this task? What is the resolution of the imaging being used?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Could be reproduced. The dataset and code are not open, neither seem to be referenced. The methods are described however some details are lacking and this is noted in other sections of the review.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Introduction
- The introductory paragraph about the use of FEA should be revised. The standard of care seems to be primarily physician opinion. FEA could be used. Is FEA used clinically with widespread deployment for this indication? That is not my understanding. The references do not refer to specific uses of modelling for CMF surgery. Has modelling been specifically used for CMF surgery and shown to be successful? If so, the authors should reference it. If not then the paragraph should be largely revised.
- Most of the second last paragraph can be put in the methods. This paragraph is repetitive and not clear until having already read the methods.
- Figure 1 – more detail is needed and more description in the caption. There are numerous rectangles that are completely unlabeled. Other than a pointnet++ feature extractor it is completely uclear what the other elements are. Dataset
- Include the spatial resolution of the imaging used. Comment on how this relates to the errors observed.
- How accurate were the registrations? Implimentation
- What boundary conditions are put on the FEA. How difficult are these to apply? Results
- The nose predictions appear to be the best. Is this because there is little change in the nose after osteotomy? How much displacement is there between the faces before and after surgery?
- Ablation study o The finding on that CPSA was more effective than closest point is overstated. The differences are small and the model has more parameters, these parameters may establish point correspondence or may do some other error correction
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- good paper, a useful goal, novel implimentation
- some issues related to clarity, details and discussion of the findings
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
A deep Learning-based method was presented to predict the facial appearance following bony movement in this paper. The PointNet++ network was adopted to extract the point-wise features of facial and bony model. And, a novel cross point-set attention module was proposed to explicitly calculate the correspondence matrix between each bony-facial point pair. The calculated matrix was then used to predict the postoperative facial change. The proposed method was evaluated on 40 sets of CT data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The whole manuscript was well prepared and clearly written, the contributions in both technical and clinical perspective were well demonstrated. The proposed method took both bony and facial surface points to be considered, it is more convincing for real clinical application. The experimental results showed good achievements and efficiency of the proposed method compared with the SOTA FEM-based method.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
One concern is that, as demonstrated in experiment part, no significant difference was found of the quantitative experimental results on six different regions. It probably because the different deformity may involve different regions, the author may further refine the task and the dataset in future work to further improve this point and apply the proposed method to real clinical application.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The demonstration of the method part is clear. The reproducibility is good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The facial deformities are complicated, the author could focus one or two kinds of main deformity to conduct and evaluate their method, as this work is really useful in clinical area. But the high accuracy is required.
- The qualitative experiments were performed on upper/lower lips, this may be the key attention of the clinicians?
- On page 5, “…. deep learning-based SkullEngine segmentation tool, SkullEngine segmentation tool [6],…”, repeated “SkullEngine segmentation tool”.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The manuscript was well prepared and written, the motivation and contribution are clear. The constructed dataset is reasonable. The demonstration of the method is good. And the author presented the good results and high efficiency of the proposed method. This work may have a potential application in orthognathic surgery.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper describes Attentive Correspondence assisted Movement Transformation network (ACMT-Net), a method to predict post-operative facial appearance after orthognathic surgery leveraging on movements of corresponding bony segments. This is achieved by exploiting point-to-point correspondences between facial and bony points, which are estimated thanks to a novel module called Cross Point set Attention (CPSA). The proposed method is evaluated on models extracted from real CTs and it achieves comparable performance to related works in terms of accuracy, while outperforming their computational performance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper proposes an innovative approach to address the problem of transferring correspondences between shapes and/or point sets, which is a well-known and open problem not only for the specific application considered in this paper. I believe this work can have a quite broad impact on all those applications that require correspondence mapping between different point sets.
In addition to the novelty of the methodology, I congratulate the authors for the clarity and the thoroughness of the paper. The methodology is clearly described with plenty of details. Figures are well-done, self-explanatory and provide complementary and essential information to the text. The conducted evaluation is complete and convincing. It assesses the performance of the method in comparison with alternative approaches both quantitatively and qualitatively, includes statistical analysis and an ablation study. Obtained results are thoroughly discussed. Limitations of the method are mentioned, as well as possible future directions.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The only main weakness that I find in the paper deals with related works. First of all, references to related works should be double-checked. For example, authors cite PointNet papers [9,10] to support the sentence “Deep learning-based approaches have been recently proposed to automate and accelerate the surgical simulation”, which is not consistent. Moreover, I would suggest authors to include some references to other works which had to cope with the problem of identifying correspondences between shapes/point clouds in surgery (such as https://link.springer.com/chapter/10.1007/978-3-030-87202-1_36 or https://link.springer.com/chapter/10.1007/978-3-030-59719-1_70). This would allow to strengthen the paper by stressing the potential impact of the proposed methodology outside the specific field considered.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper provides many details on their implementation, hence there is no major reason to argue about paper reproducibility. Authors report the different hyperparameters used (both in the data processing phase and network training and evaluation phase). Details on the dimension of each modules of the architecture are also provided. Although authors declared that they shared dataset and code in the reproducibility checklist, I do not see any link to such resources in the paper. I would expect that authors share their implementations upon acceptance.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper is strong and well-written, thus there are no major issues (apart from the ones mentioned above dealing with the related works). Some possible improvements/suggestions are listed below.
The computation time required by ACMT-Net to complete a simulation significantly outperforms its FEM-based competitor. It would be interesting to report which is the part of the proposed methodology taking most of the computation time.
Authors should consider adding further details about the experienced surgeons who took part in the qualitative evaluation. How many of them?
Typos:
- Faical -> facial
- “Skull engine segmentation tool” is repeated twice in 3.1
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
8
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper proposes a novel and interesting methodology to address a well-known and open issue in surgery. The paper is clear, well-organized and detailed. The conducted evaluation is satisfactory and thorough. Obtained results are strong and convincing.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This work presents a novel method to predict facial appearance after osteotomy using an adaptation of PointNet++ with a cross point-set attention module. All reviewers agree this is an interesting, well-written paper with appropriate validation of the method presented. Minor weaknesses were raised involving descriptions of previous works in the introduction and clarifying some figures in the methods section. Reproducibility is possible as all hyperparameters and details of the implementation were described in sufficient detail.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Author Feedback
We highly appreciate the reviewers and AC for the very positive opinions and constructive comments on our work. We made minor revisions to be included in the final version to clarify some details such as the imaging resolution and number of surgeons. We added more labels to Figure 1 and illustration to captions of the figures. We also corrected the typos. We would like to clarify a few minor concerns from reviewers as follows.
Regarding the context of the results asked by Reviewer #1, it is a consensus among clinicians that average error less than 2 mm is clinically acceptable for facial prediction [R-1]. According to this criterion, most of the prediction results in Table 1 are clinically acceptable. However, we previously found that the quantitative analysis does not necessarily reflect the clinician’s qualitative analysis. Therefore, we further performed qualitative analysis as reported in Section 3.3, in addition to the quantitative analysis, to prove the clinical significance of the study.
Thanks Reviewer #1 for pointing out the problem of the references on the use of FEA. We revised the article to clarify this point as follows. Finite-element method (FEM) is currently acknowledged as the most physically relevant and accurate method for facial change prediction. However, despite the efforts to accelerate FEM [1, 2], FEM is still time-consuming and labor-intensive because it requires heavy computation and manual mesh modeling to achieve clinically acceptable accuracy, preventing FEM simulation from being adopted in daily clinical setting [3]. In addition, surgical planning for orthognathic surgery often requires multiple revisions to achieve ideal surgical outcomes, which further emphasizes the necessity of an accelerated simulation method for clinical use.
Reviewer #2 had a concern that no significant difference was found in the quantitative experimental results on six different regions. We want to clarify the interpretation of the results. No significant difference among six different regions using ANOVA suggests that all the prediction methods performed similarly regardless of the deformity of each region. In other words, there are no particular regions that experienced inferior prediction performance. However, the method wise, there is still significant difference that our proposed method significantly outperformed others.
Reviewer #2 suggested that we could focus one or two kinds of main deformity to conduct and evaluate our method, as this work is really useful in clinical area. We thank the reviewer for acknowledging that our work is clinically very useful! Indeed, we have tried deformity area based evaluation by computing the distance error on the 100 farthest pre-facial points to post-facial points. Compared with the baseline model ‘No correspondence’, the two models with spatial correspondence (‘Closest point’ and ‘CPSA’) lowered the error from 5.90mm to 4.39mm and 3.76mm, respectively. The finding is in line with the reviewer’s suggestion. Our team plans to analyze the deformity area in more detail by following a systematical approach in the future work.
We thank Reviewer #3 for pointing us to the additional references, which look relevant to our work. We included them into our final version in addition to the above discussion to improve the clarity of our paper.
[R-1] Olivetti, E. C., Nicotera, S., Marcolin, F., Vezzetti, E., Sotong, J., Zavattero, E., & Ramieri, G. (2019). 3D soft-tissue prediction methodologies for orthognathic surgery—a literature review. Applied Sciences, 9(21), 4550.