Authors

Yankun Lang, Xiaoyang Chen, Hannah H. Deng, Tianshu Kuang, Joshua C. Barber, Jaime Gateno, Pew-Thian Yap, James J. Xia

Abstract

Dental landmark localization is an essential step for analyzing dental models in orthodontic treatment planning and orthognathic surgery. Typically, more than 60 landmarks need to be manually digitized on a 3D dental surface model. However, most existing landmark localization methods are unable to perform reliably especially for partially edentulous patients with missing landmarks. In this work, we propose a deep learning framework, DentalPointNet, to automatically locate 68 landmarks on high-resolution dental surface models. Landmark area proposals are first predicted by a curvature-constrained region proposal network. Each proposal is then refined for landmark localization using a bounding box refinement network. Evaluation using 77 real-patient high-resolution dental surface models indicates that our approach achieves an average localization error of 0.24 mm, a false positive rate of 1 % and a false negative rate of 2 % on subjects both with or without partial edentulous, significantly outperforming relevant start-of-the-art methods.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_43

SharedIt: https://rdcu.be/cVRsb

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper presents an end-to-end deep learning-based method for the localization of dental landmarks. The method follows a coarse-to-fine strategy that generates proposals with RPN and refines them with DLLNet. The presented method is an extension of DLLNet that first finds the candidate landmarks with RPN, then refines them with DLLNet. The method is evaluated on a dataset containing 77 patients and outperforms the state-of-the-art methods.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The coarse-to-fine strategy presented in the paper where landmark point detection is handled as an object detection problem with RPN is quite interesting. The method is end-to-end which allows making better detection.

The experiments including variant-1 and variant 2 are useful for showing the effectiveness of the proposed balanced focal loss and curvature-constraint.

I think the results of Table 1 and Table 2 should be given with the whole dataset to show the overall performance of both normal and partially edentulous patients. Also, the number of missing teeth in the dataset should be given.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The novelty of the paper is limited. Although there is an RPN stage and a new loss function, the contribution of improvements of the DLLNet is not clear.

In the paper, DLLNet is criticized for not performing well on meshes with missing teeth and being sensitive to the segmentation errors. More experiments are needed for this claim. Is the RPN only needed to eliminate edentolous patients or has it more advantages.

There is also reproducibility concerns about the work.

The method is claimed to work well on patients with partial edentulous. However, the experiments are limited because there are only 15 patients with partial edentulous.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The deep learning model is not directly given and will not be published after acceptance. The input size of the data and some details are missing for reproducibility.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

A discussion on the improvement of DLLNet to DentalPointNet can be given. Also, demographic information about the patients (age, sex, etc.) and the ratio of missing teeth should be given. The robustness of the method should be discussed because the FP in Table 1 is greater than in Table 2.

The term “DentalPointNet” should be given in the abstract.

The results before/after data augmentation can be useful to show the effectiveness of data augmentation.

Is curvature constraint (>0.65 threshold) held for all landmarks?

In Table 1, bold parts are inaccurate for FP and FN.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The novelty of the paper is limited, however, it addresses partially edentulous patients which is not studied before. It is an improvement on previous work and the improvements are not sufficient for a strong acceptance.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

2
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper proposes a coarse-to-fine framework to automatically localize landmarks on dental surface models. In the coarse stage, it addresses the issue of foreground/background imbalance problem by a balanced focal loss and uses the curvature as a threshold for filter predictions. In the fine stage, a DLLNet is trained to further improve the results. The proposed method achieves promising performance.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1. The paper is well-organized.
2. The improvement seems to be promising.
3. The proposed is application-oriented and could be helpful for clinical use.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Major weakness:
1. Limited novelty. In section 3.2, the authors indicate the main differences are balanced focal loss and the curvature constraints to prior work. However, the hyper-parameter in focal loss is not given, i.e., alpha and gamma for both the proposed method and PointRCNN. The proper tuning of these hyper-parameters for the use case of this paper may improve the reported baselines, although I believe the proposed method could still outperform the improved baselines. Besides, I also suggest giving a complete form of focal loss in the paper. Second, it is unclear how the curvatures are computed and how the threshold (0.65) is defined. Without curvatures (variant-1), the proposed method degenerates to a similar performance as DLLNet in table 1 but is better for edentulous patients. Please consider clarifying this point.
2. Missing results on only coarse stage. The proposed method is further refined by training using DLLNet after coarse stage training. What is the performance of only the coarse stage? Is it similar to PointRCNN? (To me, the coarse stage is PointRCNN + curvature. Correct me if I am wrong.)
The training time seems to be doubled or more (PointRCNN training can be time-consuming) than only training DLLNet. Will DLLNet get improved by longer training?

Minor issues:
1. It should be “Adam” optimizer, right? “ADMA optimizer” appears twice.
2. At the beginning of page 5, the shapes of feature matrices should be modified to be consistent. For example, F^4_s should be R^{1x256}, instead of 256x1. It confused me at the first glance.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

No special issue here.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

See weakness.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

5
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

See strength.
Number of papers in your stack

7
What is the ranking of this paper in your review stack?

3
Reviewer confidence

Somewhat Confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

This papers presents a deep learning framework to automatically localize landmarks on dental surface models. The main contributions on the paper are the two sub-networks (Region Proposal Network and Bounding Box Refinement Network) proposed to precisely localize 3D landmarks on high-resolution 3D digital dental models. Other relevant points are the comparison of the proposed methods with other published methods using the same dataset and the possibility of applying the method to edentulous patients.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Introduction of a new loss function to train the region proposal network.
- Results of the proposed method were compared with two other methods using 5-fold cross validation.
- The assessment of method on missing tooth/teeth was conducted using separated anatomical regions.
- Results have shown significant improvement compared to the other compared methods.
- The number of figures and tables are appropriate; the figures are well illustrative.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The proposed method uses parts (same parameter values and sentences) from the DLLNet method published in the MICCAI-2021.
- The choice of some parameter values used in the paper should be detailed.
- Tables 1 and 2: Apparently the FP and FP rates are computed for all five assessed regions - it would be more appropriate to present these numbers for each region.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Based on the information provided in the methodology section, I believe the proposed method could be implemented. However, since the data is not publicly available - it is not possible to reproduce the results presented in the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
** Abstract
- please remove the RPN abbreviation from the abstract since it has not been used further.
** Introduction
- page 3 - first sentence: “…, which significantly reduces false negatives on …”; This sentence should be modified - first, “significantly reduces” is too vague and second, this reduction is compared with what?
** Methods
- There are a few parameters that were set without explanation - selected 128 proposals, thresholds (0.65, 0.85), lambdas.
- Landmark annotations from Fig.3 should be included in Figs. 4 and 5.
- page 6 - first line of section 3.2 - DLLNet reference is number 10, not 9.
** Results
- Table 1 - PointConv and DLLNet have shown 0% of FP and FN? Is this correct?
** Conclusions
- Instead of “significantly outperforms competing state-of-the-art methods” please provide numbers or percentages to illustrate the performance.
** References
- Do not use “et al.” in the reference list. All authors should be included.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The manuscript is well written; the methodology is well presented and provides all required details to allow the implementation of the method. The number of Figures and Tables are appropriate. The contribution of the proposed method is clearly communicated and the results have shown significant improvement compared to the other methods.
Number of papers in your stack

1
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper presents deep learning-based method for the localization of dental landmarks. It first finds the candidate landmarks with RPN, then refines them with DLLNet. The validation is small but reasonable. Mehodological novelty is limited,
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

2

Author Feedback

N/A

back to top

DentalPointNet: Landmark Localization on High-Resolution 3D Digital Dental Models