List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Lanzhuju Mei, Yu Fang, Zhiming Cui, Ke Deng, Nizhuan Wang, Xuming He, Yiqiang Zhan, Xiang Zhou, Maurizio Tonetti, Dinggang Shen
Abstract
Accurate periodontal disease classification from panoramic X-ray images is of great significance for efficient clinical diagnosis and treatment. It has been a challenging task due to the subtle evidence in radiography. Recent methods attempt to estimate bone loss on these images to classify periodontal diseases, relying on the radiographic manual annotations to supervise segmentation or keypoint detection.
However, these radiographic annotations are inconsistent with clinical golden standard of probing measurements, and thus can lead to measurement errors and unstable classifications. In this paper, we propose a novel hybrid classification framework, HC-Net, for accurate periodontal disease classification from X-ray images, which consists of three components, i.e., tooth-level classification, patient-level classification, and a learnable adaptive noisy-OR gate. Specifically, in the tooth-level classification, we first introduce instance segmentation to capture each tooth, and then classify the periodontal disease in the tooth level. As for the patient-level, we exploit a multi-task strategy to jointly learn patient-level classification and classification activation map (CAM) that reflects confidence of local lesion areas upon the panoramic X-ray image. Eventually, the adaptive noisy-OR gate obtains a hybrid classification by integrating predictions from both levels.
Extensive experiments on the dataset collected from real-world clinics demonstrate that our proposed HC-Net achieves state-of-the-art performance in periodontal disease classification and shows great application potential.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43987-2_6
SharedIt: https://rdcu.be/dnwJo
Link to the code repository
https://github.com/ShanghaiTech-IMPACT/Periodental_Disease
Link to the dataset(s)
N/A
Reviews
Review #3
- Please describe the contribution of the paper
This paper proposes a mixed-method based periodontitis diagnosis model. Image features are extracted at both tooth-level and patient-level, and then fused to obtain the diagnostic structure of the disease. T
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main advantage of the thesis is that it combines the tooth-level and patient-level dental features, and achieves high evaluation results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper does not report the segmentation results of the instances in the paper, and incorrect instance segmentation may affect the feature extraction at tooth-level, which is not discussed in the paper. In addition, the diagnosis of periodontitis is not only related to teeth, but also to the relationship between teeth and periodontal tissues. Moreover, periodontal disease is not an acute disease but a slowly developing disease. However, the paper does not report the classification basis used to evaluate the experimental results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper does not report detailed dataset label categories. It also does not report specific steps for the instance segmentation part of the experiment, which may indeed make it difficult to reproduce the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Periodontal disease should not only consider teeth, but more importantly, the relationship between teeth and periodontal tissues. In clinical practice, we often diagnose periodontitis based on the degree of periodontal tissue recession, that is, the relationship between teeth and periodontal tissues, rather than teeth themselves.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Methodologically, this paper has some innovation. The paper does not report the impact of instance segmentation on the overall experiment. Some very important prior knowledge for the diagnosis of periodontal disease, namely, that periodontal disease is diagnosed based on the relationship between teeth and periodontal tissues, is ignored.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #5
- Please describe the contribution of the paper
The authors propose a novel hybrid classification framework to achieve accurate periodontal disease classification from X-ray images. Specifically, the authors present three modules for these tasks: 1) tooth-level Classification, 2) patient-level classification and 3) learnable adaptive noisy-or gate. The proposed methods is validated on the dataset from real-world clinics and the results demonstrate the effectiveness.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1) This paper is well-organized and easily comprehensible.
2) This paper’s method effectively combines two tasks: patient-level and tooth-level.
3) The experiments demonstrate the effectiveness of the proposed model.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1) The main contribution of this paper is not clear. In the section of introduction, the authors didn’t highlight the challenge of the task and didn’t articulate why their approach is effective.
2) The method section does not reflect the insight behind the proposed methods.
3) The experimental section is not comprehensive and contains some redundancies. For example, Figure 3 duplicates the results presented earlier.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors did not release the code and the used dataset is an in-house dataset.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
1) The major issue of this manuscript is that the method section of this article does not effectively articulate the insight behind the proposed methods. For instance, while the focus of this article is on the Learnable Adaptive Noisy-OR Gate, the author only introduces the implementation process in the method section without effectively explaining why such a method is proposed.
2) The authors said that their proposed method is only supervised by the clinical golden standard. However, it seems that the proposed methods not only need the patient-level ground-truth, but also need tooth-level classification labels and segmentation mask. Please clarify it.
3) The section of introduction should be revised. It does not clearly articulate where the difficulty of the problem lies and why the proposed method can address this difficulty.
4) Why use Euclidean Distance Transform instead of the CE loss or Dice loss to supervise the CAM? Please clarify it.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The contribution and experiment setting of this paper.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
The author described the reasons behind the lack of further experiments and promised to release the code and supplementary experiments. However, based on my assessment of the methodology and experimental sections of the paper, I still believe that the overall quality of the paper is slightly lower than the standard expected for this conference.
Review #6
- Please describe the contribution of the paper
This paper proposed a hybrid method for binary classification on periodontal disease. The authors propose to learn features on both tooth-level and patient-level for more comprehensive evaluation. Compared with naïve methods that simply perform a patient-level learning, this hybrid method learns more like the real-life clinical diagnosis process, and the experimental results also prove its effectiveness.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This work is well motivated and well-written. It may have some impact on the periodontal disease diagnosis.
- This work to some extent imitates the diagnosis process in real-life clinics, which makes its interpretability become more acceptable for doctors.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The novelty seems limited to me. The first tooth-level branch mainly predicts each tooth’s result by running a classification model on each tooth patch, while the second patient-level branch mainly predicts each tooth’s result by running an FCN on the entire image to have the final CAM (appending an MLP after the encoder for more discriminative features is also no-new thing nowadays). Both branches use ordinary pipelines.
- The main weakness lies in the experiments. The compared methods are basic models which are out-of-date, i.e., ResNet, DenseNet, a multi-task network proposed for mammogram in 2020, and TC-Net without a reference. Moreover, the dataset is small – only 426 images are involved, which makes the results less convincing. To make the result more convincing, statistical analysis of the results, e.g., DeLong test of the ROCs, is recommended.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The proposed method is mainly based on existing techniques and should be easy to reproduce. The authors also will release the code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- The task is interesting, and the methodology also looks fine to me as it to some extent imitates the doctors’ diagnosis process in real-life.
- However, the experiments are very limited and the reproducibility is poor. There is only one private dataset used in the experiments, and only four other methods are compared. It would be much better if the authors could report their results on at least two datasets, and maybe also involve the use of public datasets. Furthermore, some more recent works should also be considered, as currently there are only methods before 2020 are involved in the comparison.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the novelty of the proposed method is limited, from my point of view, the problems addressed in this paper is of clinical interest and could potentially benefit the related field.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a hybrid classification scheme to classify periodontal disease at a tooth level and a patient-level in dental panoramic X-ray images. Their proposed hybrid classification approach is interesting, which combines tooth-level and patient-level features to accurately classify the target disease. Three experts reviewed your paper. One of the three had concerns about the main contribution(s), lack of descriptions about the insights of the proposed method, and lack of comprehensive experiments.
Author Feedback
We thank all reviewers for positive and thoughtful comments to help us improve the paper’s quality. The main comments are answered below.
Q1(Meta-R&R2): The contribution of this paper A: Clinical impact: Periodontal disease affects approximately 10.8% of the global population. Clinically, manual probing on each tooth is regarded as gold standard for periodontal disease diagnosis. This paper introduces the first approach that uses these manual probing results as the benchmark for our diagnostic framework, distinguishing our work from previous learning-based methodologies, which primarily relied on labels annotated on panoramic X-ray images, thereby lacking accuracy in detecting early periodontal disease. Notably, we dedicated nearly two years to meticulously collecting panoramic X-ray images along with manual probing results for 426 patients, showing our commitment to capturing the clinically validated accuracy of our framework. The insight of our methodology: Our methodology reflects the unique challenges of diagnosing periodontal disease from panoramic X-ray images, which primarily manifests as gum loss around individual teeth and is consolidated into a patient-level diagnosis. Following the clinical process, a straightforward approach would involve creating a tooth- and patient-level classification branch, followed by fusion. However, early-stage periodontal disease, which often occurs in localized gum areas, presents a significant challenge to identify within the context of an entire X-ray image. To address this, we designed a patient-level classification method to compute dummy probabilities from a patient-level activation map. Subsequently, a learnable adaptive Noise-OR Gates was utilized to produce consistent classification results from both branches. In summary, our paper contributes significantly by providing a comprehensive system for the automated diagnosis of periodontal disease. We have invested significant time and effort to ensure clinical relevancy by collecting clinical probing results as our ground truth. Furthermore, our unique hybrid branch design, encompassing tooth- and patient-level classifications, facilitated by an adaptive Noise-OR Gate, allows us to effectively learn consistent and discriminative features necessary for accurate periodontal disease diagnosis.
Q2(R1): The instance segmentation accuracy A: We agree that reporting this metric will improve the completeness of our results. We employed the well-studied CenterNet for tooth instance segmentation, achieving promising detection (mAP50 of 93%) and segmentation (DICE of 91%) accuracy. We will include the tooth instance segmentation accuracy in our final version.
Q3(R1): This paper only focuses on the teeth objects, ignoring the relationship between teeth and surrounding tissues A: The reviewer is correct that dentists usually diagnose periodontal disease based on the relationship between teeth and tissues. Indeed, our framework is designed with the same understanding. Our tooth- and patient-level branch inputs encompass teeth and periodontal tissue information. The emphasis on teeth objects aims to draw the model’s attention to relevant tooth areas, moving away from irrelevant background.
Q4(Meta-R&R3): Lack of comprehensive experiments, including the dataset and competing methods A: We acknowledge the need for comprehensive experiments. It is important to highlight the absence of a public dataset comprising paired panoramic X-ray images and clinical manual probing results for periodontal disease diagnosis. The in-house dataset with 426 patients took a great effort to collect, using nearly two years. For comparative analysis, we select four typical methods and surpass them by a considerable margin. As suggested, we will include 1-2 additional leading methods for comparison, and conduct the DeLong test on our ROC curve to validate our results further.
Other details and the code will be provided in the final version.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors provided a good rebuttal, and one of the reviewers increased the score. As a result, the final score became among the ones on the higher-side in my pool.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
R#5 still raises pivotal concerns on the study after the rebuttal.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
new segmentation and detection results are done with center net, will be added to manuscript. Most comments are addressed, I believe that there is enough clinical and technical (little but yes) contribution in the paper.