List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Tudor Dascalu, Bulat Ibragimov
Abstract
Identifying and detecting a set of objects that conform to a structured pattern, but may also have misaligned, missing, or duplicated elements is a difficult task. Dental structures serve as a real-world example of such objects, with high variability in their shape, alignment, and number across different individuals. This study introduces an assignment theory-based approach for recognizing objects based on their positional inter-dependencies. We developed a distance-based anatomical model of teeth consisting of pair-wise displacement vectors and relative positional scores. The dental model was transformed into a cost function for a bipartite graph using a convolutional neural network (CNN). The graph connected candidate tooth labels to the correct tooth labels. We re-framed the problem of determining the optimal tooth labels for a set of candidate labels into the problem of assigning jobs to workers. This approach established a theoretical connection between our task and the field of assignment theory. To optimize the learning process for specific output requirements, we incorporated a loss term based on assignment theory into the objective function. We used the Hungarian method to assign greater importance to the costs returned on the optimal assignment path. The database used in this study consisted of 1200 dental meshes, which included separate upper and lower jaw meshes, collected from 600 patients. The testing set was generated by an indirect segmentation pipeline based on the 3D U-net architecture. To evaluate the ability of the proposed approach to handle anatomical anomalies, we introduced artificial tooth swaps, missing and double teeth. The identification accuracies of the candidate labels were 0.887 for the upper jaw and 0.888 for the lower jaw. The optimal labels predicted by our method improved the identification accuracies to 0.991 for the upper jaw and 0.992 for the lower jaw.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43898-1_29
SharedIt: https://rdcu.be/dnwA1
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
Authors propose assignment theory-augmented neural network for teeth recognition. A dental anatomical model is constructed and Hungarian method is used for the assignment, which are used to improve the recognition results of Unet. Authors validated the method on a public teeth dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Authors construct a distance-based teeth anatomical model, which is help for increasing relationship between teeth.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The abbreviation should be defined, such as FDI, PDF. What is the defination of the dental abnormality? what are the 16 tooth types? Matrix A is 3D, why its element is 2D as a_ij? The comparative experiment is insufficient, only Unet is compared. In the introduction, authors mentioned that “However, previous studies have not effectively addressed the issue of tooth labelling under the presence of dental abnormalities”, but in the experment, the accuracy result of Unet is 0.972. Confusion matrix of eath tooth is needed.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Sounds good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The definitions of varibles and concepts should be clearly, the setting of network should be provided indetail. More comparative experiments are needed to illustrate the performance of proposed method. The manusript is hard to read, Writing has significant grammar issues.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The definitions of varibles and concepts should be clearly, the setting of network should be provided indetail. More comparative experiments are needed to illustrate the performance of proposed method. The manusript is hard to read, Writing has significant grammar issues.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
Assignment-theory-augmented neural network for multi-object recognition This study introduces an assignment theory-based approach for recognizing objects based on their positional inter-dependencies. It developed a distance-based anatomical model of teeth consisting of pair-wise displacement vectors and relative positional scores. The dental model was transformed into a cost function for a bipartite graph using a convolutional neural network (CNN).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Theorethical and concept of the research were duly explained in the Method Section which starts from generation of the label, dental model, and the theory behind the DentAssignNet.
- Experiment done was detailed in the correct section and does not overload on the explaination.
- Result were aslo explained in a correct and comprehensive manner, precise and correct.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- In the Dental anatomical model section, there is a mathematical formulae embedded in the writing of which author should list it out with reference number as on the next page.
- Author may want to list out on the hardware or software used to run the simulation or experiment.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Author filled out the reproducibility checklist but does not include codes.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Paper has a definite and comprehensive structure i.e there is a balance towards theory and practical. 2.The approach for recognizing objects based on their positional inter-dependencies using neural network does produce strong result and therefore considered a good method to be followed.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The approach using assignment theory with convolutional neural network is a method worth trying as the result >90% was quite promising.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
Author feedback : “ To the best of our knowledge, our solution is the first to simultaneously address three challenges: 1) detection of objects with spatial dependencies; where 2) some objects can be missing; and 3) some objects may be duplicated. However, performance comparison on the dental labeling application should be included. The present study utilizes the dataset introduced in the 3D Teeth Scan Segmentation and Labeling Challenge at MICCAI 2022 [1].” This statement justify the issue raised by the reviewers.
Review #3
- Please describe the contribution of the paper
1.This paper introduced an assignment theory-based approach for recognizing objects based on their positional inter-dependencies.
2.The authors developed a distance-based dental model of the jaw anatomy. The model was transformed into a cost function for a bipartite graph using a convolutional neural network.
2.To compute the optimal labelling path in the graph, the authors introduced a novel loss term based on assignment theory into the objective function.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The topic is interesting and the paper is well-written.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The title of the paper might be overcliaming. The author said that they proposed an assignment theory-augmented neural network for multi-object recognition. But they only evaluated the method with one multi-object recognition task (Dental instance classification). Also, the experiments were weak and they can not show the superiority and the innovation of the proposed method.
-
There are no comparisons with many other deep learning based multi-object recognition methods.
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- The author should modify the title to avoid overclaiming that the proposed method ccan solve many multi-object recognition task.
- The abstract section is long. The authors can directly show the clinical significance of the dental instance classification.
- More experiments should be performed to show the effectiveness of the proposed method.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The title might be overclaiming, and the experimental study can not prove that the proprosed method is for multi-object recognition task.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
I keep my decision.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
However, some abbreviations are not defined, several notations and terms are not explained, details of the network architecture are not provided, making it difficult to comprehend the work. The manuscript is hard to read and has significant grammar issues. Furthermore, the paper’s title may be overclaiming, as the authors only evaluate their method with one multi-object recognition task (Dental instance classification). The experiments are found insufficient, failing to demonstrate the superiority and innovation of the proposed method. There are also no comparisons with many other deep learning-based multi-object recognition methods.
Author Feedback
We appreciate the insightful feedback provided by the reviewers. It is encouraging to see that the reviewers acknowledge the potential of our proposed solution, which leverages positional inter-dependencies for multi-object recognition (R1, R2).
Comparative experiments: The reviewers (R1, R3, Meta-R) highlighted the need for additional comparative experiments to demonstrate the advantages of our proposed solution. To the best of our knowledge, our solution is the first to simultaneously address three challenges: 1) detection of objects with spatial dependencies; where 2) some objects can be missing; and 3) some objects may be duplicated. However, performance comparison on the dental labeling application should be included. The present study utilizes the dataset introduced in the 3D Teeth Scan Segmentation and Labeling Challenge at MICCAI 2022 [1]. The challenge evaluated the algorithms based on teeth detection, labeling and segmentation metrics, on a private dataset that only the challenge organizers could access. The CGIP team (Hoyeon Lim et al.) adapted the Point Group method with a Point Transformer backbone, and achieved a labeling accuracy of 0.910. The FiboSeg team (Mathieu Leclercq et al.) used a modified 2D Residual U-Net and achieved a labeling accuracy of 0.922. The IGIP team (Shaojie Zhuang et al.) utilized PointNet++ with cast patch segmentation and achieved a labeling accuracy of 0.924. Notably, we achieved superior identification accuracy, with scores of 0.992 and 0.991 for the lower and upper jaw, respectively. This comparison was not included in the original submission as the challenge organizers published the summary paper after the submission deadline [2]. In the revised manuscript, we will incorporate these comparisons and the reference to the challenge results.
Title scope (R3, Meta-R): We understand the reviewers’ concerns about potential overclaiming. The title was meant to convey the fact that our assignment-theory augmented neural network could be applied on various datasets with positional inter-dependencies, such as facial structure recognition or astronomical datasets. However, given that the primary empirical evaluation in our paper is centered on dental instance classification, we propose the following revised title with a narrower scope: “Assignment theory-augmented neural network for dental arch labeling”.
Manuscript structure and clarity (R1, Meta-R): While R2 and R3 commented positively on the overall structure and clarity of the manuscript, R1 and Meta-R noticed some typos and inconsistencies. As suggested, the abbreviations and remaining issues will be fixed in the revised version.
[1] Ben-Hamadou, A., Smaoui, O., Chaabouni-Chouayakh, H., Rekik, A., Pujades, S., Boyer, E., Strippoli, J., Thollot, A., Setbon, H., Trosset, C., & Ladroit, E. (Oct 2022). Teeth3DS: A benchmark for teeth segmentation and labeling from intra-oral 3D scans. arXiv:2210.06094 [2] Ben-Hamadou, A., Smaoui, O., Rekik, A., Pujades, S., Boyer, E., Lim, H., Kim, M., Lee, M., Chung, M., Shin, Y.-G., Leclercq, M., Cevidanes, L., Prieto, J. C., Zhuang, S., Wei, G., Cui, Z., Zhou, Y., Dascalu, T., Ibragimov, B., Yong, T.-H., Ahn, H.-G., Kim, W., Han, J.-H., Choi, B., van Nistelrooij, N., Kempers, S., Vinayahalingam, S., Strippoli, J., Thollot, A., Setbon, H., Trosset, C., and Ladroit, E. (2023). 3DTeethSeg’22: 3D Teeth Scan Segmentation and Labeling Challenge. arXiv:2305.18277
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper presents an approach to teeth recognition based on their positional inter-dependencies using a convolutional neural network and assignment theory. The method constructs a distance-based anatomical model of teeth and uses the Hungarian method for assignment to improve the recognition results of the UNet. The authors validate their method on a public teeth dataset. Reviewers have raised concerns about the clarity of presentation and the lack of comparisons with other approaches in the field. The authors have addressed some of these concerns in their rebuttal and have promised to make revisions accordingly. The major weakness highlighted is the absence of comparative experiments on the same dataset for a fair evaluation. The authors have included some comparison results from a different private dataset, which may not be directly comparable to the results reported in the paper. Conducting comparative experiments on the same dataset would be more appropriate. Overall, the paper addresses an interesting problem that could be interesting to a targeted audience at MICCAI. However, due to the need for significant revisions and improvements, including improving the presentation quality, adding comparative experiments on the same dataset, and revising the title accordingly, the current version of the manuscript may not be ready for acceptance and would require another round of review before being accepted.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This work proposes a strategy to identify tooth instances from intra-oral 3D scans via a bipartite graph matching setup. This identification is claimed to be very robust in the presence of missing teeth or anomalies like multiple teeth. Its key strength is a strong result compared to SOTA for a recent MICCAI challenge dataset. Regarding weaknesses, I would not say that using the Hungarian algorithm for identification is a novel idea, this has been used in many contexts in the last decades. Therefore, innovation is limited, and the decoupling of the UNet based candidate generation from the labelling is probably not the best way to approach this problem. Finally, reviewers and meta reviewer complain about inconsistencies in formal notations and the document being hard to read due to many editorial problems. Authors address concerns in their rebuttal and promise to improve presentation quality as well as to weaken their claim of multi-object recognition to identification of teeth in the dental arch. Overall, I think despite the above mentioned weaknesses, this work could be interesting to a sub-group of the MICCAI community invested in 3D dental arch labelling. Therefore, I slightly tend to vote for acceptance of this paper, under the premise that the authors carefully revise the document according to the reviewing comments.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The methodology is sound and the experiments have also shown promising performance. Major weakness was on the lack of comparison against other approaches. The rebuttal has provided a few comparison results from the challenge which were not originally included in the manuscript and the authors have promised to include them in the final manuscript. However, the results seems to be from a different private dataset and may not be fair enough to be directly compared with the results from the proposed approach. The work could have conducted some comparative experiments on the same set of data for fair evaluation. The authors have also promised to clarify the raised concerns and revise the titles in the revision. Despite the weakness on comparisons, I think the methodology is interesting and the results also show promising performance, which could be of interest to the community for discussion.