List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Marek Wodzinski, Mateusz Daniol, Daria Hemmerling, Miroslaw Socha
Abstract
Each year thousands of people suffer from various types of cranial injuries and require personalized implants whose manual design is expensive and time-consuming. Therefore, an automatic, dedicated system to increase the availability of personalized cranial reconstruction is highly desirable.
The problem of the automatic cranial defect reconstruction can be formulated as the shape completion task and solved using dedicated deep networks. Currently, the most common approach is to use the volumetric representation and apply deep networks dedicated to image segmentation.
However, this approach has several limitations and does not scale well into high-resolution volumes, nor takes into account the data sparsity. In our work, we reformulate the problem into a point cloud completion task. We propose an iterative, transformer-based method to reconstruct the cranial defect at any resolution while also being fast and resource-efficient during training and inference. We compare the proposed methods to the state-of-the-art volumetric approaches and show superior performance in terms of GPU memory consumption while maintaining high-quality of the reconstructed defects.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43996-4_32
SharedIt: https://rdcu.be/dnwPb
Link to the code repository
https://github.com/MWod/DeepImplant_MICCAI_2023
Link to the dataset(s)
https://www.sciencedirect.com/science/article/pii/S2352340921001864
Reviews
Review #3
- Please describe the contribution of the paper
The paper describes a method for reconstructing missing fragments of a skull based on a 3D binary volume. The pipeline includes creating a point cloud from the binary volume, splitting the point cloud into coarse point clouds, calculating the missing point cloud for each group using a modified geometry-aware transformer network and a Density Aware Chamfer Distance (DACD) objective function, merging the reconstructed point clouds, and post-processing for evaluation. The method is evaluated using the Skull-Break and Skull-Fix datasets, and the authors perform several ablation studies.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The introduction provides an excellent overview of the existing literature on cranial defect reconstruction using deep learning methods. The authors identify the limitations of the existing approaches and explain the need for a more efficient and effective solution. They also provide a rationale for their proposed approach, highlighting the advantages of point cloud completion over volumetric segmentation. However, it would be useful to include information about the dataset and evaluation metrics for the existing methods.
The paper provides a detailed description of the proposed method and its various steps. The use of a modified geometry-aware transformer network and a DACD objective function is a novel contribution, and the iterative completion process is a useful improvement. The evaluation using the Skull-Break and Skull-Fix datasets and ablation studies is comprehensive and provides insights into the method’s performance and limitations.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The abstract does a great job of summarizing the research and outlining the problem statement and approach. However, it would be helpful if the authors included more information about the dataset used and the evaluation metrics employed.
The results section provides limited data, and it would be beneficial to include more detailed results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors have provided detailed information about the datasets used, the network architecture, the objective function, and the evaluation metrics, which is essential for reproducibility. Additionally, the authors have provided a detailed description of the pre-processing steps, post-processing steps, and the iterative completion process, which can also aid in reproducibility.
However, it should be noted that the authors have not provided code or data as part of their reproducibility efforts. While this is not a requirement for acceptance, it can significantly enhance the reproducibility of the research. Providing code and data would allow other researchers to directly reproduce the results and also enable them to build upon the existing work. Therefore, it is recommended that the authors consider providing code and data in the future, if possible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I have read your paper titled “High-Resolution Cranial Defect Reconstruction by Iterative, Low-Resolution, Point Cloud Completion Transformers” and I appreciate the significant contribution you have made towards the field of cranial defect reconstruction using deep learning approaches. Your paper provides a clear motivation for your proposed approach and highlights the limitations of existing methods. However, i have some concerns about the contents of the paper.
More details on datasets and evaluation metrics: While the abstract and introduction provide a clear overview of the problem statement and motivation, it would be helpful to include more information on the dataset used and the evaluation metrics employed to assess the performance of the proposed method. This information would help readers understand the specific challenges of the problem and how the proposed approach addresses them.
Additional details on limitations and potential sources of bias: While the results of the paper are impressive, it would be beneficial to include more information on the limitations of the study. Were there any confounding variables that you were not able to control for, or were there any potential sources of bias in your data collection process? Providing more detail on potential limitations would help readers understand the scope of your findings and implications.
Code and data availability: The reproducibility of the paper is an important consideration. While you have filled out the reproducibility checklist, it would be helpful to provide code and data to make it easier for others to replicate your work. This would also increase the transparency of the research and provide a basis for future studies.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Originality: There is very limited research on this topic in the previous research, however, this paper is not very good but considerable.
Technical quality: The quality of the research methodology, including the soundness of the experimental design, data collection methods, analysis, and statistical techniques used.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This study proposes a method to reconstruct cranial defect at any resolution by treating the problem as a point cloud completion task. The developed method achieved high-quality reconstruction comparable to the conventional volumetric reconstruction method while requiring significantly less computational resource (GPU memory)
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors reformulated cranial defect reconstruction into point cloud completion task in an effort to overcome the limitations of methods based on volumetric domain. The point-cloud-based approach can be very efficient in memory consumption than volume-based approach. The quality of reconstruction result of this study was comparable to that of the state-of-art networks while improving computational efficiency without losing generalizability. The authors modified geometry-aware transformers so that the network can deal with solid 3D models. The reconstruction was performed iteratively by repeatedly processing coarse point clouds with a different split to improve the result, which is another advantage of using point-cloud-based approach.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
It’s hard to qualitatively evaluate the result despite the provided comparable quantitative evaluation results compared with the existing methods. Because of the nature of point-cloud data, I believe the result may suffer from noise and requires further processing to reconstruct surface mesh or volumetric data, which can also be technically challenging or may induce additional noise. So quantitative evaluation only does not prove the clinical feasibility or usefulness of the current study.
Lack of justification for the importance of memory efficiency to the cranial defect reconstruction task. Considering modern GPUs, even those for general consumers, have large enough memory for the volumetric approach. In addition, I believe the reconstruction task is not time sensitive. Therefore, simply adding more GPUs can be a reasonable solution to further improve the quality of the volume-based approach, which already has been performing well. Despite the resource efficiency, the training time of the proposed method was comparable or even longer than the volume-based method. The authors claim the point-based method is more efficient in dealing with sparse or high-resolution data set, which was not necessarily proved in this study.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility can be rated as sufficient.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The authors pointed out that one of the disadvantages of the volume-based approach is it requires further postprocessing to transfer the reconstruction into a printable model (surface model). However, I think this is not fair because the same is true for the point-based approach that also requires postprocessing of converting point-cloud into surface model. I would like to recommend providing surface representation of the result in Fig 2 instead of the point-cloud. Eventually, the surface model is required for the 3D printing of personalized implant. 3D surface mesh images can help better understand the quality of the outcome for the clinical application. Please clarify how N coarse groups of PC were acquired. Were they down-sampled to the specific number of points and random sampling was applied N times to get N groups?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Interesting point of view to tackle the existing problem of cranial reconstruction. However, rationale for such innovative approach is weak from both technical and clinical perspective.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
The contribution of the manuscript is a novel point cloud-based method for the task of cranial defect reconstruction. The point cloud completion network uses transformers and can be used in an iterative fashion at inference time to improve performance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Novel application of point cloud completion transformers to cranial defect reconstruction o Previous approaches to craniofacial reconstruction have predominantly relied on voxelated representations of skull geometry and deep learning networks1 or triangulated surface representations and statistical shape models2 for the cranial defect reconstruction task. o The authors combination of a deep learning based network and a point cloud representation is novel for this task o The network architecture is also novel, it is an extension of previous work to 3D solid objects. This could have more applications and is useful.
- Rigorous comparison with the state of the art deep learning based methods o The authors implement and compare their results to multiple other deep learning based methods. o Further the authors compare their results to values reported in the literature on open datasets. o The approach presented in this investigation use a point cloud representation rather than a voxel based one.
- Point cloud representation allows the network to be used with an iterative approach. Sub sets of the entire point cloud representing the skull can be run through the network to generate a better resolved prediction
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The authors do not adequately consider statistical shape models in their comparison. Statistical shape models(SSM) have been extensively applied to skull reconstruction in the literature2. SSMs share many similar traits with the authors approach, including many of the advantages, more efficient in terms of computational resources, 3D points-rather than a voxelated representation, better generalizability, and worse raw performance in comparison to voxelated deep networks. The investigation would be greatly strengthened by comparison to SSMs
- The authors do not put their results in the context of the clinical need. There is little discussion of the exact target for accuracy that is needed. Are there results good enough?
- The network architecture is not well described. The authors essentially refer to another paper. Making the reference is appropriate but there should be a brief description of the network architecture this would aid readers in understanding the manuscript. Further the explanation of the modifications is extremely brief. What makes the authors implementation “[focus] on solid 3-D models”
- The ablation results are quite limited and their meaning is not discussed. The differences do not seem large for multiple iterations for DSC but do have more of an impact for CD and HD95. Is there a mechanism in the iterative approach to remove points that appear spurious on iteration using some sort of voting or outlier detection? Why do you see the results that you see. Did you consider many more iterations like 10 or 100? Why so few?
- The lower computational resource use seems pretty limited. The authors seem to mean that less memory is used. The inference and training times do not appear to be smaller for the point clouds than the voxelated methods. Can the authors report training time or inference time for the algorithms? The computational resources seem to be a main feature here and the compute time is another relevant factor.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- good things about reproducibility o open datasts used o objective function and training are well described
- limiting the ability to reproduce o code not provided o network architecture not well described as discussed in weaknesses
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- discuss statistical shape models and their performance for cranial defect reconstruction in the intro, consider adding them as another algorithm to be compared against. They are used for this task and share many similarities with the proposed method.
- Grammatical errors o Change “The cranial damages are” to “Cranial damage is” o Change “We train the network supvervisedly…” to “We train the network using a fully supervised approach..”
- “3D printing” this should be broadened to CAD/CAM or computer aided design and manufacturing or something similar. More techniques than just 3d printing are used to create custom implant shapes.
- Results section: State the main results. All that is in the results section is a reference to the tables and figures without statements on what the results are. It would be helpful to have a summary of the results
- Up and down arrows would be helpful in the tables to identify the desired direction for the individual metrics
- Figure 2 could be improved by a colouring scheme that shows distance to ground truth. Is the idea that the point clouds are noisier?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The application of point clouds to cranial defects is novel and could be useful. The main drawbacks are modest results, including ablation results not showing much variability and a few minor points of clarification and omissions
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
I recommend acceptance.
As mentioned by the reviewers this is a novel point cloud-based transformer method to help reconstruct cranial defects. In general the reviewers found this paper and interested approach to the topic with a significant technological contribution. There are a wide range of suggestions for further improving this paper including providing more details on the dataset and evaluation, describing the rationale behind the approach proposed, and describing how their method compares to more classic approaches such as statistical shape models. However, there are no major flaws in the study that any two reviewers brought up, and most of the suggested changes are textual to give more insight into their approach, limitations of the study and a comparison with other approaches which are all appropriate to mention but do not significantly change the key findings of the manuscript.
Author Feedback
First of all, we would like to thank the reviewers and meta-reviewer for constructive feedback and the acceptance. Please find below short responses to the selected comments (due to the limited amount of space): 1) The authors do not adequately consider statistical shape models in their comparison… The investigation would be greatly strengthened by comparison to SSMs. Thank you for pointing this out – we agree with this comment. We are currently exploring both the SSMs and the reconstruction directly at mesh representation. Nevertheless, we decided to exclude the preliminary SSM results due to limited space and to focus directly on the proposed approach and comparison to other deep learning-based methods. We plan to extend the paper intro journal article and present a strong comparison to SSMs. 2) The authors do not put their results in the context of the clinical need. There is little discussion of the exact target for accuracy that is needed. Are there results good enough? The paper presents an algorithm for cranial defect reconstruction. This is a first step in the procedure of personalized implant modeling. The reconstructed model cannot be directly applied in clinical practice (many reasons) – another, following method for modifying the model into implantable implant is required. 3) The ablation results are quite limited and their meaning is not discussed… The number of iterations is defined as the number of reconstruction using the entire dense point cloud (not the number of group passes). Repeating the process for more than few iterations will not improve the results. The whole idea to repeat the process a few times is to close the holes appearing in the point cloud representation. The process improves the DSC, however does not affect that much the boundaries (HD95, BDSC). 4) The lower computational resource use seems pretty limited.. From the perspective of clinical application the most important aspect of the computational complexity is the ability to run the method. The defect reconstruction is not required in real-time - everything at the level of up to one minute is acceptable. However, the required CPU/GPU memory is crucial to ensure that the method can be eventually applied in practice. The proposed method requires just several seconds for inference (or less if the number of iterations is reduced). 5) It’s hard to qualitatively evaluate the result.. This is indeed true, however, as mentioned earlier – the defect reconstruction is a first step in the personalized implant modeling. Another algorithm and different type of evaluation is required to evaluate the clinical feasibility of the proposed method (e.g. expert assessment or quantitative evaluation based on desired geometrical/mechanical properties like dilation between the skull and the implant and others). 6) Lack of justification for the importance of memory efficiency to the cranial defect reconstruction …which was not necessarily proved in this study. Indeed, in principle one could apply e.g. data model parallelism and propose volumetric model working on reasonable resolution. However, this would require access to a whole GPU cluster with a lot of GPU days. It is not a problem during the training phase (one time development cost), however, introduces significant constant cost during the inference (practical use of the system). Nevertheless, we are working in parallel also on the reconstruction in the volumetric and mesh representations and hope to finally decide which representation is the most practical for the task. We will incorporate majority of the suggestions in the CR version of the paper (typos, limitations, others). However, few of them may be difficult due to the limited space available. Part of the source code will be freely released (repo in the CR version), however we cannot release the whole implant modeling framework due to the grant agreement – it is being developed as a ready-to-use system that will be eventually certified.