List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Elena Denisova, Leonardo Manetti, Leonardo Bocchi, Ernesto Iadanza
Abstract
Three-dimensional (3D) rendering of biomedical volumes can be used to illustrate the diagnosis to patients, train inexperienced clinicians, or facilitate surgery planning for experts. The most realistic visualization can be achieved by the Monte-Carlo path tracing (MCPT) rendering technique which is based on the physical transport of light. However, this technique applied to biomedical volumes has received relatively little attention, because, naively implemented, it does not allow to interact with the data. In this paper, we present our application of MCPT to the biomedical volume rendering – Advanced Realistic Rendering Technique (AR2T), in an attempt to achieve more realism and increase the level of detail in data representation. The main result of our research is a practical framework that includes different visualization techniques: iso-surface rendering, direct volume rendering (DVR) combined with local and global illumination, maximum intensity projection (MIP), and AR2T. The framework allows interaction with the data in high quality for the deterministic algorithms, and in low quality for the stochastic AR2T. A high-quality AR2T image can be generated on user request; the quality improves in real-time, and the process is stopped automatically on the algorithm convergence, or by user, when the desired quality is achieved. The framework enables direct comparison of different rendering algorithms, i.e., utilizing the same view/light position and transfer functions. It therefore can be used by medical experts for immediate one-to-one visual comparison between different data representations in order to collect feedback about the usefulness of the realistic 3D visualization in clinical environment.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43987-2_34
SharedIt: https://rdcu.be/dnwJP
Link to the code repository
N/A
Link to the dataset(s)
https://www.kaggle.com/datasets/imaginar2t/cbctdata
Reviews
Review #1
- Please describe the contribution of the paper
This paper describes a framework to generate 3D rendering of volumes acquired by Cone Beam Computed Tomography (CBCT) with an application of Monte-Carlo path tracing( MCPT) – Advanced Realistic Rendering Technique (AR2T). The main objective was to achieve more realism in the rendered volume and to enable a direct comparison of different rendering algorithms. The authors provide advanced camera settings unlimited number of light sources. The authors used CBCT volumes acquired by Anonymous CT Medical Imaging Platform to present and compare their results with the results of the most popular techniques for volume data rendering; maximum-intensity projection (MIP), iso-surface rendering, and direct volume rendering (DVR).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The greatest strength of this article is the technical detail of the rendering description implemented around the new stochastic rendering algorithm. Although MIP and DRV are not new techniques and also not fully exploited in the clinical environment, the approach here proposed is innovative and allows direct comparison between the various algorithms. This comparison is very important for a fair analysis of the rendering resulting from the different algorithms.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
As main weaknesses, I think the authors focus too much on the technical description and leave little space for the results. Also, maybe a schematic of the framework would be helpful. The application and success of the rendering depends a lot on the data and the tissues that are intended to be visualized. Despite mentioning that the data is from CBCT, some information is missing about the application of this new technique to these specific data. To visualize data without being bone, how does it work? How does the rendering time compare with the other techniques? How were the transfer functions generated? The authors mention that the new technique “produces more realistic images and increases the level of details was achieved”. How was this measured? Information is missing in the captions and description of the figures, it is not easily noticeable which technique/settings they refer to. Furthermore, I cannot find information about the Anonymous CT Medical Imaging Platform (and no reference is given).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper has a Satisfactory reproducibility. The authors have a clear and detailed description of the algorithm, but but many of the used parameters are not specified. For example, in the reproducibility checklist, the authors mention “camera properties”, but throughout the paper, no details are given about this.
About the dataset, the autors mention the CBCT (Cone-Beam Computed Tomography) volumes acquired by Anonymous CT Medical Imaging Platform but no additional information is given about the platform. The exception is the Manix data set from https://docs.set.health/docs/introduction/public-medical-data which the authors refer to use and to which they present references.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
References are not correctly numbered and presented: they start at number 26 instead of 1.
-
The statement that “The most popular techniques for volume data rendering do not represent the data as seen in reality” should be better substantiated since they are the main competitors of the technique presented here. They are faster and some are already used in the clinic.
-Is this new technique/framework easily implemented in the widely used open-source Visualization Toolkit (VTK)?
-
Figures must have self-explanatory captions. As they are, we can’t see the difference between figures (a) and (b) for example.
-
Statements like “With the AR2T, the goal to present the technique that produces more realistic images and increases the level of details was achieved” in the discussion of results have to be supported in some way. As the results are presented, this conclusion is not straightforward.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The methodology presented in this paper is very interesting, clear and there is a lack of work about this topic in medical imaging (strength). However, the way the authors presented and discuss the results is weak.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Even though the authors justify the non comparison with other techniques, I believe the time issue should be in the text. As mentioned, not only the strengths of the method should be pointed out.
Some information about the transfer function should be added to the manuscript (such as the authors mentioned in the rebuttal).
I believe the paper could be improved even more but I understand the limit of number of pages. In general, the authors reply in a favourable way to my questions.
Review #2
- Please describe the contribution of the paper
The paper presents the implementation of a preexisting rendering method (Monte-Carlo path tracing) applied to the field of volumetric medical imaging. It is a path tracing-based method, therefore aimed at generating more realistic patient image renderings than the standard. The method is too computationally demanding to run in real-time, but thanks to taking an hybrid approach to rendering the authors implemented an interactive-time version of the method.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well written, even captivating. The flow of ideas is very clear and easy to follow. It is very pleasant to read.
The method seem to indeed produce photorealistic renderings of anatomical images.
The paper details clever implementation tweaks.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
In the introduction section, the authors state that: “It is well-known that realism is extremely important for clinicians. In fact, some of them refuse the technologies that represent the data in an unrealistic way.” This statement should either be backed by a reference or removed. I disagree with this statement. Clinicians use data representations that are not realistic all the time and are perfectly able to interpret them in most cases. I would argue that clinicians need representations that are anatomically accurate and clearly interpretable, much more than photorealistic.
All figure captions apart from 1 and 5 are insufficient. What are all subfigures? For Fig 2, what values for aperture and focal are shown? For Fig 6, I presume the left one is AR2T and the right DVR, but this shouldn’t be left to the reader to presume, it should be explicit in the caption.
It is mentioned that other standard methods are implemented too, but there is nothing any of them anywhere in the paper. How is your method better than those prior methods? There are no side-by-side comparison in any of the figures and no results about them either. Was the claimed realism of the renderings actually assessed with potential users? This point outlines a broader problem with the paper. The method wasn’t evaluated in any meaningful way. The only metric measured is convergence time, but the main motivation behind the development of the method, namely realism, is never assessed. Has any clinical user looked at the renderings?
The method could be released as open source rather than as free access to the executable. It would allow much more rapid development.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The method is the author’s own implementation, not released as open source. The data used by the authors is accessible however.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The paper can’t be accepted without any evaluation of the method, but it is worthy of doing so and resubmitting.
See other comments in strengths and weaknesses above.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The lack of evaluation makes it impossible to accept the paper as is, even though the method is interesting and look promising. I strongly encourage the authors to thoroughly evaluate their method against previous methods (already implemented, according to the paper) and resubmit somewhere else.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper introduces a novel advanced realistic rendering technique (AR2T) for 3D medical images, which aims to produce more realistic images with increased level of detail. The technique is implemented within a practical framework that allows for interaction with the data and permits comparison with different rendering techniques using the same view/light position and transfer functions.
The paper also proposes several improvements to existing techniques, including the advanced Woodcock tracking technique for volume sampling and the use of four well-known phase functions. The paper presents experimental results demonstrating the effectiveness of the AR2T technique, showing that it produces more realistic images with better visualization of anatomical structures compared to existing techniques such as Direct Volume Rendering (DVR).
The contribution of the paper lies in the development of the AR2T technique and its implementation within a practical framework, as well as the proposed i
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novelty: The paper introduces a new rendering technique, called AR2T, that combines volumetric rendering, Monte Carlo integration, and tracking techniques to produce more realistic and detailed 3D images of medical volumes.
Practicality: The paper presents a practical framework that allows interaction with the data, comparison of different rendering techniques using exactly the same view/light position and transfer functions, and evaluation of the rendering quality in terms of objective and subjective metrics.
Efficiency: The paper employs several optimization techniques to improve the rendering speed and reduce the memory footprint, such as Woodcock tracking, voxelization, early ray termination, adaptive sampling, and hierarchical interpolation.
Flexibility: The paper supports various types of transfer functions, phase functions, light sources, and noise reduction methods, and allows the user to adjust the parameters according to the specific requirements and preferences.
Validation: The paper evaluates the performance and quality of AR2T on several medical volumes acquired by different modalities, and compares it with other state-of-the-art rendering techniques in terms of visual fidelity, accuracy, and speed. The paper also presents the convergence analysis and the feedback from clinicians.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Limited evaluation: The paper presents only a qualitative evaluation of the AR2T technique using a few medical datasets and a single light source. A more extensive evaluation with a larger variety of datasets, transfer functions, and lighting conditions would be necessary to validate the general effectiveness of the proposed technique.
Lack of comparison: The paper does not compare the AR2T technique with other state-of-the-art methods in medical volume rendering, which limits the understanding of its strengths and weaknesses relative to other techniques.
Limited generalizability: The proposed AR2T technique may be sensitive to the specific data and transfer functions used in the paper, and its effectiveness on other datasets and transfer functions remains to be seen.
Computationally intensive: The AR2T technique requires significant computational resources and time, limiting its applicability in real-time or interactive settings.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper provides a detailed description of the methods and techniques used to implement the AR2T rendering framework, including the transfer functions, phase functions, and voxelization process. Additionally, the paper provides examples of the rendered images, as well as a convergence criterion for the iterative process. However, the paper does not provide the exact data sets used for the experiments, making it difficult to reproduce the results. Furthermore, while the authors mention evaluating the framework’s speed and quality, they do not provide any quantitative measurements or benchmarks for comparison with other approaches. Finally, the availability of the executable to the research community for further evaluation and comparison is currently under evaluation, which may limit reproducibility.
Therefore, while the paper provides a detailed description of the methods used, there are some weaknesses in terms of reproducibility, making it difficult to fully evaluate the framework’s effectiveness and compare it with other approaches.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I found your work to be a valuable contribution to the field of medical image rendering, and I appreciate the efforts you have put into developing the AR2T technique and the practical framework for its implementation. However, I also have some constructive feedback that I believe could further improve the quality and impact of your paper: Clarify the novelty of your technique: While you have provided a comprehensive overview of the AR2T technique, it is not entirely clear what sets it apart from existing rendering techniques in the field. I suggest that you include a more detailed discussion on the specific contributions and innovations of your approach compared to previous work.
Discuss limitations and potential drawbacks: While you have emphasized the strengths of your approach, it would be beneficial to discuss any potential limitations and drawbacks that may arise from its use. This could help readers better understand the practical applications and implications of your technique.
Provide more details on reproducibility: While you have mentioned that your framework is open-source, it would be helpful to provide more details on how others can reproduce and replicate your results. This could include providing specific instructions, code examples, or a link to a repository where others can access your framework and test your technique.
Discuss future directions: While you have briefly touched on the potential for future research in your conclusion, I suggest that you expand on this and provide more specific recommendations for future work that could build upon your approach. Overall, I think your paper is well-written and presents an important contribution to the field of medical image rendering. With some minor revisions and additions, I believe your work can further advance the state-of-the-art and provide valuable insights for researchers and practitioners in the field.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Based on the strengths and weaknesses identified in the paper, I would recommend acceptance with minor revisions. The paper presents a novel and practical framework for advanced realistic rendering technique (AR2T) that produces more realistic images and increases the level of details in medical imaging. The authors provide a thorough description of the methodology used and the experiments carried out to evaluate the effectiveness of their approach. The paper’s main strength lies in its practicality, as the framework allows interaction with the data and permits comparison between different rendering techniques.
However, the paper also has some weaknesses. The main weakness is the lack of a clear comparison with other state-of-the-art techniques, which would have helped to demonstrate the effectiveness of AR2T in comparison to other approaches. Additionally, the paper could benefit from a more in-depth discussion of the limitations and potential drawbacks of their approach, as well as
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper introduces a realistic rendering technique using Monte-Carlo path tracing and introducing a number of improvements so that it works in real-time. The method was then evaluated based on convergence time.
All reviewers agree that the paper is well written, the method is interesting with nice implementation details as well as the fact that the renderings are quite interesting. At the same time despite the mixed reviews some issues resonate were mentioned by all reviewers specifically the limited evaluation of the method and lack of comparison to other state-of-the-art medical volume rendering methods (even photorealistic ones). The authors should try to address the very detailed comments of the reviewers in their rebuttal with a focus on evaluation and comparison to state of the art and discussion on generalizability of the method to other types of volumes.
Author Feedback
We would like to thank all reviewers for their very polite and constructive reviews. We are also very pleased with your positive opinion of our Advanced Realistic Rendering Technique (AR2T) and the images it produces. We will try to address the major issues that our paper has according to the reviewers.
Q: Limited evaluation and lack of comparison A: Indeed, we do not provide a quantitative comparison between the images we generate with AR2T and the other rendering techniques because, to the best of our knowledge, there is no comparison metric that measures the photorealism of the image. At present, all comparisons reported in the literature are based on visual assessment. In the paper, we provide the results of the advanced deterministic direct volume rendering (DVR) with the different shading techniques applied, as shown in fig. 1a-1d. We also provide a one-to-one visual comparison (e.g., same viewing angle, same transfer function, same light position) between AR2T and DVR highlighting the level of detail, see fig. 6 (from left to right: broken knee with AR2T, broken knee with DVR Phong shading, dog abdomen with AR2T, dog abdomen with DVR Phong shading). Techniques such as maximum intensity projection or iso-surface are not discussed for the sake of brevity, since these methods are inferior to DVR and do not represent a novelty. Our method was informally evaluated by a group of clinicians, including experienced orthopaedic surgeons and radiologists, who confirmed the superiority of the proposed method; we did not mention this in the paper, as it remained an informal survey without a quantitative evaluation for the time being. Our further research will focus on comparison metrics based on photographs of scanned bodies. Our platform will allow a fair comparison between different techniques, which was not possible before. We refrain from comparing the speed of our method with DVR or others, since it is known that these methods allow real-time interaction, while our method is slower and it would not be fair to compare a stochastic algorithm with a deterministic algorithm. The comparison with cinematic rendering is not possible because there is no free access to this technology. Instead, we give the convergence results in terms of time and iteration number.
Q: Limited generalizability A: In our paper, we present the images generated from the publicly available spiral CT Manix (fig. 7). In the other examples, we use the CBCT acquired with the imaging platform in our layout (fig. 2, 3, 4, 6). The name of the platform is anonymized according to the anonymization breach rule. We also tested our method on publicly available MRI of the brain, but we did not include the generated images in the paper due to the paper size limits. We will substitute one of the CBCT example with the MRI. We applied our method using different transfer functions set manually, and the method works very well on hard and soft tissues and regardless of HU calibration (see fig. 2, 3, 4, 6, 7). Of course, transfer functions have a big impact on any rendering technique, and it is a matter of good knowledge of the properties of the tissues to be visualized.
Q: Limited reproducibility A: We have spent most of the paper describing the details of the AR2T implementation to allow easy reproducibility of our method even without open access to the source code. Moreover, we will provide the access to all datasets we have used in the paper.
Q: Incomplete captions A: We apologize for the missing information in the captions of some images. Where missing, the images are presented only as proof of concept, and the letters are necessary for reference in the convergence plots (fig. 5). The captions will be updated with all necessary information.
We hope to present our paper at MICCAI 2023 to stimulate interest in photorealistic visualization of volumetric biomedical acquisitions and to receive feedback from the community on the importance of our research.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I could go either way on this on the one hand it’s great to see a medical visualization paper and the results are quite impressive, on the other hand the paper does suffer from a lack of evaluation and comparison to other methods which I do believe could have been done.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Effective visualization of high dimensional medical images is an important topic and all reviewers acknowledge this.
However, photorealistic rendering of CT images is not new and commercially available products exist in this space. Thus, the complaints raised during review (the presentation of this work in isolation and without evaluation/benchmark to other techniques) are major and not adequately addressed in the rebuttal. While indeed there may be no way other than user studies to validate these developments, presenting this work without any such studies is insufficient. As did some of the reviewers, I urge the authors to perform this validation (as did other papers, as mentioned in the rebuttal) so that in the future work like this can be presented at MICCAI.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper presents a monte carlo path tracing based method for photorealistic volume rendering in biomedical images. Even though there are no metrics that can be reported for these types of methods, detailed clinician feedback should have been provided. Also only two example datasets are shown with no evaluation against other methods. For an application paper with requirements of more exhaustive evaluation and analysis for a specific domain, this paper does not meet the merits for publication.