Authors

Michael Sommersperger, Shervin Dehghani, Philipp Matten, Kristina Mach, Hessam Roodaki, Ulrich Eck, Nassir Navab

Abstract

Swept-Source Optical Coherence Tomography (SS-OCT) allows surgeons to perform certain ophthalmic procedures under the exclusive guidance of real-time volumetric optical coherence tomography (4D OCT). In such scenarios, surgeons are no longer limited to rigid views through an operating microscope. Instead, direct volume rendering (DVR) of 4D OCT enables surgical maneuvers to be performed from arbitrary viewpoints. While 4D OCT maximizes the use of the depth-resolved OCT data by displaying it from an oblique perspective, performing complex instrument maneuvers from such views places a higher mental demand on the surgeon. In this work, we propose an Intelligent Virtual B-scan Mirror (IVBM), a novel concept for surgical 4D OCT visualization to provide additional guidance for targeted instrument interactions. The IVBM integrates a virtual mirror into a selected cross-section of the OCT volume. This mirror acts intelligently by only being sensitive to voxels associated with surgical instruments. Furthermore, volume structures aligned with the IVBM are highlighted, while structures behind the IVBM are preserved through an adaptive opacity transfer function. Unlike previous perceptual OCT visualization concepts, which primarily address depth perception in axial OCT direction, this novel approach aids surgical interactions from arbitrary views. This paper presents the definition and implementation of an IVBM in a 4D OCT integrated microscope. Our user study in a virtual simulation environment confirms the benefits and provides insights into the interaction with the concept.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43996-4_40

SharedIt: https://rdcu.be/dnwPl

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

Please describe the contribution of the paper

The paper presents an original visualisation approach named “Intelligent Virtual B-scan Mirror” for making easier understanding of complex 4D OCT images. It relies on a virtual mirror paradigm to enhance surgical instruments visible in the images. The method includes a segmentation component which segments the surgical instruments from the images based on a U-net like architecture. Then, a dedicated ray tracing approach is proposed to implement different visualisation paradigms. The visualisation mode is evaluated from a computation time point of view demonstrating the interactive aspect of the proposed method (around 45ms per rendering). Finally, the visualization is evaluated through a user study involving 15 peoples and comparing navigation of an instrument with or without the proposed rendering mode. Results show the enhanced performance allowed by the visualization and lower workload assessed by Nasa-TLX score.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is very clear and strongly scientifically grounded and conducted. The proposed approach won’t be a revolution in the visualization community. But the work is well done with a great demonstration of the added value.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

Few weaknesses It needs an evaluation study with clinical experts to really show the added value in term of targeting and its clinical relevance. Does the difference in targeting is clinically significant ? This is not proved here.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Source code and data could be made available. It seems not to be the case here.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

I recommend to perform an evaluation study with clinical experts to really show the added value in term of targeting and its clinical relevance in concrete clinical procedures. Does the difference in targeting is clinically significant ?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The paper is very clear and strongly scientifically grounded and conducted.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #2

Please describe the contribution of the paper

This paper outlines some of the limitations associated with using Swept source Optical Coherence Tomography as image-guidance in ophthalmic surgical procedures. The particular issue being addressed here is the difficulty, even in a 3D OCT volumetric environment, of precisely approaching targets within the volume. To overcome this issue, the authors propose a virtual semi-transparent “mirror” embedded within the volume as a selected plane containing the target, so that as the target is approached, the image or the instrument and the target converge. The proposed method is trained to only recognise the surgical instrument of interest, so other extraneous devices do not participate in the reflected scene.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

This paper builds on the an extensive literature over the past decade or so dealing with the application of virtual mirrors in a number of scenarios, and describes this approach in a novel application to OCT-guided ophthalmic surgery. This approach represents a feasible addition to the standard surgical workflow that utilizes OCT. Key components of this work include the ability to place a semi-transparent mirror at will within the OCT volume; the intelligent recognition of the surgical instrument from the OCT volume so that only the salient components of the instrument are reflected in the “mirror image”; and the integration of the “mirror” within the OCT volume, rather than by rendering the scene from an alternative viewpoint and showing the mirror view next to the virtual objects. The paper is well written, contains a statistically meaningful user study that deals with both the targeting precision and user workload, compared to a baseline study that has the user perform the targeting task with a fixed visualization direction.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The authors state that visualization is achieved via a stereo VR headset. Is this normal for navigation in the OCT/ophthalmological surgery context? Since stereo visualization offers another depth cue, it would have been interesting to have some data comparing non-stereo visualization with stereo in both mirrored and non-mirrored examples. While a full set of experimental results on this would inflate the size of the paper beyond the page limit, if some (even anecdotal) data were available , it would be helpful to mention it. While the authors demonstrate that the targeting results are a statistically significant improvement over not using the mirror technique, can the authors nevertheless comment on the whether the increase in targeting accuracy is clinically significant in terms of projected patient outcomes. In other words, how often in practice does an OCT-guided procedure result in a less than satisfactory outcome because of the limitations imposed by the visualization environment. What is the ophthalmic surgeon’s take on the proposed technology? The paper mentions that the trial participants were “biomedical experts” familiar with ophthalmology and OCT, but does not identify whether they are familiar or experienced with the surgical procedure. The conclusions state that “we have demonstrated the IVBM in vitreoretinal surgery”. Strictly speaking , this should refer to a “virtual model of …”. The title of the paper should be more specific to reflect the application of the work in OCT-guided ophthalmological procedures. B-scan immediately suggests Ultrasound!
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

All relevant aspect of reproducibility have been addressed.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

Overall this is a well written paper that has a clear clinical benefit. While it exploits a technique (the virtual mirror) that has been around for some time, this is the first time to my knowledge that it has been employed as part of a 3D volume constructed from a 3D reconstruction of a tomographic image.
Although I see this technique as a definite aid to navigation for OCT-guided ophthalmic procedures, I do not see a clearly articulated clinical “Unmet Need”. What kinds of procedures are limited by not having this feature available? What are the anticipated improvements in patient outcomes/reduction in adverse outcomes, resulting from the use of this approach? Although the study shows a clear progression towards increasing precision amongst the evaluators as the trial progresses, it is not clear to me how this translates to the experience of an experienced ophthalmic surgeon. Is the goal to improve their performance, or to provide a platform that provide a more rapid path top precision surgery for a surgeon new to OCT? What degree of targeting accuracy is considered essential for the procedures described here?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Clinical applicability Clarity of writing Quality of results
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

7
[Post rebuttal] Please justify your decision

The authors have provided solid responses to reviews, and I stand by my original ranking. I understand that the author has another paper (2069) that builds on the same foundations, and is applicable to the same domain to address a similar problem. However the methodology is different but complementary. I would like to see both posters presented side by side, and if one is selected for oral presentation, I would encourage the presenter to allude to both approaches in the same talk.

Review #3

Please describe the contribution of the paper

This paper presents an interesting visualization technique that improves depth perception in a Virtual Reality setting, taking as application targeted instrument navigation in 4D OCT for vitreoretinal surgery. A user study was conducted with preliminary results.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- well written and structured document;
- the visualization technique seems to be slightly novel, but a novelty nevertheless;
- the application domain (vitreoretinal surgery) is a timely and fashionable research area
- well conducted user study
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- the document reports a few overclaims, namely the results do not correspond to a real surgical setting, which is a bit of a disillusionment.
- several technical details are missing, in particular, those related to the VR pipeline
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- several technical details are missing, in particular, those related to the VR pipeline
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

IVBM does not provide cognitive cues nor perceptual cues but depth cues. Please rectify the naming throughout the document.

1-2 sentences could have been added to better explain the surgical task at hands, which is to capture a minute mass with a pair of tweezers, correct?

It is not clear why “visualizing the intersection of the instrument with the cross-section” acts as a depth perception cue? Shouldn’t the instrument be always reflected on the virtual mirror? if the instrument intersects the plane then it seems to generate a depth conflict.

Sub-Section 3.3: join the paragraphs by removing the ‘return’ character between “(…) s with the surgical instrument. We define the IVBM plane(…)”

all vector quantities should be formatted using bold text type (e.g., Eq. (3) all quantities are vectors but are formatted as scalars.

Eq. (4) is not clear; what is the value of Mtool? Is it always a binary value (0 or 1)? If so, it should be Mtool = max{0, (1 intersection I(pt))> ttool}

The color map is an interesting feature but is it necessary to control all color dimensions? Since this application just needs a very limited color palette (i.e., just very few colors), modeling all three channels (R, G and B) seems to much (in my opinion the authors overcomplicated this aspect).

Please fuse Fig. 4 and Fig. 5 into a single Figure with part (a) and part (b). Placing two individual figures side-by-side is a bad formatting practice.

Strangely authors only mention VR in Section 4 … the whole experiment is a simulated one and does not correspond to a real surgical setting. This for me the major limitation of this work.

There is no mention to which VR headset nor any mention the VR software and versions used (was it Unity? Unreal? OpenXR?)
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

4
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The whole experiment is a simulated one and does not correspond to a real surgical setting. This for me the major limitation of this work. Several technical details are missing which make this work non-reproducible.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

The paper presents a novel visualisation approach “Intelligent Virtual B-scan Mirror” (IVBM) to improve perception of complex 4D OCT images. The method, as the name suggests, uses a virtual mirror to enhance surgical instruments visible in the images. An evaluation is done with novice users showing better target accuracy with the proposed method as well as lower TLX scores.

Although the reviewers agree that this is an interesting approach to improve perception in vitreoretinal surgery some issues have been raised by the reviewers including a better motivation and explanation of the surgical tasks, input from clinical experts, grounding the results in a real clinical task (e.g. in terms of required and current target accuracies), and more technical details about the VR set-up. The authors should addressed the detailed comments of the reviewers.

Author Feedback

We are very pleased that the reviewers find our manuscript “very clear and strongly scientifically grounded and conducted” (R1), and that they see a “clear clinical benefit” (R2) with a “statistically meaningful user study” (R2) that contributes to a “timely and fashionable research area” (R3). To address the reviewers’ comments, we provide more insights into the clinical requirements, feedback from surgeons, and more technical details of our user study. In the following, we summarize the reviewers’ comments and provide our answers.

Clinical targeting accuracy (R1, R2, R3): With the validation of intraoperative OCT through various clinical studies, the emergence of 4D OCT systems prompts the anticipation of more precise and efficient microsurgical treatments in the future. A common task in virtreoretinal surgery is to grasp and peel off an epiretinal membrane, a 60 μm thin layer that forms on top of the retina. With a manual tremor of 50 μm, surgeons navigate a pair of tweezers toward the on average 250 μm thick retina, while avoiding tissue damage. In this sense, any improvements in targeting accuracy, such as those shown in our study, can make a substantial difference and may lead to a safer and more efficient treatment. These details will be added along with references to our paper.

4D OCT system and user study (R3): Our system has been developed in close collaboration with clinical experts. We indeed integrated our method into a 4D OCT system and performed technical feasibility tests on a phantom eye, as shown in our figures as well as our supplementary video. Only our user study has been conducted in a virtual environment where 4D OCT was simulated with high fidelity, employing a method previously published in a biomedical journal (omitted due to anonymity, will be added upon acceptance). Rather than evaluating an imaging system, the main goal of our study is to evaluate a visualization concept, where both the baseline and IVBM visualizations have the same level of fidelity. Our simulation environment allows us to extract precise user information at each frame, providing the unique possibility to analyze the users’ approach thoroughly. While such user data collection would be very tedious with a real system, this is the first time such detailed analysis is reported, with the objective of providing valuable insights. VR setup (R2, R3): Stereo vision through a microscope is an inherent part of common ophthalmic procedures. In this sense, stereo rendering on a VR HMD integrates required cues for ophthalmic surgery. We will update our manuscript and provide more details on our VR setup with HTC Vive Pro and the Unity version.

Feedback from clinicians (R1,R2,R3): In the meantime, we have obtained direct feedback from four clinical experts, who confirmed the value of both the transparent property of the IVBM that does not obscure anatomical structures behind it and the in-situ visualization that does not distract the focus from the surgical area. We could integrate this early feedback, if wished for. In the longer term, we plan for extensive experiments with clinical experts on 4D OCT systems. To be sure about the concept and its face validity for community exposure, the current studies were essential, before taking the time of expert surgeons and planning extensive clinical experiments.

Based on the reviewers’ constructive feedback, we will make modifications regarding the formatting style of figures and mathematical symbols, use the term depth cue instead of cognitive/perceptual cue and mention the VR user study environment in the abstract. We agree with R3, that the intersection of tool and virtual mirror might create a perceptually challenging scenario, which would be interesting to investigate further. We thank the reviewers for their constructive comments. We hope this paper allows the advancement of intraoperative visualization methods to enhance the benefits of emerging real-time 4D imaging modalities.

Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

It was raised by a number of reviewers and meta reviewer that the content of this paper is too similar to an already accepted paper at MICCAI without enough novelty between the two to warrant accepting of both papers.

Meta-review #2

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors have addressed the reviewers’ comments. The paper has sufficient value for acceptance to MICCAI.

Meta-review #3

Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

The authors responded adequately to the reviewers’ comments

back to top