List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Baoru Huang, Yicheng Hu, Anh Nguyen, Stamatia Giannarou, Daniel S. Elson
Abstract
In surgical oncology, it is challenging for surgeons to identify lymph nodes and completely resect cancer even with pre-operative imaging systems like PET and CT, because of the lack of reliable intraoperative visualization tools. Endoscopic radio-guided cancer detection and resection has recently been evaluated whereby a novel tethered laparoscopic gamma detector is used to localize a preoperatively injected radiotracer. This can both enhance the endoscopic imaging and complement preoperative nuclear imaging data. However, gamma activity visualization is challenging to present to the operator because the probe is non-imaging and it does not visibly indicate the activity origination on the tissue surface. Initial failed attempts used segmentation or geometric methods, but led to the discovery that it could be resolved by leveraging high-dimensional image features and probe position information. To demonstrate the effectiveness of this solution, we designed and implemented a simple regression network that successfully addressed the problem. To further validate the proposed solution, we acquired and publicly released two datasets captured using a custom-designed, portable stereo laparoscope system. Through intensive experimentation, we demonstrated that our method can successfully and effectively detect the sensing area, establishing a new performance benchmark. Code and data are available at https://github.com/br0202/Sensing_area_detection.git.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43996-4_25
SharedIt: https://rdcu.be/dnwOZ
Link to the code repository
https://github.com/br0202/Sensing_area_detection.git
Link to the dataset(s)
https://github.com/br0202/Sensing_area_detection.git
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a deep-learning-based method to estimate the location where the signal originates when a miniature gamma probe is used during cancer surgery. A mock probe with the ability to indicate the location where the probe central axis and the tissue surface intersects is used to generate a dataset for supervised learning. Two different network realizations, one with a ResNet and the other with Vision Transformers, was investigated. The error in estimation was presented along with some qualitative results.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
A deep-learning-based method was proposed to regress the sensing are of a miniature gamma probe. Since only the laparoscopic images (monocular or stereoscopic) are used, the method does not alter the surgical workflow. The dataset released with this paper can potentially aid in training and evaluation of other tasks in surgical computer vision as well.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
[1] The validation is not complete: The authors report the estimation error of the sensing area in pixels in image space. How this translates to an error in 3D space is rather important. No failure cases have been discussed.
[2] The authors mention about a segmentation-based approach to the problem. Yet, it has not been compared to the regression-based approach.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors mention that the dataset will be made available upon acceptance of the paper. In addition, the parameters used in the experiments are discussed in the paper. With the source code available as well, I think that an interested reader will be able to reproduce the results easily.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
[1] The paper starts with a good introduction to the paper. Related work is discussed with references in section 2. In section 4.1, several alternative methods are discussed again. I think, the readability of the paper will improve had the discussion on prior-art in 4.1 is some how integrated to section 2.
[2] The authors discuss a method using image segmentation to solve the problem at hand. However, no results were reported. Neither did they compare this method to the regression-based method.
[3] Table 1 and 2 summarize the results obtained by different network architectures with different imaging modes (i.e. stereo vs monocular). When results are compared, the authors seem to merely look at the mean/median. Proper statistical tests should be used when different methods are compared. Conclusions reached without such tests do not carry much scientific meaning.
[4] The estimation error is reported in pixels. Instead, in surgery, it is more meaningful had it been reported in milimeters as error in 3D. With camera intrinsics known, it shouldn’t be difficult to the authors convert the error reported in pixels to milimeters. With 3D errors, more meaning clinical decisions can be made on how useful the proposed technique in the clinic.
[5] What are the limitations of the proposed method? A discussion on failure cases would have added significant value to the paper.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The use of deep-learning-based method to regress the sensing area of a gamma probe is novel. However, the validation is not adequate for the paper to be presented at MICCAI.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The author(s) propose a method to improve visualization during an endoscopic radio-guided cancer resection surgery. The proposed framework superimposes the intersection between the gamma probe axis and tissue surface into the laparoscopic image, without the need to track the probe. To train the framework, the author(s) developed a method to generate labeled image data using tissue phantoms.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed method would support a relatively newly developed technology of miniature surgical gamma probes and might have the potential to help with a faster adoption if this technology in clinical practise.
- The author(s) developed a training dataset with ground-truth data
- Author(s) reported a update frequency which would make the method applicable in a real-time navigation application.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It is unclear to me how the proposed method would be clinical applied.
- In general, the manuscript was well written and structures, but is missing some important details, such as details about the phantom, or split between trainings and test set.
- The authors created an impressive dataset for training the proposed method, but only using a phantom model. Testing was performed also only using phantom data, which is why it is unclear if the results are generalizable to a patient population.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The presented work has in my opinion a moderately strong reproducibility, mainly since the author(s) provided a public version of the dataset. However, some required details are missing in the manuscript, for example data split.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- I’m not clear on how your proposed method would be used clinically. From my understanding of the gamma probe, the output of the probe is not binary, but provides a range of gamma intensities. Would this not make the sensing area a cone? In your proposed method, sensing area is modeled as a point in the center of the sensing area. It would be helpful, if you can provide more details how the proposed method would be integrated in the surgical workflow. How would be additional information be used by the surgeon to improve the procedure.
- One of the limitations in your work is the use of tissue phantoms in generating the training dataset. To evaluate the possible generalizability of your results, more information about the tissue phantoms would have been helpful. How similar are the laparoscopic images from real surgical data?
- More details about the data collection for the training dataset are required. For example, what was the minimum and maximum distance between the gamma probe and the tissue? What was the visibility of the gamma probe in the images (minimum and maximum of how much percentage of the whole probe was visible in the image).
- What was the pre-processing for the PCA analysis of the probe axis? Thresholding, edge detection, …?
- Please provide more detailed reference information (full URL) for UK Cancer Research.
- In your conclusion, you summarize your work as “a new framework for using laparoscopic drop-in gamma detector …”, which is a bit general. A more specific summary would be beneficial for the clarity of the manuscript.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the work has some weaknesses, the topic is interesting and novel and might generate interest in the MICCAI community.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Based on the authors feedback I changed by opinion from a “weak accept” to a “accept”. A main reason for the improvement of the score is the addition of 3D measurements by the authors. The method is novel and I believe would make an interesting contribution to MICCAI.
Review #3
- Please describe the contribution of the paper
This paper aims to develop a tool to enhance the visualization of cancerous tissue in laparoscopic surgery to augment the possibility of adequate tumor removal.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
clear and defined need in the surgical domain
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
only phantom testing is performed
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
data set is public available, thus reproducibility can be achieved
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
congratulations for your work, do you have planned perform animal testing?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
abstract is aligned with the topic of the conference , with a potential clinical translation
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The manuscript received mixed reviews with the strongest positive review also being the least specific. The strengths are appreciated as an innovative method to an emerging technique, together with a highly specialized and new dataset that may enable novel research.
The chief weaknesses are identified as:
- Incomplete validation (no 3D error is provided, which seems relevant for the task)
- Concerns around the generalization of the method to patient specimens
- Need for clarification on several method aspects.
From my perspective, considerng the innovation of this approach that all reviewers agree on, issues around generalization from this phantom setup to real patient population can be addressed through discussio; I don’t think patient demonstration is necessary given the other contributions.
Author Feedback
We thank the reviewers and have addressed all the points raised.
Reviewer 1:
Related work amendment: The discussion about the alternative methods in section 4.1 has been integrated into section 2 in the final version.
3D error: The calculation of error in 3D needs depth maps, which are not available for the dataset we used for this paper. However, we have acquired a new dataset with the same experimental settings and with a custom-built structured lighting system for depth ground truth. The 3D errors for the Stereo-Resnet-MLP methods are: mean 8.9 mm, std 11.9 mm, median 4.7 mm, while the corresponding 2D errors are: mean 57.4 pixels, std 47.8 pixels, median 43.2 pixels. The results for all the tested methods have been added to the final version.
Segmentation-based vs. regression-based approach: The results of the segmentation-based approach results are in the supplementary material, and the regression-based results are in the main paper.
Statistical tests: In addition to adding the 3D error mentioned above, the 2D and 3D error versus the percentage of the probe in the laparoscope field-of-view was plotted and added to the final version, together with the error distribution. Additionally, the R-squared evaluation was performed and the score for the Stereo-Resnet-MLP method was 0.77.
Limitations and failure cases: The regression-based methods rarely failed but presented with higher errors than the segmentation-based methods. Conversely, more failure cases were noted for the segmentation-based approaches (no segmentated map was found) but the errors were smaller (documented in the supplementary material). Both methods have advantages and disadvantages and may be combined.
Reviewer 2:
Clinical application: Precise detection of the central point of the sensing area can help to target cancer or affected lymph node dissection, as the probe can be retained inside the abdomen and used at any time. The proposed method can be integrated into the laparoscopic vision system to provide augmented reality cancer tissue location, as requested by focus groups of surgeons evaluating the clinical system, although other methods - for instance, display on a separate monitor - are also possible.
Phantom details, similarity between phantom data and real surgical data: The silicone phantom was 30x21x8 cm and was rendered with tissue color manually by hand to be visually realistic. The laparoscope was identical to those in the operating room. Hence, the images from the laparoscope are similar to real surgical data. More data is currently being collected on human ex vivo tissue, which will be used as further validation.
Generalizability: As the laparoscope field-of-view is small compared with the phantom, the combination of the sample rotation and adjustment of the SENSEI/laparoscope position produces different images for the dataset. We also acquired further datasets to validate the 3D error. The consistent results demonstrated good generalizability, please see Review 1 Q2.
Dataset details: The distance between the gamma probe and the tissue varied from 1 to 5 cm. The minimum fraction of the probe appearing in the field-of-view was 50% and the maximum was 100%, mimicking the real surgical use.
PCA and training/test split: Edge detection was used for the PCA analysis. The training/validation/test split was 800/200/200 and the details have been added to the final version.
URL of CRUK, conclusion summary: The full URL of CRUK has been added in the final version and the general conclusion has been amended to be a more specific summary.
Reviewer 3:
- Animal testing: We have built a portable fully sequential and automatic hardware platform for data collection. More data are being collected on the real human ex vivo tissue in the operating room, which will be used as further validation.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
During initial review, the key limitations were 1) the incomplete validation, 2) generalization of the method, and 3) clarifications.
From my appreciation of all reviewing details, 1) validation was addressed partially by acuiring an additional dataset with 3D reference standard and evaluating the method on that data. Doing so is against MICCAI guidelines, and the errors reported in the response are large, making it unclear what exactly is going on. Consequently, while I appreciate the authors’ efforts, the value of this experiment is unclear as it cannot be well reviewed. 2) The authors comment on the realism of the phantom, but mention that real tissue studies are underway. It is unclear to me whether this study should have included at least qualitative results on that data already, to better mitigate this concern. 3) Clarifications: While there are several sensible responses in the rebuttal, it is not clear how they will affect the overall manuscript.
In summary, because of one of the reviews was considerably short and of little detail, the other reviews take higher priority and are diverging in rating. Given the above analysis, my assessment is that this is a borderline paper, unfortunately leaning toward rejection at this time.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Although the paper tests the algorithm only on a single phantom without clinical validation, I think the novelty of the paper is sufficient for acceptance to MICCAI.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This submission presents an approach to estimate the intersection point between a laparoscopic gamma probe’s beam in a stereo laparoscopic scene. The application is clinically relevant (CAI), since improved visual guidance is key to make the use of such tools viable for tumor resection and safe margin assessment especially for lymph nodes. This work is of technical interest because it is a markerless approach, with an effective self-supervised setup to obtain ground truth (laser pointer housed in a probe shell). The reviews were mixed. R1 proposed weak-reject, with the main concerns being inappropriate performance validation metrics (the use of pixel rather than 3D error distances), and lack of comparison with segmentation-based approach. The other reviewers were generally positive and most of their concerns were well-addressed in the rebuttal. Having consulted the rebuttal - I believe that the strong aspect outweigh the negatives, and it should be accepted. This is for the following reasons:- The authors have proposed to add 3D error metrics (I agree - very relevant) and comparison with the segmentation approach. While the accuracy is not super (likely not sufficient for clinical application), presenting these results are important for future improvement (especially as the dataset will be public)- The authors have clarified the second main concern from R1 (lack of comparison against segmentation)- The use of a laser pointer to get self-supervised labels will be interesting to the broader CAI community, since it could be applied for visualizing intersection points of other tools that may be difficult to track optically e.g. flexating needles. - CAI research to improve the use of pickup gamma probes has a lot of potential for clinical impact- While this submission has limits (phantom study, single device, lack of comparison with a more traditional approach e.g. markerless tool tracking), I believe it should be accepted.