List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Mykhaylo Zayats, Christopher Hansen, Ronan Cahill, Gareth Gallagher, Ra’ed Malallah, Amit Joshi, Sergiy Zhuk
Abstract
Efficient algorithms for solving inverse optical tomography problems with noisy and sparse measurements are a major challenge for near-infrared fluorescence guided surgery. To address that challenge, we propose an Incremental Fluorescent Target Reconstruction scheme based on the recent advances in convex optimization and sparse regularization. We demonstrate the efficacy of the proposed scheme on continuous wave reflectance mode boundary measurements of emission fluence from a 3D fluorophore target immersed in a tissue like media and acquired by an inexpensive consumer-grade camera.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43999-5_49
SharedIt: https://rdcu.be/dnww2
Link to the code repository
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents a novel approach to solving the FDOT problem with a strong potential for adoption in FGS-related applications. The authors have successfully demonstrated the effectiveness of the proposed IFTR scheme through a series of numerical experiments, showing its applicability in various experimental scenarios.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper proposes a new algorithm for solving inverse optical tomography problems that arise in near-infrared fluorescence guided surgery. The proposed algorithm, called Incremental Fluorescent Target Reconstruction, is based on convex optimization and sparse regularization. The goal is to efficiently and accurately reconstruct a 3D fluorophore target immersed in a tissue-like medium using continuous wave reflectance mode boundary measurements acquired by a consumer-grade camera. The paper demonstrates the efficacy of the proposed scheme in addressing the challenges posed by noisy and sparse measurements.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Introduction: The introduction provides an extensive background on the current state of NIR fluorescence imaging and its applications. However, it would be helpful for readers if the authors could provide a more concise problem statement early in the introduction, followed by the limitations of existing approaches, and the motivation for the new approach.
- The paper would benefit from the inclusion of some experimental results and case studies, comparing the proposed IFTR scheme to existing FDOT methods. These comparisons should include both qualitative and quantitative results, such as reconstructed images, error metrics, and computational efficiency.
- The paper mentions the use of the FEniCS and CVXPY packages. It would be helpful to provide more information about the specific roles these packages play in the proposed method, and how they are integrated into the algorithm. Additionally, the authors should discuss any limitations or challenges associated with using these packages.
- In the description of the tissue phantom and fluorescent target, it would be helpful to provide more context about why these specific materials and dimensions were chosen. Are these parameters representative of typical in vivo situations or based on specific clinical scenarios?
- While the authors present three sets of experimental measurements, it is not clear how these measurements were processed and analyzed to evaluate the performance of the proposed IFTR method. It would be helpful to provide a more detailed description of the specific steps taken to process the raw data and compare the reconstructed images to the ground truth.
- It is important to provide context on the choice of the OSQP and MOSEK optimization solvers. A brief discussion of their advantages and disadvantages, as well as any potential limitations, would help the reader understand their relevance in the context of this study.
- A thorough discussion of the results, including a comparison to existing methods and potential limitations, would further strengthen the paper. This discussion should include an analysis of the trade-offs between reconstruction error, execution time, and the number of measurements.
- In the Conclusions section, the authors should provide a concise summary of the main findings of the paper and highlight their significance. The current conclusion is rather brief and could be expanded to provide a more compelling closing argument.
- The authors should consider addressing the limitations and potential future work related to the proposed IFTR scheme. Discussing these aspects will provide a more comprehensive and balanced view of the proposed method and set the stage for future research in this area.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
After carefully examining the reproducibility checklist submitted by the authors and considering the requirements for reproducibility, I must state that reproducing the results of this paper may be difficult.
While the authors have provided a detailed methodology and experimental setup, they have not included code or data to support the results. Without access to the data and code, it may be challenging to reproduce the experiments described in the paper. Additionally, some of the experiments rely on proprietary software or tools that may not be easily accessible to others.
Furthermore, the paper lacks a clear description of the hardware and software used in the experiments, which could affect the ability to reproduce the results. Additionally, there is no mention of the version control system used to manage the code and data, which could make it difficult to track changes and reproduce specific versions of the experiments.
In summary, while the authors have made efforts to ensure the reproducibility of their results, the lack of code and data, as well as some missing details about the experimental setup, may make reproducing the experiments challenging.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
a) Provide a clearer and more detailed explanation of the methodology used in the experiments, including a discussion on the choice of optimization solvers, software packages, and other tools. b) Improve the presentation of results, particularly in figures, by providing more context and appropriate labeling. Make sure all figures are accurately referenced within the text. c) Provide a thorough discussion of the results, including comparisons to existing methods, analysis of trade-offs, potential limitations, and implications for future research. d) Expand the Conclusions section to provide a more compelling closing argument, summarizing the main findings and their significance. e) Address limitations and potential future work related to the proposed IFTR scheme to provide a more comprehensive and balanced view of the method. f) Ensure that the provided link to the data and code is active and that the data and code are properly documented for increased transparency and reproducibility.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
By addressing these suggestions, the authors can further strengthen the paper, making it a valuable contribution to the field. Overall, the paper has merit and potential, warranting a weakly accept decision, provided the necessary revisions are made to improve its quality and clarity.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The authors propose an iterative algorithm for fluorescent target shape reconstruction. To solve the ill-posedness of the inverse reconstruction problem, the authors apply four regularization terms to restrict the search space. The authors analyze performance and speed of three variants of the same optimization framework using both quadratic solver and conic solver. The results show the feasibility of using the proposed algorithm to reconstruct a cubic fluorescent target immersed in tissue mimicking solution.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors demonstrate the feasibility of using the proposed algorithm to reconstruct the shape of fluorescent target with an inexpensive imaging setup, which suggest the potential of the proposed method in clinical applications such as surgical guidance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The notations in equations are messy. Some equations are not well defined or clearly explained.
-
The proof for the discretization from the original partial differential equations is not provided.
-
The algorithm is only validated on the target with a simple cubic shape. It is not demonstrated whether the algorithm can reconstruct more complex shapes.
-
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The work is easy to reproduce since the manuscript describes the experimental setup in detail and provides the specific information about each hardware component. About the algorithm, although some equations and notations are not clear, they are still understandable and can be reproduced with the assist of the data and code that will be provided by the authors.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Major comments:
-
The discretization from Eq. 9-14 to Eq. 1 using finite element method is not straightforward. It may be better if the authors can provide proof in Supplementary or provide relevant references for proof.
-
$S_x(\chi)$ as a function of $\chi$ does not seem to be well defined. Since $S_x \in R^{N \times N}$ and $\chi \in R^{N}$, how do they have one-to-one correspondence? The same problem applies for $S_m(\chi)$ and $M(\chi)$.
-
In Eq. 3, what is the criteria for splitting the domain $\Omega$ into non-overlapping subdomians?
-
In page 4 last line, where does the subscript $i$ in $\chi_i$ come from?
-
In page 5 line 4, should $\chi \phi_x$ be $\chi^T \phi_x$?
-
In Eq. 5-7, it is not explained why the error terms are normalized by the norm of $y$.
-
In Eq. 6-7, the function is minimized by choosing both the optimal $\chi$ and the optimal $\phi_m$. However, only $\chi$ is written on the left hand side of the equations.
-
In Eq. 6, $E_{em}$ is not explained.
-
In Eq. 8, should $\chi_{n-1}$ and $\chi_{n}$ be $\chi^{n-1}$ and $\chi^{n}$ because $n$ indicates the iteration time instead of node indices?
-
In Fig.2, it is better to label the hardware described in data collection section paragraph 2.
-
In Fig. 4 left column middle row, why does variant II have higher reconstruction error using QSQP compared with the top row while it is easier to reconstruct?
Minor comments:
-
In method section, foward and inverse problems, “(see Appendix ??)” should be changed.
-
In Supplementary, Eq. 11 is blank.
-
In Supplementary Eq. 13, the variable $S$ is not explained.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The idea in this work is interesting and its feasibility is proved by phantom experiments. However, many notations are messy and some equations are not well defined or explained, which affects the readability of the paper. Moreover, only one phantom with simple cubic shape is demonstrated, which restricts the applicability of the proposed method.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
The authors have made great efforts to address certain concerns raised by the reviewers. However, there are still some drawbacks in the manuscript that require major revisions.
One concern is that there are insufficient responses to the notation-related questions. The reviewer remains unconvinced that the authors will adequately address the notation problems in the revised manuscript, which will affect its clarity and readability.
Additionally, the meta reviewer has pointed out that the use of normalized L2 error as a metric is not a standard practice and has recommended the utilization of the Dice coefficient instead. Although the authors provided a justification for using normalized L2 error to capture deviations from 0 or 1, it is suggested that the authors should consider employing soft Dice as an alternative metric for the same purpose.
Overall, while the manuscript holds value and presents an interesting approach, it currently still requires substantial improvements before it is suitable for publication, which is the primary reason for this decision.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Paper summary
This submission presents a novel approach to perform 3D reconstruction of a sub-surface anatomical structure using flourescence optical tomorgraphy and consumer-grade cameras. The main innovation is, as the authors claim, to be able to reconstruct the shape of a tumor-like target from reflectance mode steady-state CW measurements CW measurements, which, advantageously, can be acquired by a cheap standard camera. Flourencence imaging is a very active topic in both surgical clnical research, and increasingly in practice. Therefore, methods that can exploit this imaging modality for 3D reconstruction of sub-surface structures are scientifically of interest, with potential for real-world clinical translation.
Method
The problem is presented as the solution to a coupled system of elliptic partial differential equations (PDEs) that relate optical intensity measurements with an excitation of the target with a light source in the near infra-red wavelength. A mathematical formulation of the problem is presented (equation 1), which is then solved using a discretization scheme, and an inversion of the PDE. Importantly, the inversion problem is ill-posed, so regularization is employed that constrains the search space of admissible shapes. To this end, piecewise total variation (PTV) is emplotyed, which is related to total vartiation, but calculared over non-overlapping subdomains. A binarization of the solution space is also presented as a regularization strategy, however, I disagree. This is about shape representation, and it is confusing to state its as a regularization aspect.
The regularized PDE is inverted by non-linear optimization with three proposed variants involving different constraint relaxations, using off-the-shelf solvers (MOSEK and OSQP). The Born approximation is used to obtain an intial solution, which is then refined using one of the problem variations.
Experimental validation
The method is experimentally validated in a proof-of-concept physical phantom study, comprising a glass box filled with a fat emulsion with light scattering properties that mimic human tissue. A flourescent target is embedded, which has been ingected with ICG. Experimental measurements were collected in different conditions (top and side box measurements with the target immersed at two depths (3mm and 6mm).
Reconstruction error was assessed, in addition to execution time. The results shows that at 6mm depth, accurate reconstruction is very challenging, and all solution variants produce high relative reconstruction errors. Better results are obtained at 3mm depth, and with more observations (top and side images of the box). Substantial differences were observed between variants in terms of accuracy and computation speed, where variant 1 appeared to have greatest accuracy.
Main strengths
The main strengths of this submission are as follows:
- An innovative econstruction formulation to solve OCT reconstruction using NIR fluorescence imaging, with standard NIR cameras (expressed also by R1). This is both technically novel and of real clinical importance given the increasing proliferation of fluorescent imaging in surgery.
- A first proof-of-concept demonstration using a physical phantom (expressed also by R1)
Main weaknesses
-
Reconstruction beyond 3mm does not seem feasible. This work is missing comments on this, is it a fundamental barrier to clinical translation? What can be done to increase accuracy with greater depth?
-
Methodology and metrics to assess reconstruction accuracy are not clear: Please clarify how raw measurements were transformed to error metrics (including optical and phantom coordinate system alignment, etc.). The meaning of the reconstruction error metrics in Figure 4 are not clear (they are not standard). Since this is a binary 3D reconstruction problem. I believe standard metrics such as DICE score should have been used. I recommend DICE scores are described in the rebuttal.
-
Regularization parameter: Please clarify how the weight of the PTV term was tuned. This is likely to be a critical method hyper-parameter.
-
Notation: I found this very hard to read (as a field expert). It would have helped to have related mathematical terms with figure 1.
Rebuttal
Based on the main criticism expressed by myself (above) and the other two reviewers, I recommend that the authors submit a rebuttal, to precisely comment on them. Concerning reviewer 1, various important issues are raised, and please carefully consider them and respond. Note that although new experiments cannot be performed for the rebuttal, the authors should defend their experimental choices and absence of method comparison. Reviewer 2 also expresses various major comments that should be carefully considered and addressed in the rebuttal.
Author Feedback
We thank the reviewers for valuable comments. We will do our best to implement all the suggested editorial modifications and amend notation. In this rebuttal we address the main critical points.
On a comparison against existing FDOT methods Existing FDOT methods either employ both reflection and transmission mode Continous Wave (CW) measurements, or use time-consuming frequency-domain or time-domain measurements. Those measurements types are not suited for real-time 3D localization of tumors in an operating theater, where only low SNR CW reflectance mode measurements are available. This motivated the new approach presented in the manuscript which is, to the best of our knowledge, the first report of recovering 3D shape of tumor-like target from reflectance mode steady-state CW measurements acquired by a consumer grade camera. The key reason for not making a comparison against the SOTA FDOT methods is that none of the known ones supports the same type of measurements. Thus an adaption of SOTA algorithm would be required which is a non-trivial task given the ill-posedness of the inverse problem and its high sensitivity to changes in parameters. In our opinion such an adaption seems to be equivalent to the development of a new method.
On experimental setting choices We chose the phantom material to be Liposyn or intralipid, which is an accepted and widely reported material to have optical properties similar to soft tissue. The phantom dimensions were chosen to represent a typical surgical field while excising tumors to simulate provision of 3D guidance of surgeon via a flexible endoscope type fluorescence imager. The raw fluorescence data was minimally processed: the images were median filtered for smoothing and co-registered on the FEM mesh representing the computational domain. Cubical shape was only chosen as this proof-of-concept experimental data was acquired with cubical tissue mimicking phantoms. The method is general and depending on mesh discretization level, scalable to arbitrary domain and target shapes.
On the meaning of the reconstruction error metrics Although the considered problem is a binary reconstruction problem it is formulated as a convex optimization with box constraints: components of the unknown vector can take values from the interval [0;1]. Hence, it is an important for the algorithm to recover values that are very close to either 0 or 1. To capture that ability we used relative L2 error which is more robust in this regard than DICE score that requires projection of components of the reconstructed vector into either 0 or 1. In fact, we tracked DICE scores as they were used for verifying stopping criteria and report them for IFTR Variant 1 solved with OSQP and MOSEK respectively. For 6mm deep target both are equal to 0.831 (corresponding relative L2 errors are 0.528 and 0.573); for 3mm deep target DICE scores are 0.93 and 0.934; for 3mm deep target with observations on top and sides DICE scores are 0.952 and 0.953. We believe the values of DICE scores demonstrate that the quality of reconstruction even for a 6mm deep target is good. The shape of the target is reconstructed mostly correctly which is also demonstrated in Figure 1 in the manuscript.
On Piece-wise Total Variation (PTV) regularization parameter To make IFTR scheme robust with respect to PTV regularization parameter we scaled the data misfit and PTV term to similar magnitude. For this, we normalised the misfit term by the norm of the observations vector and rescaled PTV term by the number of subdomains and each local total variation weight by the number of nodes in that subdomain. The robustness to regularization parameter choice was confirmed by our experiments with several different values of such parameter. We also note that PTV impacts the loss function in a different way compared to L1 or L2 regularization: the latter has the unique global minimizer (0-vector), the former has many global minimizers and IFTR benefits from this.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Both reviewers initially recommended acceptance, but they, as well as I (acting 3rd reviewer) raised various major concerns. The authors have responded, however, reviewer 2 dropped their post-rebuttal recommendation to reject. The main outstanding issues were the lack of manuscript clarity / readability that cannot be adequately addressed in small edits allowed for the camera-ready version. I also agree that the written quality, particularly concerning mathematical notation and descriptions requires improvement.
On the other hand - this is innovative work that I think would be of good interest to the MICCAI community. The approach is novel, and while this is only an early proof-of-concept paper, in my opinion, it may stimulate further research on 3D reconstruction in fluorescence OCT with low-grade cameras. My own criticism about the use of L2 versus DICE was mostly answered, but at the same time I don’t accept the argument that because real-valued numbers are returned, DICE should not be used. e.g. we often have real-valued outputs from CNN segmentation networks, that are thresholded to obtain binary segmentations. The DICE scores were encouraging. I still feel that there is an important gap between phantom and real conditions, which may cause translation barriers, but time will tell. I highly recommend that the authors do their very best to improve mathematical notation and definitions in the camera ready version, with the use of supplementary material for notation definitions.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have adequately addressed the reviewers’ comments. There is sufficient value in the paper for acceptance to MICCAI. However, I would suggest the authors thoroughly address the reviewer’s comment on the notations used in this paper. This is required to improve the readability and accuracy of the paper.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes a method to 3D reconstruct sub-surface ICG contrast from a consumer-grade camera measurement. The method performs a discretised binary voxel reconstruction as a convex optimisation problem (with several regularisations), based on a physics model of photon propagation. The method is validated on a phantom with a cube target of known dimensions and position.
Strengths:
- All reviewers acknowledge that this an interesting idea
- All reviewers value this work as a nice a proof-of-concept
Weaknesses
- A reviewer mentions that experiments are only performed on a very simple, regular shape and it’s unclear how close this is to any real clinical scenario. Discounting the cubic shape, the rebuttal does not clarify if the dimensions/depth of the target are representative of any clinical scenario. It’s not clear if the heavy regularisations perform well with more complex shapes.
- A reviewer finds it difficult to follow the mathematical formulation, and I agree. I find it particular unclear how the authors define the split for subdomains I_i (eq 3). As a non expert, it would seem to me that this could be done in arbitrarily different ways - and it may heavily influence results. A few other latex typos/glitches are also making the formulation difficult to understand. None of this was clarified in rebuttal - so we have to assume the risk of these remaining unsolved in a hypothetical publication.
In my opinion this paper should be rejected as it needs significant revisions in presentation, and potentially other experiments that can assure the reader that all regularisations work well for shapes / traget depths of clinical relevance.
Side comment: I wonder if the minimum volume regularisation can potentially generate a non-existing reconstructed object if there is no presence of ICG is in the FOV.