List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Chi Xu, Alfie Roddan, Joseph Davids, Alistair Weld, Haozheng Xu, Stamatia Giannarou
Abstract
Probe-based confocal laser endomicroscopy (pCLE) allows in-situ visualisation of cellular morphology for intraoperative tissue characterisation. Robotic manipulation of the pCLE probe can maintain the probe-tissue contact within micrometre working range to achieve the precision and stability required to capture good quality microscopic information. In this paper, we propose the first approach to automatically regress the distance between a pCLE probe and the tissue surface during robotic tissue scanning. The Spatial-Frequency Feature Coupling network (SFFC-Net) was designed to regress probe-tissue distance by extracting an enhanced data representation based on the fusion of spatial and frequency domain features. Image-level supervision is used in a novel fashion in regression to enable the network to effectively learn the relationship between the sharpness of the pCLE image and its distance from the tissue surface. Consequently, a novel Feedback Training (FT) module has been designed to synthesise unseen images to incorporate feedback into the training process. The first pCLE regression dataset (PRD) was generated which includes ex-vivo images with corresponding probe-tissue distance. Our performance evaluation verifies that the proposed network outperforms other state-of-the-art (SOTA) regression networks.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16449-1_16
SharedIt: https://rdcu.be/cVRUW
Link to the code repository
Link to the dataset(s)
The link is being prepared and will be released once the paper is published.
Reviews
Review #2
- Please describe the contribution of the paper
X
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
X
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
X
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
X
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
X
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper describes the authors approach for feedback control of a probe-based confocal imaging system. The system is image-based, whereby the distance from probe to sample is optimized by examining the probe output. The network architecture is interesting, using both spatial and frequency domain learning, and a loss that function that encodes both the (supervised) optimal distance value and image sharpness. Overall it seems to be a worthwhile paper that solves an important problem. Evaluation against manually labeled distances shows good performance compared with other architectures.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
- The first approach is proposed to automatically regress the distance between a pCLE probe and the tissue surface during robotic tissue scanning.
- The first pCLE regression dataset (PRD) was generated which includes ex-vivo images with corresponding probe-tissue distance.
- A novel FT module to synthesise pCLE images between fine distance intervals to incorporate image level supervision into the training process.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is well-writen and well-organized.
- The first approach to tackle the pCLE probe and tissue surface with regression. The performance outperformed other methods.
- Dataset construction is helpful to the community and further experiments.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- In the Methodology part, a general picture and mathematical problem formulation are recommended.
- More details of fig.1 and 2 are preferred.
- The viusalization of result is recommended.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Video dataset is constructed.
- Public Data and code is expected to release.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- Too much details are introduced in the first paragraph.
- More introduction in the figure 1, 2 and its description.
- More straightforward visualization of result, rather than the Signal direction only.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The organization and clarity are good.
- The contribution is clear and solid.
- The Result is compatible while not clear to visualize.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
This paper presents a network, called the spatial-frequency feature coupling network (SFFC-Net), for pCLE probe to tissue surface distance estimation. The proposed network use both the image and frequency information for distance estimation. The author also proposed a feedback training strategy to boost the training. The experimental results show successful application on pCLE,
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- This is the first application for pCLE probe and tissue surface distance estimation.
- The authors proposed the SFFC-Net utilized both the image and frequency domain information.
- The authors proposed feedback training to boost the network training, with augmented data
- The results looks promising.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Equation 3, why use linear interpolation instead of other interpolation?
- Wouldn’t the interpolation create artificial image with artifacts or blurring that confuse the network?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
N/A
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please see above
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well-written and easy to follow. The application is new and the method consisting of SFFC-Net and FT seems novel. The experiment results also looks promising.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors address my comments.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Strengths: • The reviewers agree that the paper is well organized, novel and with promising results. Weaknesses: • The reviewers requested for more details on the methodology and the interpolation method used in their paper.
Points to be addressed by authors: • I would encourage the authors to rebut the suggestions/criticisms raised by the reviewers.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Author Feedback
We thank the reviewers for their constructive feedback. Please find our response below. Minor issues will be addressed in the revised paper.
Reviewer #3
Methodology and Fig. 1&2 description - The following will be included in the revised paper to improve the methodology description provided by Fig. 1&2 and include mathematical problem formulation: In Section 2: “The outline of the proposed framework is shown in Fig. 1. The overall architecture of Spatial-Frequency Feature Coupling network (SFFC-Net) is depicted in the grey box, which utilizes a Resnet18 backbone and a new Fast Fourier Convolution (FFC) block for spatial and global feature extraction. Further, the SFFC module is used to exchange and combine features between spatial and frequency domains. The feature vectors extracted from spatial and frequency domains are stacked and become input to the fully connected layer to infer the predicted tissue-probe distance df = f(x, \theta), where f(.) represents the SFFC-Net. The procedure of Feedback Training (FT) is demonstrated in the blue box. According to df, the discretization layer and differentiable frame-selection are used to select two existing adjacent frames to synthesized image Ipre=g(df), where g(.) represents the FT module. Then, Ipre is synthesized via per-pixel linear interpolation. Overall, the synthesized Ipre and predicted df are both supervision signals for image sharpness and regression learning, respectively. Our network is trained with the loss functions in Eq. (4)-(5).” In Section 2.1:“The SFFC-Net has 4 layers to extract spatial and global features. Each layer is constructed by the FFC layer, spatial convolutional (SC) layer and SFFC module. Since all the layers have the same architecture, the detailed architecture of the 1st layer only is demonstrated in Fig. 2.”
Result visualisation - The visualisation of the results will be enhanced by overlaying in the convergence graphs in Fig. 3 (right) pCLE images at different stages of convergence.
Release data and code - The code and dataset will be released when the paper is published.
Reviewer #4
Reasons of using linear interpolation - Frame interpolation is used during the training phase only, to synthesise the predicted image Ipre from two adjacent pCLE frames. For the generation of our pCLE Regression Dataset (PRD), the pCLE probe is moved with 5 micrometres step. For the collected pCLE frames, the Mean of Intensity (MoI) increases monotonically when the probe approaches the optimal scanning position and decreases monotonically when the probe moves away from it. Since the distance between adjacent pCLE frames is only 5 micrometers, the change in MoI between adjacent pCLE frames can be considered a linear function. Hence, linear interpolation can successfully reconstruct Ipre, while enabling the model to learn the relationship between MoI and probe-tissue distance effectively.
Artifacts or blurring issues caused by interpolation – During the generation of our pCLE Regression Dataset (PRD), the pCLE probe is moved with a 5 micrometres step. This small probe movement eliminates motion artifacts between consecutive pCLE frames and therefore prevents interpolation artifacts in the predicted image. Possible interpolation artifacts in the predicted image will be minor and cannot affect the performance of our method significantly. This is because the predicted image is not used as input to any module of the network. It is only used for image-level supervision which is minimally affected by interpolation artifacts as the proposed reconstruction loss incorporates both local and global image information. Our ablation study verifies that interpolation does not confuse the network as the FT module (image-level supervision) improves its performance.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have addressed the reviewers’ comments. I recommend accepting the paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have adequately clarified issues related to the methodology and validations. To help make the results reproducible, they promised to make the code and dataset available online when the paper is published. This is an interesting contribution and I recommend accepting it.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Strenghts lie in the methods novelty, its promising results, as well as clear presentation. Primary weaknesses pertained primarily to desired clarifications that were addressed in the rebuttal.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4