Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Sidaty El hadramy, Juan Verde, Nicolas Padoy, Stéphane Cotin

Abstract

This paper addresses the need for improved CT-guidance during needle-based liver procedures (i.e., tumor ablation), while reduces the need for contrast agent injection during such interventions. To achieve this objective, we augment the intraoperative CT with the preoperative vascular network deformed to match the current acquisition. First, a neural network learns local image features in a non-contrasted CT image by leveraging the known preoperative vessel tree geometry and topology extracted from a matching contrasted CT image. Then, the augmented CT is generated by fusing the labeled vascular tree and the non-contrasted intraoperative CT. Our method is trained and validated on porcine data, achieving an average dice score of 0.81 on the predicted vessel tree instead of 0.51 when a medical expert segments the non-contrasted CT. In addition, vascular labels can also be transferred to provide additional information. Code will be publicly released.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43996-4_28

SharedIt: https://rdcu.be/dnwO2

Link to the code repository

https://github.com/Sidaty1/Intraoperative_CT_augmentation

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    This paper presents a framework to project a patient-specific vessel segmentation obtained from a pre-operative contrast enhanced CT on an intra-operative non-contrast CT for the purpose of CT guided abdominal needle interventions. To accurately project the intra-operative contrast, the authors use a U-Net that receives an intra-operative CT without contrast and a pre-operative CT without contrast masked by the regions where vessels are present (pre-operative CT includes registered contrast and non-contrast CT). The network then learns to deform the intra-operative CT onto the masked pre-op CT, and the result is used to warp the pre-operative vessel tree onto the result. The network is trained and tested separately in 4 singular animal cases, where each case contains contrast and non-contrasted CT for both intra-operative and pre-operative settings. The approach is compared to manual segmentation by 2 clinical experts, and Dice score is used as a metric.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper is well motivated.
    • The framework is potentially easy to translate to the clinic.
    • Even though the dataset is limited, the proposed augmentation seems to be a good workaround.
    • The authors present additional experiments to further motivate the proposed method.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • It is not clear that the framework is performing the deformation compensation that the authors claim.
    • No fair comparisons are presented.
    • Results are limited to Dice score analysis, without any additional detail on the accuracy of specific landmarks and vascular labels which are shown in the results
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The authors provided the majority of necessary details to reproduce the work. Main aspect I did not find was the method to obtain ground truth vessel segmentations from the contrast-enhanced CT scans.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    • From the presented visual results, I am not sure the framework is achieving any deformable update on the vascular tree predictions as the authors claim. If I understood correctly, the aim is to deform the pre-operative vessel tree so that it matches the intra-operative non-contrasted CT. In Figure 5, I observe that the Augmented CT is a deformed version of the Intra-operative CT, suggesting that that the framework deforms the intra-operative CT to the pre-operative and not the opposite.
    • From Figure 6, I observe that the main update on the vessel tree is actually a translation and not a deformation (the portal vein vessels just translate almost) – looking at Figure 5 again, it seems the difference between non injected CT (left) and augmented CT (middle) is a translation as well. This raises the question of whether a simpler rigid registration would lead to comparable results or not. This is also supported by the Voxelmorph experiment – although I do agree the experiment does show that a deformable registration algorithm does not capture liver tissue deformations, one can see that the source and target image are not rigidly aligned. In that case, the obtained deformations are not completely valid. What would be the Dice score if a rigid registration was performed and the vessel contrast overlaid?
    • The method is compared only to the manual segmentation of two clinicians. I wonder if this is a fair comparison, as segmenting vessels manually from a non-contrasted CT seems a very demanding task. A simpler rigid method (as in the point above) or one of the methods the authors introduced in the background method would strengthen the paper and convey the effectiveness of the approach.
    • I agree that the possibility of providing an overlay with the vascular labels is of great interest. The authors could have used this feature to get other error metrics apart from DICE (for example, Target Registration Error on vessel branching points). Also, some validation on the use of these labels could have been included, given that they compose a significant part of the method.

    Some minor points:

    • When mentioning the rigid registration case leading to non accurate results in the Introduction, I suggest including a reference or experiment to support this statement. When mentioning deep learning techniques to solve the problem, I would also suggest including a clearer statement on what is the necessary accuracy for CT guided needle interventions.
    • The paper has a few typos such as “dilatation (dilation)” and would benefit from proofreading.
    • Authors should specify if the expert clinician Dice scores in table 1 are the mean across two datapoints.
    • Figure 5 should have a better contrast to help the reader better understand the results.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The lack of comparison with previous methods or with a simpler rigid registration. From my understanding, the method is mainly capturing a rigid registration between two CT scans.

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    6

  • [Post rebuttal] Please justify your decision

    The authors have addressed my main concern and ensured that better visual results will be included to demonstrate the effectiveness of their approach. I therefore have increased my score towards acceptance.



Review #2

  • Please describe the contribution of the paper

    The authors present a method for the augmentation on non-contrast enhanced intraoperative CT liver acquisitions with the vessels extracted from preoperatively acquired contrast enhanced CT. In minimally invasive liver interventions, non-contrast enhanced CTs are used as control to guide the medical doctor while inserting the needles in the correct place. However, tumors and risk structures like the vascular trees are not visible without contrast injection. In the proposed method the previously extracted vessels (vessel map) together with the intraoperative CT are the inputs for a U-Net that predicts the vascular map of the intraoperative image. The predicted vascular map is converted into a segmentation that is overlapped into the CT. The augmentation is enhanced with labels using an underlying graph. 4 couples of multiphase CT porcine abdominal acquisitions were used for the evaluation.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • Relevance and novelty of the topic: The topic treated by this paper is relevant. Most of the methods that are proposed in the state of the art are focused on physicians that use ultrasound images as intraoperative modality. However, not so many methods deal with the challenge of registering intraoperative non-contrast CT with their preoperative counterpart. All in all, there are innovative aspects in the proposed method. In particular, the need of augmenting the intraoperative image with the preoperatively extracted liver vasculature is very relevant and often disregarded in other works, that focus often on the use of ultrasound images intraoperatively. This makes it an interesting paper to read.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • Objectiveness of the evaluation: The evaluation assumes that the extracted vessels are only slightly deformed between phases and use them thus as ground truth. This reduces the statistical meaning of the results, as deformations between different phases are not small. It is probably something that the authors could not notice while working with a small set (4) of porcine data. Furthermore, judging the results is difficult if the measure that is used as objective metric are the vascular system that is not visible in the image. Basically, something that cannot be proven to be correct or wrong. The authors could consider adding a multiphase registration as new step, as suggested in the comments to the author. This might help in this regard.
    • Feasibility: The evaluation should be planned in am more realistic way. The deformations given during intervention are large. And the intraoperative CT images look quite different than the preoperative ones. The places where the needles are injected suffer usually large deformations that are extended towards the inner part of the liver. Furthermore, imaging artifacts (caused by the needles) arise that are visible in the intraoperative CTs but not in the preoperative ones. Due to these reasons, it remains open whether the method proposed could work in a real environment.
  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The method is lacking information regarding how the vessels are segmented from the contrast enhanced images. Given that vessel map is one of the two inputs to the neural network that will result in the “intraoperative” vascular tree, it is huge importance to understand how this should be done.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    The state of the art presented on the paper is quite complete. However, it could be improved by considering not only methods that use deep learning for image registration. There are plenty works that have been presented during the years for the registration of liver acquisitions. Specially, interesting for this work are papers for multiphase liver registration. While the idea is innovative, there are some aspects of the proposed workflow that are not comprehensive:

    • In the paper the vessels segmented from the contrast enhanced images are used as vessels of the non-contrast enhanced CT. The authors use a dilation operation to cover the “slight anatomical motion” between acquisitions. However, the deformations given different phases of liver CTs are not that “slight”. And this assumption might limit the accuracy and usability of the proposed method. A potentially interesting why to solve this problem would be to add a multiphase registration prior to the assumption that the vessels can just be taken from a different phase.
    • Regarding the labelling, I would just like to share some information that could help the authors in the future. The assumption that the branch with the highest radius can be considered as the root edge in the graph is risky. Whereas intuitively one can think this way, imaging artifacts among other reasons, cause this not to be always the case unfortunately, after analyzing many clinical images. All in all, the idea is innovative. Multiphase registration methods usually do not relay on anatomical landmarks like the liver vasculature, since they are usually not visible in different phases. While it would be great to achieve an accuracy overlapping of the vasculature in non-contras enhanced images an evaluation of the method based on a vascular system that is not visible difficult as objective metric.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The paper is interesting and the topic relevant. However, it has some major weaknesses regarding the feasibility of the method that was presented.

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #3

  • Please describe the contribution of the paper

    The authors agilely used existing segmentation tools based on convolutional networks and classical methods of non-rigid recording based on the deformation field of the control point grid to enable the transfer of information about the vascular tree and anatomical landmarks to intraoperative CT images without contract to improve the perception of the situation during needle-based liver intervention.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. Adaptation of well-known methods to the limitations of the clinical protocol.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Verification on a small amount of data
    2. Verification on animal data only
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    If, as the authors declare, the source code and data available in the publication will be made available, there will be a full possibility of repeating the results.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    Better justification and detailing:

    1. choice of U-net architecture
    2. the type of deformation field and the approach used in non-rigid registration would increase the publication impact.
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The problem raised by the authors is a current and important clinical topic.

    Existing challenges for the implementation of the method into clinical practice:

    1. Creating a deformation model for a specific patient - is it possible to model deformations well for a specific patient based on a limited set of data of his images
    2. Time needed to generate such a model (science)
    3. The time needed to use the model - if it is possible to use it during the procedure
  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper describes a deep learning network to augment a non-contrast CT with the vascular tree segmented from diagnostic CE-CT of the patient for liver ablation procedures. The reviewers agree that there is significant clinical value and could be potentially of great use to clinicians. However, there were several technical questions that were raised. The most significant of the questions was the validation for the dataset. The reviewers commented that there should be a better method to validate the algorithm’s results, especially since the vessels are not visible on the non-contrast CT images. Further, comparison with other registration frameworks based on the liver anatomy to evaluate the overlap of the vasculature would also be useful.



Author Feedback

Dear Area Chair, dear Reviewers,

Thank you for your valuable feedback on our manuscript. We appreciate the opportunity to address certain aspects of our method and results. We are pleased that the reviewers recognize the motivation, novelty, and organization of our paper.

In response to R1: Thank you for bringing the visualization point to our attention. We apologize for the mistake in Fig. 5. We want to clarify that there is no deformation between any of the three images, and the leftmost image was accidentally chosen from a different slice, leading to the misperception of deformation. Thus, we will correct this error in the final version of the paper.

Additionally, regarding Fig. 6 we agree that the deformation depicted appears more like a translation, but it is not the case. The main portal vein does not translate as expected in relation to the rest of the vascular tree. For the other subjects, the existence of a more complex (non-rigid) deformation appears more clearly. We propose to use another illustration for this figure to avoid any misunderstanding. In any case, the method itself was inspired by VoxelMorph, which is well known to be efficient at non-rigid registration. Similar to Voxelmorph, we optimize the parameters of our network using a training dataset including a non-rigid deformation between each pair of images. Therefore, our method is specifically designed to handle non-rigid scenarios.

Regarding the comparison with a rigid registration method, the VoxelMorph benchmark that we propose in the results section includes a first rigid registration step between the fixed and moving images (made with ElastiX). Thus, our method outperforms a simple rigid registration in terms of dice score.

About the validation metric, we acknowledge that using TRE on anatomical landmarks could provide additional insights regarding the quality of our results. However, we opted for DICE as it is a commonly used metric for segmentation problems, which somehow aligns the nature of our problem as well as the clinical impact of our solution.

In response to R2: Due to space limitations, we have focused on recent works in the state of the art section. However, we identified numerous interesting studies on registration methods. Regarding the deformation between phases of the same series, we would like to clarify that multiphase contrast-enhanced CT typically involves scanning the volume of interest at different time points during contrast injection. Specifically, for the liver, arterial and portal phases are often triggered after approximately 35 and 80 seconds, respectively [1]. Prior to the contrast injection, a setup non-contrasted phase is usually performed. It is important to note that both injected phase scans are commonly acquired during the same apnea period, minimizing motion and deformation. For the labeling, we select the root branch among all the branches that do not either have a child or a parent in a random directed graph, which reduces the risk of choosing the wrong branch as the main portal vein has a very high radius compared to the last order branches.

In response to R3: As mentioned above, our method was inspired by VoxelMorph that we have further improved and adapted to our problem by considering vessel maps. Regarding the type of deformation field, we currently apply random Gaussian displacement fields on the pre-operative phases to generate our training data for each subject. We have thought about using a patient specific biomechanical model in order to generate realistic deformations for training. However, such an approach is more time consuming when it comes to generating the training data.

[1] Ichikawa, T., Erturk, S. M. & Araki, T. Multiphasic contrast-enhanced multidetector-row CT of liver: Contrast-enhancement theory and practical scan protocol with a combination of fixed injection duration and patients’ body-weight-tailored dose of contrast material. European Journal of Radiology 58, 165–176 (20



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    Unfortunately, the paper still does not have sufficient novelty or validation for acceptance to MICCAI. The paper still suffers from significant technical deficiencies.



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    After carefully reading the paper and all reviews, including the meta-review, as well as the rebuttal provided by the authors, I agree that the clinical benefit would be potentially interesting and the paper provides proper justification with the relevance for percutaneous procedures. However I am sorry to say that I found to no technical novelty nor extensive validation for this paper. The method is basically using standing DA methods with a U-net to provide several CT scenarios, and the method is only trained/tested on 4 patients, and is clearly insufficient to have any significant conclusion of this work. This is clearly insufficient contribution and proper validation to support acceptance.



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The chief concerns identified during initial review were 1) algorithmic validation (to contend with the fact that vessels are not visible on non-contrast CT) and 2) comparison with other registration frameworks.

    The rebuttal punts concerns around 1), stating that Dice score is a standard metric and TRE would be useful but will not be considered here. This is a serious limitation, because it does not address the concern in any way. R1 did increase their score based on changes to an incorrect figure, however, this qualitative alteration does not affect the issue raised above. Further, regarding issue 2), authors refer to the fact that their framework uses initial rigid registration and builds on VoxelMorph, which then provides baselines that are outperformed. However, again, this does not adequately address the limitation that insufficient baselines were considered.

    Overall, from all the information available to me, this is a borderline paper.



back to top