List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Ardit Ramadani, Peter Ewert, Heribert Schunkert, Nassir Navab
Abstract
Accurate catheter tracking is crucial during minimally invasive endovascular procedures (MIEP), and electromagnetic (EM) tracking is a widely used technology that serves this purpose. However, registration between preoperative images and the EM tracking system is often challenging. Existing registration methods typically require manual interactions, which can be time-consuming, increase the risk of errors and change the procedural workflow. Although several registration methods are available for catheter tracking, such as marker-based and path-based approaches, their limitations can impact the accuracy of the resulting tracking solution, consequently, the outcome of the medical procedure.
This paper introduces a novel automated catheter registration method for EM-guided MIEP. The method utilizes 3D signal temporal analysis, such as Dynamic Time Warping (DTW) algorithms, to improve registration accuracy and reliability compared to existing methods. DTW can accurately warp and match EM-tracked paths to the vessel’s centerline, making it particularly suitable for registration. The introduced registration method is evaluated for accuracy in a vascular phantom using a marker-based registration as the ground truth. The results indicate that the DTW method yields accurate and reliable registration outcomes, with a mean error of 2.22mm. The introduced registration method presents several advantages over state-of-the-art methods, such as high registration accuracy, no initialization required, and increased automation.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_75
SharedIt: https://rdcu.be/dnwMz
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors consider the problem of accurate catheter tracking during minimally invasive endovascular procedures. Specifically, they address the problem of aligning EM tracking locations to pre-operative imaging by introducing a dynamic time warping (DTW) algorithm in this context.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The problem space, clinical relevance, and limitations of existing techniques are well described. The overall solution that they propose is compatible with existing hardware (e.g. it uses existing EM sensors and the acquisition software & algorithms are implemented standard laptop computers). It appears to be compatible with clinical workflow as well, which would be beneficial from the standpoint of clinical translation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
Evaluation is limited to a single vascular phantom
-
Deformation of the phantom between pre-operative and intraoperative conditions does not appear to be addressed
-
Conditions where the algorithm is likely to be inaccurate (perhaps these would include vessels where there are multiple solutions yielded by time-warping, e.g. largely straight vessels?) are not considered
-
The data acquired look during experiments look promising but they do not appear to be sufficient to justify the claim that it has “reliable registration” (Conclusion section)
-
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Moderate reproducibility; the data are not made available but the algorithms and hardware are well described.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
More detail on fundamental details of the DTW algorithm (e.g. its parametrisation or more generally, what flexibility there is with warping) would be very useful
-
Please comment on cases in which the DTW is likely to be under-constrained, e.g. where different time warps lead to very similar solutions. What happens in largely straight vessel segments, for instance?
-
Please qualify your statements on accuracy and robustness for this technique, given the limitations of a single dataset
-
Please briefly comment on the extent to which there is likely to be deformations in vasculature between pre-op and intra-op images (e.g. in which clinical contexts and how severe these are likely to be
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presents a topic of interest to the MICCAI community and method and results are very promising. However, given the standards for a MICCAI paper, the very limited test data and limited analysis of those data point led to a “weak accept” recommendation.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper describes a method to perform the rigid registration between an EM tracking system and the patient image space for vessel catheterization. The registration relies on the alignment of the EM tracker path with the vessel centerlines. Dynamic time warping is proposed to filter out false point correspondences. Correct correspondences are used with the Coherent Point Drift algorithm to solve the registration. The method was evaluated on phantom data to provide a mean 2.2 mm accuracy, slightly better than ICP (2.86mm).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- paper is easy to understand, well structured
- problem of actual clinical interest
- leveraging DTW is interesting, even if immediate issues are not addressed (see below)
- evaluation on phantom is promising, with a solid ground truth
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- the clinical application is not obvious: the targeted clinically acceptable range of error is 5 mm, which can only be related to abdominal aorta; the reported mean error is 2.2 mm, which makes it applicable to kidney of liver, and closer to being applicable to the heart, but in that case there is motion involved. Motion will rapidly be an issue and is not discussed here.
- the method currently relies on the catheter being pushed: no rotation or backward motions are allowed, which is unworkable.
- the evaluation is only made on phantom, and the vessel centerlines are extracted from an STL model: in practice, this centerline will be tainted with errors and noise. This should have been included in the evaluation.
- the comparison with ICP is not fair… for the authors’ method, since ICP is reported to fail a lot. This should be part of the results. On the other hand, initializing ICP with the result of the authors’ method would not allow ICP to escape a poor local minimum in case the authors’ method produces poor result, thereby minimizing ICP performance.
- the method should be compared to other methods like RANSAC, or Robust ICP
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Though no code is publicly available, a motivated and informed student could reimplement it in a reasonable time. The phantom used is based on a publicly available data set (Johns Hopkins University Data archive).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The method is promising but is too premature for publication in my opinion. Questions about: anatomy motion, catheter motion (backward shift and rotation) appear very soon when reading the paper, and are only, and partly, discussed at the end. The catheter motion in particular should be investigated further: in the experiments, the catheter is pulled, but in clinical practice, it is mainly pushed, implying contacts between the tip (where the EM sensor is) and the vessel surface (hence, not on the centerline), with bumps due to friction, etc. These issues are not addressed and will have a negative impact on the registration error. Moreover, the registration error is minimized in my opinion, due to two things. First, the figure of merit in Eq 1 systematically considers the closest point on the centerline. Second, the error is only reported on the vessel centerline, which consists in the anchor for the registration. What would happen when the catheter is moved into another vessel? Should the registration be continuously performed? This seems inconsistent with medical practice. The error should be computed on the whole ROI and not only on the vessel centerline used for registration. The error should also be reported, depending on the tracked length. The only unclear part is on p. 5: How are the three segments selected? How do the authors explain that standard deviation is much lower with ICP? Are the differences always significant?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Promising method, but in a too early stage to be published. Experiments closer to clinical reality should be performed, in particular the forward/backward motion of the catheter should be handled.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
The authors’ rebuttal offers some insight on the robusness of the algorithm (behavior at bifurcations, performance in case the catheter is pushed instead of pulled). But no further results are to be included to support their claims. I also still think there is a fundamental flaw related to the catheter motion: DTW aligns two temporal signals; the authors consider the 3D centerline as one signal; but its temporality goes in a single direction; I reckon this makes the method work when the catheter is either pulled or pushed (still unclear about the accuracy when pushing); but I still do not believe it can handle natural motion, i.e. when the catheter was both pulled and pushed during EM signal recording for the registration. As a consequence, the clinical application would require the interventionalist to register a specific catheter motion (e.g. pull for 1 second) to perform/update the registration. But this does not comply with any motion or deformation after the recording was made. I would keep my reject recommendation, but I also acknowledge that my fellow reviewer have a more positive opinion on the paper, adding to the authors’ supplementary insight in the rebuttal. Thereby I raise my recommendation to weak reject.
Review #5
- Please describe the contribution of the paper
A novel Dynamic Time Warping-based registration method for minimally invasive endovascular procedures was proposed, with high registration accuracy, no initialization required, and increased automation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The introduction of Dynamic Time Warping to the registration of centerline and catheter path is interesting and novel. It does not relay on the the initialization procedure and can shorten the pre-operative preparation time consumption.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some procedures in the phantom experiment may not comply with clinical operating workflows. For example, it is easy to guide the catheter along the vessels due to the phantom are transparent. So, the catheter path is easy to collected to perform the registration. However, in clinical practice, we often relay on the registration matrix between the EM system and the preoperative image to guide the catheter’s movement in the surgical navigation system. How can we collect the catheter path in clinical practice?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of the paper is satisfactory.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
This manuscript describes a novel Dynamic Time Warping-based registration method for vessel’s centerline and catheter path, which can improve the registration accuracy, optimize the workflow, and increase automation in minimally invasive endovascular procedures.
Major strengths: 1) The introduction of Dynamic Time Warping to the registration of centerline and catheter path is interesting and novel, and experimental results demonstrate its feasibility and accuracy performance. 2) The manuscript is well written and easy to follow, and the discussion part is profound.
Major weaknesses: 1) The clinical application potential of this work is not clear. All experiments are conducted on a transparent phantom, which is different from the clinical setup. 2) The time efficiency of the proposed work is not clear.
Suggestion: 1) The author could discuss how to integrate the proposed registration method in clinical practice, where the vessels are not transparent. 2) The author could discuss the source of the final registration error. The EM tracking error could not be ignored.
In summary, the manuscript is a good and novel. I suggest an accept of this manuscript.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty of the method.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
I maintain my original decision
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper describes a new method to register the EM sensor readings to the preoperative imaging using the Dynamic Time Warping algorithm followed by the CPD registration. Although interesting, the validation of the algorithm seems very weak. The validation was conducted on a single rigid phantom. Further, deformations in the vascular tree intraoperatively compared to the segmented centerline have not been considered. It is also not clear how pulsatile blood flow, back-and-forth motion of the catheter, etc. influence the registration accuracy, as pointed out by the reviewers. Further, it was also mentioned in the paper that, “The DTW algorithm registration process assumes that the orientation of the phantom (patient) and the preoperative model are similar.” , which is never the case intraoperatively. An initialization procedure would be required to align the phantom and the preoperative model, which would no longer satisfy the claim in the paper that “no initialization is required”. Although the paper has good potential, it appears that there are a few loopholes. I would urge the authors to address the reviewers’ comments in the rebuttal phase.
Author Feedback
We highly appreciate reviewers’ feedback and are especially pleased with the recognition of the novelty of our approach. We have categorized the feedback into three research directions (Phantom/Evaluation, Motion/Deformations, and DTW algorithm) and addressed them in a combined response. We want to emphasize that all data, including code will be made public as well.
(R1, R3, R5) We consider that a rigid phantom selection with features resembling real anatomy is essential to ensure reproducibility in future studies. The rigidity and transparency of the phantom serve two purposes: accurate ground-truth comparison and visual feedback during catheterization. We believe that our data model using this phantom provides a reasonable representation for evaluation in line with the state-of-the-art. We believe that, overall, the results indicate a positive trend where DTW demonstrates good accuracy in registration within the scope of these experiments and possibly beyond.
(R1, R3, R5) Deformations and influence of blood flow motion would impact all registration algorithms equally, potentially leading to increased errors. In endovascular procedure, including cardiac/aortic, kidneys, liver, or peripheral arteries, the motion compensation would range from sub millimeters up to few millimeters. (R3) However, the focus of this work is the registration algorithm itself, and we did not explore motion compensation or deformation studies per se, as we consider them as next logical steps. The nature of DTW itself offers robustness in signal deformations and movement, hence, an advantage in projecting the pose of the catheter tip onto the pre-op images. However, the accuracy of this projection may vary depending on the extent of motion/deformation and density of the vasculature tree.
(R3) We agree and recognize the difference between the path covered while pushing or pulling the catheter, and acknowledge that the centerline does not accurately depict either of these covered paths. We addressed these differences in our paper. We opted for pulling the catheter in our experiments to ensure consistency, reproducibility, and meaningful validation. Nevertheless, the method is able to register successfully in both catheter movement directions.
(R1, R3) The DTW algorithm aligns two signals by stretching them and minimizing the sum of Euclidean distances between corresponding points. To ensure representation from the entire signals and avoid local minima, we divide the signals into three equal segments and select the correspondences with smallest Euclidean distance from each segment as representative points for registration. In cases when the catheter is moved into another vessel branch, the DTW algorithm would automatically warp the signal accordingly to the correct centerline.
(R1, R3) The registration results and standard deviation of our method are primarily influenced by the outcomes of branch 4 and 5. This is because the catheter path in these branches is mostly straight, causing the DTW feature points to align without triangulation. Consequently, the registration of these points yields less accurate alignment. We could briefly discuss this in the camera-ready version.
(R1, R3, R5) Finally, as this novel concept is presented and tested for the first time, we find the current results valuable and appropriate. We acknowledge that additional tests are necessary to ensure its clinical acceptance and the move towards translation. However, we believe that this MICCAI paper could open the path for the research community to use this novel concept and explore further its capabilities.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have adequately addressed the reviewers’ comments. There is sufficient value in the paper for acceptance to MICCAI.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper presents an automated catheter registration method for EM-guided procedures using temporal analysis of 3D EM signals and DTW for registration. The paper is well-written, the clinical motivation and problem space is well-placed, and the proposed approach is novel and promising. The topic is of interest to the community. The rebuttal provides some insight on the main concerns and criticism regarding the limited evaluation (on a single phantom), limited data and scope of experiments/analysis, clinical relevance of the validation experiments (e.g. clinical scenarios such as motion and deformation not being considered), and limitations of the approach with regards to catheter motion.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Although the paper has good potential, it appears that there are a few loopholes, and it cannot be accepted for publication in its current form. Reviewers raise several concerns regarding the evaluation and practicality of the proposed method. The evaluation is limited to a single vascular phantom, and the deformation of the phantom between pre-operative and intra-operative conditions is not addressed. The conditions where the algorithm might be inaccurate are not considered, particularly in cases with multiple solutions yielded by time-warping. The data acquired during experiments are deemed promising but insufficient to justify the claim of “reliable registration,” especially considering the range of acceptable error and the limitations related to motion. The evaluation on phantoms using a centerline extracted from an STL model is deemed unrealistic compared to the potential errors and noise in real-world scenarios. In addition, the comparison with the ICP method is deemed unfair, and alternative methods like RANSAC or Robust ICP should have been included.