Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Patryk Rygiel, Paweł Płuszka, Maciej Zięba, Tomasz Konopczyński

Abstract

Estimation of patient-specific hemodynamic features, and in particular fractional flow reserve (FFR) in coronary arteries is an essential step in providing personalized and accurate diagnosis of coronary artery disease (CAD). In recent years, in the domain of computed tomography angiography (CTA), a virtual FFR (vFFR) derived from coronary CTA using computational fluid dynamics (CFD), has been used as a compelling, non-invasive, in-silico replacement for invasive diagnostic techniques. Unfortunately, the time and computational demands of CFD are major obstacles to introducing vFFR from CT as a commonly used prophylactic tool. In this work, we propose a novel geometric-based artificial deep learning (DL) architecture, CenterlinePointNet++, which acts as a surrogate for CFD engines for the task of hemodynamic features estimation of the coronary arteries. Our architecture works directly on the vessel geometry represented as a surface point cloud and a centerline graph. As a result of that, it utilizes implicit geometry embedding without the need for hand-crafted features to estimate directly hemodynamic features. We evaluate our approach on the task of pressure drops and vFFR estimation for a synthetically generated dataset of coronary arteries and showcase significant improvement over commonly used geometry-based approaches.

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43990-2_73

SharedIt: https://rdcu.be/dnwMx

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #2

  • Please describe the contribution of the paper

    Authors propose a geometric-based artificial deep learning (DL) architecture, CenterlinePointNet++, for the task of hemodynamic features estimation of the coronary arteries.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    Interesting idea of using centerline graph for coronary artery pressure drop and vFFR estimation.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. The writing is unclear and confused,
    2. There is no comparison with other SOTA methods on vFFR estimation.
    3. Paper [11] have conducted pressure field estimation using a point cloud neural network. The following sentence is inaccurate. “To our best knowledge, this is the first method that is tackling the problem of pressure drops and vFFR estimation utilizing a point cloud neural network.”
  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Not easy to reproduce. A lot of important implementation details are not given.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    1. How is the centerline graph built? ‘t’ parameter set to ? “ The centerline graph, which can be extracted from the mesh with an algorithm of choice is denoted as C = (VC, EC) , where VC is the set of nodes and EC a set of undirected edges.” What algorithm? How to construct the edges?

    What’s more, paper [1] have constructed the centerline graph and proposed the topology-aware grouping method. What is the difference between them?

    [1]. Yao, L., Shi, F., Wang, S., Zhang, X., Xue, Z., Cao, X., Zhan, Y., Chen, L., Chen, Y., Song, B., et al.: Tag-net: Topology-aware graph network for centerline-based vessel labeling. IEEE Transactions on Medical Imaging (2023)

    1. After getting the point neighborhoods, how to conduct the features aggregation as the numbers of neighbors of different points (centroids) may be different. How to update the features of points in each layer. PointNet++ has multi-resolution setting, how to implement here?

    2. Which loss function? As the output of the network is per-point hemodynamic features of choice. Do you mean the pressure of each point?

    3. The correctness of the network is questionable. In Fig.1, the dimensions of features are wrong. After skip connection, they should be doubled. For example, 256 + 256 = 512.

    4. In Table 1, for 75th MAE under vFFR (100 mmHg), 0.50 using CPN, 5.73 using PN, Why has this value increased so much?

    Minor: Section 2.2: stands for “decoding. –> stands for “decoding”.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    3

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    1. The writing is unclear and confused,
    2. There is no comparison with other SOTA methods on vFFR estimation.
    3. Experiment result and the details of network designing.
  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #6

  • Please describe the contribution of the paper

    This paper presented to use the vessel surface point cloud and centerline graph to directly predict vFFR. It bypassed the tedious process of hand-crafted feature computation. The paper also proposed a new centerline grouping mechanism where the geodesic distance rather than the Euclidian distance was used in computing the neighbourhood info of the points. It was claim to better learns the sequential embedding of the complex vessel trees. Excellent agreement with CFD cFFR results were achieved on synthetic vessel trees.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    • The paper was well written and easy to understood.
    • I like the idea of embedding the centerline graph into the learning process to help grouping the points
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    • The authors did not disclose the parameters used for synthesizing their vessel tree. Not sure how diver is the gemory of the vessel tree
  • Please rate the clarity and organization of this paper

    Very Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The synthetic data cannot be accessed openly, and neither can the parameters used for the simulation.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

    • What’s the color map denotes in figure 3. I originally thought its the radius of the vessel but it doesnt look like so. Please clarify?

    • The first layer of the pointcloud network is downsizing the number of points from 20,000 to 2048, I’m wondering if the original ~200,000 surface points were necessary. Is there any effect of the density of the surface point cloud on the prediction results? On the same note, how possible is it to just use points along the centerline and use the coordinate + radius as input to the pointcloud network.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    6

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The idea is noval and the early results on simulated data looks very promising

  • Reviewer confidence

    Confident but not absolutely certain

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Review #7

  • Please describe the contribution of the paper

    This paper presented a novel geometric deep learning approach to estimate vFFR given point cloud and centerline coordinates of the coronary artery vessel tree. It adapted PointNet++ to incorporate centerline information to better incorporate the topological relationship among the vessel branches and demonstrated clear improvements in accuracy.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
    1. This paper addresses an important limitation of CFD-based FFR estimation - the computational time and proposed a geometric deep learning-based approach that can drastically reduce computational time for different vessel geometries, under the same boundary conditions.
    2. The paper improved PointNet++ by proposing a novel point grouping and abstraction method to consider centerline information. The method was clearly described, makes sense and the results demonstrated clear improvements.
  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
    1. Only geometric information was fed into the model and not the physiological information such as the boundary conditions. The training/evaluations were performed on the same fixed sets of boundary conditions with some discrete choices of values. However, the pressure or flow rate varies across different patients, and the proposed vFFR estimation method does not handle different boundary conditions.
    2. Only synthetic geometries were used in training and evaluations. Patient-specific geometries reconstructed from image data would be helpful to further verify the effectiveness of the approach.
    3. There was no mention of outlet boundary conditions and how varying that would affect the results.
  • Please rate the clarity and organization of this paper

    Excellent

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Methods were clearly described and code will be released. The paper seems to be reproducible.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
    1. Only geometric information was the input to the model and thus cannot predict FFR estimation under different boundary conditions. I think this limitation needs to be clarified in the paper.
    2. I’d like to invite the authors to evaluate their method on a set of realistic patient geometries. It would be interesting to see the performance of ML estimation of FFR.
    3. Implications of MAE on clinical decisions were not discussed. What is the percentage of test cases that had wrong predictions of the intervention/no intervention as a result of the FFR prediction errors?
    4. What boundary conditions were applied at the outlet?
  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    7

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
    1. A novel geometric deep learning method for vFFR estimation that greatly reduced computational cost compared to CFD.
    2. Novel network design that adapt PointNet++ to handle vessel geometries by incorporating centerline information
    3. Very well written, method and results are convincing.
  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    N/A

  • [Post rebuttal] Please justify your decision

    N/A



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    The paper presents a geometric-based artificial deep learning (DL) architecture, CenterlinePointNet++, for the task of hemodynamic features estimation of the coronary arteries. This is an extension of previous work (point-net++). There is a discrepancy between reviewers, while 2 were in favor of the paper, the third reviewer raised serious concerns about novelty compared to a previously published paper and quality of evaluation as it includes only synthetic data and comparison to point-net++ rater to other more common and CFD based methods. The authors should address these issues in their rebuttal.



Author Feedback

We thank all reviewers for their useful suggestions and comments. Given the strict author feedback limitations, we address the most critical remarks highlighted by the metareview first, then the remaining comments.

NOVELTY Thank you for drawing our attention to the work by Yao et al.[1]. We’ll revise the related work section and cite accordingly. However, our problem is fundamentally different. [1] performs the task of anatomical labeling of head and neck vessels based solely on centerlines. We’re working on the estimation of hemodynamic features of a coronary artery (CA) geometry and utilize both the centerline as well as the surface of the geometry during reasoning. The idea of encoding the surface point cloud of a CA is crucial in our work. On top of that, although [1] uses a centerline as we do in their grouping strategy, it is again, a different method compared to our Centerline Grouping Strategy. We propose to group points utilizing geodesic distance along the centerline, while [1] groups points that are topologically connected within the Euclidean distance. Given the above, we want to highlight that it is not feasible to use their system to solve our problem without major changes in their architecture and the novelty of our work compared to [1] is undeniable.

OTHER METHODS As said in Section 3, it is not possible to reproduce the only two relevant works known to the authors [8,24]. Even though we did try to reproduce their results, we were unable to do so. These works do not provide necessary details of their hand-crafted features, and their software is not available. [8] refused directly to share the details. [20] reports the same problem. We’d like to highlight that we do report comparison to both a PointNet++ and a pure CFD method.

SYNTHETIC DATA We were limited to experimenting with synthetic data only. We do plan in the future to perform experiments on real data.

OTHERS R2 t param: The values of grouping threshold t for all encoder blocks are provided in the supp. material in Table 1 as D_1 and D_2. R2 f aggregation: The feature aggregation is performed by PointNet (Fig.2a). It takes any number of input points and yields aggregated 1D feature vector. There is no need for additional tensor dimension normalization at this point. The multi-resolution setting is performed exactly as in PointNet++. R2 loss f: We use MSE for each point between the GT and prediction. The loss is an average across all point errors. We will revise the method section and add this information. R2 f dimensions: The dimensions in Fig.1 denote not input but output feature dimensions. The decoder block may take 256 + 256 features and output 256 features. R2 typo: it should be 0.5. We will correct that. R6 color map: The color map in Fig 3 denotes the geodesic distance along the geometry from the point P_i. The figure will be revised to include the legend. R6 cloud size: The input vessel geometry has 200k surface points, due to the CFD simulation needs. As pointed out, we find that for our method, we require only about 20k points. It is a tradeoff between surface details and network performance. Regarding the side note – as said in section 3.1, coordinates and radius, together with the geodesic distance from the inlet are features of surface points. Centerline points consists of coordinates only. R7 BC: Our approach cannot estimate FFR under different boundary conditions (BC) - for each set of BC, a new model must be trained. We’ll revise the conclusions section to add this information. The BC at the outlets were selected in alignment with standard recommendations for fixed inlet pressure cases. The velocity field at the outlets was treated with the Dirichlet boundary condition. For outlet pressure, the Neumann boundary condition was applied. R7 Clinical MAE: That’s a good suggestion. Setting the threshold of 0.8 FFR as clinically accepted, we report accuracy of 94.11% for PN, and 95.58% for our method. We will add this to camera-ready.



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    Authors have addressed several concerns raised by the reviewers. However, the exclusive use of synthetic data in the conducted experiments somewhat diminishes the overall enthusiasm. Nevertheless, the proposed method is intriguing and holds potential interest for the MICCAI audience.



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    I have read the comments and rebuttal. This paper is about a deep-learning method for hemodynamic feature estimation of coronary artery geometry. The differences between the proposed work and the work by Yao et al have been given and addressed in the rebuttal. The concern about the comparison with other SOTA methods has been discussed in the rebuttal.



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    R2 rejected this paper for reasons including lack of advanced comparison experiments and unclear details. In the rebuttal, the authors explain why the methods of [8, 24] are not reproducible, while showing that comparisons with PointNet++ and CFD methods are indeed reported. At the same time, the authors clarify the differences between their work and [1]. The authors provided a detailed response to the issues raised by R2. Regarding the weaknesses pointed out by R2, the other two reviewers appear to consider that the authors have handled them better. Although R2 did not improve their score, we think that the novelty of this work should not be overlooked and we decided to accept the paper.



back to top