List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Baihong Xie, Xiujian Liu, Heye Zhang, Chenchu Xu, Tieyong Zeng, Yixuan Yuan, Guang Yang, Zhifan Gao
Abstract
The assessment of fractional flow reserve (FFR) is significant for diagnosing coronary artery disease and determining the patients and lesions in need of revascularization. Deep learning has become a promising approach for the assessment of FFR, due to its high computation efficiency in contrast to computational fluid dynamics. However, it suffers from the lack of appropriate priors. The current study only considers adding priors into the loss function, which is insufficient to learn features having strong relationships with the boundary conditions. In this paper, we propose a conditional physics-informed graph neural network (CPGNN) for FFR assessment under the morphology and boundary condition information. Specially, CPGNN adds morphology and boundary conditions into inputs to learn the conditioned features and penalizes the residual of physical equations and the boundary condition in the loss function. Additionally, CPGNN consists of a multi-scale graph fusion module (MSGF) and a physics-informed loss. MSGF is to generate the features constrained by the coronary topology and better represent the different-range dependence. The physics-informed loss uses the finite difference method to calculate the residuals of physical equations. Our CPGNN is evaluated over 183 real-world coronary observed from 143 X-ray and 40 CT angiography. The FFR values of CPGNN correlate well with FFR measurements r=0.89 in X-ray and r=0.88 in CT.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_11
SharedIt: https://rdcu.be/dnwLl
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #5
- Please describe the contribution of the paper
This manuscript presents a novel deep-learning-based system for fractional flow reserve (FFR) estimation. Specifically, it combines the capability of graph convolutional networks to exploit the topology of vessel structures with physics-informed learning, which incorporates the known physics laws and that explicitly uses the boundary conditions by adding them to the input of the network. The method was trained on over 6k synthetic coronary geometries with the ground truth computed by a computational fluid dynamics (CFD) solver . The proposed network was then compared against other graph convolutional layer types on unseen synthetic data. Finally, the method was compared against CFD simulations by applying both technologies to clinical data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The presented method is interesting and novel since it successfully demonstrates the power of choosing the right inductive biases, i.e., the explicit integration of the topology & boundary conditions via graph neural networks and the network regularization via a physics-derived loss term. Despite training on synthetic data, the results suggest that it is able to generalize well to real-world data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weaknesses of the paper are:
- several spelling and grammar issues, which make it difficult to follow the method description and the results
- some details are missing
- the method is only compared against graph convolutional networks, but it would be particularly interesting to see how they perform against standard ML or DL methods
- a proper discussion of the results is missing
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The manuscript presents most of the details to reproduce the results, but the text is often difficult to follow and leaves room for guessing.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The proposed physics-informed graph neural network is very interesting to me and I believe that the idea and the results are of merit to the MICCAI community. However, there are several concerns and questions that could greatly improve the manuscript:
- Since there are several spelling and grammar errors, which make it difficult to follow the text, I would kindly ask the authors to take some time to revise the wording.
- p.2, second to last sentence: Could the authors please clarify what they mean by “the finite difference method uses more inputs information to approximate the derivatives”? From what I understand, the FDM is necessary since one cannot use automatic differentation as in standard PINNs due to the one-dimensional coordinate z.
- Section 2.1, Problem Statement: From my point of view, the description is written to generically and I would strongly advise to show the actual hemodynamics equations as in the supplementary material. This would greatly help in understanding the different loss terms and the general idea of physics-informed learning.
- Please define all symbols, e.g., w in Eq. 2 and C for the dimensionality of the morphology features is missing.
- Fig. 1 / Description of CPGNN: Could the authors please explain why the conditional features for the flow and the pressure are computed at different resolutions and why Hp uses summation and Hq uses concatenation? What kind of graph convolutional layer is used for the MSGF?
- Conditional Feature Decoding: The manuscript states that 1D convolutions are used, but the supplementary material states a 3x3 kernel. Could the authors please clarify what was implemented? Could the authors please also explain how the convolutions are applied? Is it branch by branch?
- Fig. 2: Could the authors please also show in the supplementary material a visualization of the difference between GT and each prediction?
- Comparison to other graph convolutional networks: Could the authors please list the details for these architectures?
- Could the authors please comment on how the proposed method would compare against other ML and DL methods that do not rely on graph convolutions?
- Table 2: What are the graph numbers S1, S2, and S3 refering to?
- Could the authors please share some insights into how it is possible that the proposed method is achieving a higher correlation and AUC than the CFD solver on the clinical data considering that the solver is explicitly solving the same equations?
- Please add a proper discussion of the results and the limitations of this work.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Even though there are several questions and concerns, the method is novel and demonstrates the strength of coupling graph convolutional networks with physics-informed learning. It should therefore be of great interest to the computational modeling community.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper proposes a GNN-based architecture with a physics-informe loss to estimate the FFR from ] images. They train in a synthetic dataset compare it to the SOA, showing better performance in the synthetic and real imaging dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors address an interesting problem with recent advances on physics-informed neural networks. They do a comprehensive comparison with the state of the art and an ablation study.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The main weakness of the paper is that the physical equation, the data and models are not clearly described; making it difficult to analyse the results.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors provide a general structure of their model.They do not report hyperparameters nor the exact architecture they use. but the description of the in-vivo is missing (I assume they are private datasets).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
My main concern is the lack of description of the physical equations. They are somewhat described in the supplementary material, but there are important details missing.
1) From the equations (11) , there are no finite differences. Please check there is no a typo. 2) It is not clearly stated that equations do not include viscosity. If there isn’t, what is the value of the interior nodes of the non-stenotic coronaries, since momentum will be exactly preserved. 3) Also, how are the material properties of the vessel approximated/defined, relating area with pressure? 4) Is the FFR on the clinical datasets derived from a computational model, or uses invasive pressure measurements? If it is the first, specify the specific model/clinical product used. 5) Also, it would be beneficial to see the computation time of both the CFD and DL based approaches to better quantify the benefit.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper’s results are relevant to clinical practice, and seem solid. My concerns can be easily solved by improving the description of the physical model.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
The paper presents a model of blood flow in coronary arteries based on physics informed neural networks. The authors describe a few novel contributions to improve the performance of the method. On the one hand, they propose to leverage graph convolutional neural networks as the core architecture to build predictive features implicitly incorporating the topology of the arterial network. On the other hand, the input features include not only centerline information (as coordinates of the centerline points) but also cross-sectional area and boundary conditions. This helps the network produce solutions that are in particular consistent with boundary conditions. The method is applied to the prediction of FFR and its accuracy in this task is evaluated by comparing against simulated FFR in synthetically generated geometries as well as invasively measured FFR in patient-specific data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The topic of efficient and accurate estimation of FFR from geometry information of the coronary arteries is important. The use of advanced deep learning techniques is of interest because it has the potential to enable faster computation and potentially simpler workflows. The modeling choices proposed in the paper are presented clearly and well justified. The results are interesting and promising. The evaluation is conducted on a large set of testing data, including both synthetically generated and based on patient data. A direct comparison with multiple alternative methods is also presented.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I think it would be useful to discuss in more detail the practical implementation of the proposed approach. How would data preparation differ from that required for a traditional computational modeling method such as Simvascular? How does the runtime compare with that of a traditional computational modeling approach? Given the small size of the computational model and the fact that the modeling assumptions seem to imply that the coronary tree is represented as a 1D structure, computational modeling techniques would be competitive in terms of runtime. Beyond the technical novelties required to bring the performance of a PINN-based method on par with that of the comparator (computational modeling), the paper lacks the discussion of the implications for clinical practice.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The method description is fairly clear and complete. I found it not very clear how the Hq and Hp features are obtained (how does the “element-wise fusion” work?).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
For the results in fig. 2, it would be useful to explain how the input features look like for all the comparators. Is conditional feature encoding used with the other network architectures as well?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I found the approach interesting, with clever technical solutions and compelling results (in particular the ability of the model to generalize well to unseen geometries). I think the paper should be presented and discussed at the conference. I believe a discussion of the practical utility of the proposed approach should be included (how does it compare in terms of complexity / requirements / runtime versus alternative approached, in view of potential deployment scenarios in clinical practice.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper presents a physics-informed neural network approach for estimating fractional flow reserve from coronary tree geometry. The evaluation demonstrated the method applied to both CT images or X-ray angiography images. All reviewers agreed on an overall high ranking for the paper. Please address the minor comments by the reviewer in the final submission.
Author Feedback
We sincerely thank AC and all Reviewers for their constructive comments The meta-reviewer praises “all reviewers agreed on an overall high ranking” R1 praises “clever technical solutions” R3 praises that results are “seem solid” R5 praises “of great interest”
1.Physical equation (1) For unclear description of finite differences(FD) For R5, we agree “the FDM is necessary” and will delete the second-to-last sentence in p.2. Automatic differentiation cannot use to compute the spatial derivative of cross-sectional area as in standard PINNs, due to the spatial coordinate and cross-sectional area both are the inputs of the network For R3, the Eq.3 of the supplementary material approximates for first-order derivatives using FD (2) We will adopt the suggestion by R5 to “show the actual hemodynamics equations in the supplementary material” (3) R3 question ”equations do not include viscosity” The equations include viscosity in the coefficient C in Eq.5 (4) R5 question missing symbols definition w define network parameters and a clear definition of C will be added (5) R3 question missing material properties The vessel wall is rigid and blood is Newtonian fluid
2.Model design (1) R1 asks how “Hq and Hp are obtained” and “how does element-wise fusion work” Hq is obtained by concatenating all node features from different graphs. Hp is obtained by two steps. First, we upsample all graphs to match the node size of the largest graph. Second, we element-wise fuse the graphs by summing the features at the same position
(2) R5 asks why the conditional features at different resolutions and why “Hp uses summation and Hq uses concatenation” Flow is constant on branch due to mass conservation under steady-state modeling. Pressure varies according to the position. This lead to different fusion way for the conditional features Hq and Hp, and thus at different resolution. We use concatenation for Hq because it is a common way to fuse the features. Hp need to keep position info for pressure prediction. We use element-wise summation for Hp because concatenation cost larger memory (3) R5 asks “what was implemented” for Decoding The decoding uses a 1D convolution network and shares the same weight for each branch. The convolutions apply for the branch feature which are viewed as a batch and decoded in parallel
3.Experiment (1)R3 question missing clinic product info The FFR was performed using pressure guide-wire by the manufacturer Abbott with model HI-TORQUE (2) R1 R5 ask for the setting of comparison networks Conditional feature encoding are also used for all comparison methods. We will list their architectures and show visualization of the difference of results in the SM
(3) For R5, S1, S2 and S3 represent the number of graphs are1, 2 and 3 in the MSGF respectively
4.Discussion
(1) R1 asks “how would data preparation differ” from CFD The proposed method directly predicts the pressure from condition inputs. CFD require a stenosis detection algorithm on radius sequence to determine the equation coefficient
(2) R5 asks for insights about how it is possible to achieve a higher correlation Coupling stenosis detection algorithm may cause error accumulation when using a CFD solver. Since the stenosis features are synthetic during training, the loss of the equation with well-determined coefficient may guide network to catch the correct stenosis features. Therefore, the proposed method enable automatic stenosis feature extraction which avoids error accumulation and achieve good performance.
(3) R5 asks how about “against standard ML or DL”. The traditional ML is subject to handcraft features which is difficult to represent the complex coronary topology. FNN-based method lacks topology constraints. LSTM- and RNN-based method can model the sequence in single direction but may fail to learn the relation between branch because the flow direction changes
(4) R1 R3 ask for the runtime comparison. The computation speed is 0.03s per case(vs SimVascular 20s)