List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Paula Feldman, Miguel Fainstein, Viviana Siless, Claudio Delrieux, Emmanuel Iarussi
Abstract
We present a data-driven generative framework for synthesizing blood vessel 3D geometry. This is a challenging task due to the complexity of vascular systems, which are highly variating in shape, size, and structure. Existing model-based methods provide some degree of control and variation in the structures produced, but fail to capture the diversity of actual anatomical data. We developed VesselVAE, a recursive variational Neural Network that fully exploits the hierarchical organization of the vessel and learns a low-dimensional manifold encoding branch connectivity along with geometry features describing the target surface. After training, the VesselVAE latent space can be sampled to generate new vessel geometries. To the best of our knowledge, this work is the first to utilize this technique for synthesizing blood vessels. We achieve similarities of synthetic and real data for radius (.97), length (.95), and tortuosity (.96). By leveraging the power of deep neural networks, we generate 3D models of blood vessels that are both accurate and diverse, which is crucial for medical and surgical training, hemodynamic simulations, and many other purposes.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43907-0_7
SharedIt: https://rdcu.be/dnwb5
Link to the code repository
https://github.com/LIA-DiTella/VesselVAE
Link to the dataset(s)
https://github.com/intra3d2019/IntrA
Reviews
Review #1
- Please describe the contribution of the paper
- Propose a 3D blood vessel generative framework which first utilizes Recursive Variational Autoencoders to generate vessel geometries.
- Synthetize multi-branch blood vessel trees by learning the hierarchical organization and geometry features.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The first to utilize Recursive Variational Autoencoders to generate 3D multi-branch blood vessels.
- Efficient encoding and decoding of vessel structures using binary tree. The binary tree representation enables recursively multi-branch vessel generation.
- Detailed experimental setup description, including centerline sample and radius, class weight scale, child node weighting scheme, et al.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The statement of “Network Architecture” and “Objective” in Section “Methods” is not clear enough. The organization for the input, output and components of encoder, decoder and three shallow fully-connected neural networks is a bit messy. And it seems “a regression network” in “Objective” has no obvious correspondence in the figure and formula.
- Insufficient application feasibility. As the authors point out, trees with higher depth and non-binary bifurcations or loops, were excluded from their study. And the experiment was conducted on only one dataset.
- The quantitative comparison with SOTA methods is missed.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Generally meeting the requirements.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Organize the description of encoder, decoder and three shallow fully-connected neural networks in order and use “emphasis” to highlight important statements.
- Improve the network structure or add processing method to solve potential abnormal conditions like self-intersections.
- Add the quantitative comparison with SOTA methods.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novel application for Variational Autoencoders in 3D blood vessel synthesis and the recursive framework to generate multi-branch vessel geometries using binary tree representation.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper proposed a novel data-driven generative model for synthesizing 3D blood vessel geometry. Training data of 3D blood vessel meshes are first converted into a binary tree representation, then the proposed RNN-based VeselVAE framework is trained to learn the hierarchical tree representations for the blood vessel topology and structure. 3D meshes are synthesized later given the generated tree-structured centerlines. Both qualitative and quantitative experiment results look very convincing that the proposed method is able to generate realistic and diverse 3D blood vessel geometry.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
(1) This paper investigated the opportunities of utilizing the deep generative model for learning and synthesizing blood vessel geometry, and well-motivated these contributions of generating more realistic and diverse results compared to existing rule-based methods. The proposed VesselVAE was well-specifically designed for the proposed problems of generating blood vessels by learning the joint distribution of latent variables and the tree representations of blood vessels, such that the model can later recursively generate the blood vessel structure. Personally, I really enjoy how the authors approach the problems by (1) pre-processing to get the tree representations of the original blood vessel data, (2) learning a deep generative model on this simplified yet well-representative trees, and (3) finally generating the structure of blood vessel and then post-processing to add mesh details. (2) Overall this paper is well-written and organized, and easy to follow. (3) The experiment results look promising. Similarity metrics are presented to evaluate the real and synthesized samples. Qualitative results clearly showed that the proposed method can generate more realistic blood vessels compared to baselines, especially the overall structure of blood vessels (such as their curves, orientations, and growing patterns).
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) My major concern is how sensitive the VesselVAE is to the pre-proposed tree representations. Does it require very detailed and “high resolutions” tree representations (especially for the curving and/or bend segments of blood vessels) to get good results when training VesselVAE? Did the authors have any evaluations or expectations of how the proposed framework performs on more complex blood structures like capillaries, where the detailed tree sturecture can be difficult to build? (2) I might miss it, but what is the exact metric for evaluating the diversity of the generated vessels, as the authors claimed in “By using these metrics, we can determine….., as well as the diversity of the generated output.” (page 6)? The metrics in Fig 3 look like for similarities only. Additionally, Similarity metrics of the baseline methods can be added as well to strengthen the evaluations.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Looks no problem
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
See section 6. Additionlly, besides similarity and diversity evaluations presented in this paper, some readers may be interested in the performance of downstream tasks using the proposed framework. The authors are encouraged to consider conducting user studies for clinicians, and/or experiments of downstream supervised tasks to further demonstrate the usefulness of the synthesized blood vessels from the proposed framework in the future.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method is very novel, promising, and well-designed for the specific research problem that this work want to tackle. Good writing plus convincing experiment results.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
Thanks the author’s response that solved my concerns.
Review #3
- Please describe the contribution of the paper
The authors proposed a 3D blood vessel synthesis method, where the autoencoder was utilized to generate the vessel skeleton with radius, then a 3D mesh synthesis method was applied for vessel 3D mesh generation. In practice, the vessel skeleton was represented as the graph and the recursive variational Neural Network was utilized to capture the vessel connection in a latent space.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1.The authors proposed a novelty method for 3D vessel synthesis, where the connection and radius of node in skeleton was learned with recursive variational Neural Network
- The result show that their result is more accuraucy and the generated structure is smothly
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
In their statement, the learned latent space can be utilized to generate new vessel geometries, but there is no result related to this contribution. Because the input and the output of the autoencoder are the same, and the final 3D mesh is generated from the pedicted skeleton, it can be doubt that the autoencoder is an identity mapping if there is no other evidence for the learned latent space.
-
the proposed model structure is not introduced clearly, especially the encoder and decoder structure.
-
the input size is not clear, since the length of vessel structures are various. It not clear how many node was utilized for each sample
-
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
there are some problems to reproduce this paper since some details are missed and also the structure is not introdeced clearly
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
It could be better to introduced the method with more detail to make it easier for readers follow
-
about the result, showing some sampled new vessel geometries might be better to show the powerful of the learned latent space
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
-
The novelty of the proposed method
-
The logic of writhing
-
the adequacy of results
-
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a recursive VAE-based method for synthesizing 3D blood vessel trees.
The reviewers appreciate the paper both for its interesting problem and for being well written. On the other hand, several concerns are also brought up, and these will need to be addressed during the rebuttal:
- There are some concerns regarding the clarity of the method (reviewer 1)
- A number of real life vessel trees are not modelled well by the proposed methods (reviewers 1 and 2)
- The experimental validation misses comparison with state of the art (reviewer 1)
Author Feedback
We thank the reviewers for their thoughtful feedback and invaluable suggestions. Below, we provide a point-by-point response to the main raised concerns.
Unmodeled Vessel Trees (R1/R2) We appreciate the concerns raised by reviewers regarding the capability to model certain vessel trees. In this work, we set an empirical depth threshold of ten for trees. This decision reflects a balance between the computational demands of depth-first tree traversal in each training step and the complexity of the training meshes. However, this threshold can be changed for training with larger trees, at increased computational costs. Moreover, while our current method works for binary trees, the exclusion of non-binary bifurcations is not a fundamental limitation, since it is feasible to convert non-binary trees into binary trees. Generating higher-order trees would require architectural modifications, without much benefit in return.
When it comes to modeling capillaries structures (R2), our framework would require significant adaptations to accommodate the non-tree-like structures found in capillary networks. Exploring this problem would certainly be an interesting direction for future research and we will discuss this in the revised manuscript. While we understand that our method may not perfectly model every type of vessel, these structures are generally tree-shaped and we believe our method represents a substantial step forward in this research area where SOTA have trouble generating realistic shapes and bifurcations. As for the concern of utilizing a single dataset (R1), we agree that training on additional data may strengthen our results. However, curating extensive datasets for these intricate 3D structures is a challenge in itself. As research in this field progresses, we anticipate that more high quality datasets will emerge, and we are committed to incorporating them into our future work as it becomes available.
SOTA Comparison (R1) We appreciate reviewers feedback on our validation and aim to provide further context regarding the lack of quantitative comparisons against SOTA methods. Unfortunately, the work of Wolterink 2018 is not publicly available, it was not provided upon request, and it wouldn’t be fair to compare our method using ad-hoc implementations. Additionally, it does not produce bifurcations, which further complicates comparisons. Hamarneh’s 2010 method is not data-driven, and the 3D generation is significantly affected by the choice of physics-based parameters, such as oxygen demand and terminal node pressures. These factors aren’t directly relevant to our context, which renders a direct quantitative comparison less meaningful. Consequently, we focused on a quantitative analysis showing that generated data distribution is aligned and consistent with real blood vessel data, and a qualitative comparison with the state-of-the-art that, as pointed out by R2, showed that VesselVAE produces more realistic blood vessel structures.
Clarity of the Exposition (R1) As signaled by R2, significant effort was invested in ensuring the manuscript was well-written and organized. In regards to the specific concern (R1) about the reference to “a regression network” in page 4, we would like to clarify that it refers to “Features Dec-MLP” in Fig. 1, and it’s directly related to the Lrecon objective. In light of your comments, we will revise the Methods section to ensure it provides a more straightforward presentation. For the sake of clarity, and since we are committed to open science, our code, weights, and demo will be publicly available upon acceptance.
Diversity Metric (R2) As pointed by R2, the metrics from Figure 3 are similarities. These metrics effectively demonstrate the diversity of the generated vessels in terms of their tortuosity, radius, and length, as their distributions closely resemble real data. Our goal is to generate vessels that simulate real ones, and the results align with our expectations.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposes a recursive VAE-based method for synthesizing 3D blood vessel trees. Strengths include methodological novelty and promising qualitative results. Weaknesses brought up by the reviewers included clarity, coverage of real-life scenarios, and quantitative comparison with existing work.
In their rebuttal, the authors were asked to comment on the weaknesses, and in particular, they decline comparison with existing work due to differences in modelling objectives, or that it would be unfair to make their own implementation of somebody else’s non-available method. This, however, is how comparison to existing work is often done – you implement existing methods and do your best to get them to work on your own data, which often involves tweaking even for publicly available code.
Thus, due to the missing comparison with existing work, this paper drops in rank compared to other submissions, and I cannot recommend acceptance at MICCAI’23.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper introduces a data-driven generative model designed for synthesizing 3D blood vessel geometry. The training process involves converting the 3D blood vessel meshes into a binary tree representation. Subsequently, the proposed RNN-based VesselVAE framework is trained to acquire a comprehensive understanding of the hierarchical tree representations pertaining to blood vessel topology and structure. By utilizing the generated tree-structured centerlines, the framework synthesizes 3D meshes. The experimental results, both qualitatively and quantitatively, strongly support the notion that this approach is capable of generating realistic and diverse 3D blood vessel geometry. The reviewers acknowledged the paper’s intriguing problem statement and well-written content but expressed concerns regarding certain clarifications and comparisons with state-of-the-art (SOTA) methods. In the rebuttal, the authors addressed these concerns to some extent. The qualitative comparisons were particularly appreciated as they effectively demonstrated the effectiveness of the proposed methods. While it is generally expected for authors to compare their method against existing SOTA approaches, there may be instances where the implementation of such methods is not publicly available, or the assumptions of the related method are not directly applicable to the proposed approach. Considering the rebuttal, it seems reasonable to accept the paper, given the novel application of VAEs in 3D blood vessel synthesis and the recursive framework employed to generate multi-branch vessel geometries using a binary tree representation.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After careful evaluation of the authors’ feedback and the final decisions of the reviewers, this paper has received mixed scores. One reviewer leans towards rejection, highlighting concerns regarding the lack of clarity in the technique and experimental results.
On the other hand, two reviewers lean towards acceptance. However, they also express reservations about the experimental results, including comparisons with the current body of literature and lack of clarity in the technique.
Considering the mixed scores and the concerns raised by the reviewers, the Meta Reviewer leans towards rejection. The overall evaluation of the paper’s scores supports this decision.