List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Xiao Zhang, Feihong Liu, Yuning Gu, Xiaosong Xiong, Caiwen Jiang, Jun Feng, Dinggang Shen
Abstract
Systolic and diastolic registration of coronary arteries is a critical yet challenging step in coronary artery disease analysis. Most existing methods ignore the important relationship between vascular geometric shape and image contextual information in the two phases, leading to limited performance. In this paper, we propose a novel intrinsic structural point learning-based point cloud registration method, which comprehensively captures both point-level geometric features and image-level semantic features as enriched feature representations to assist coronary registration. Specifically, given the systolic and diastolic CCTA images, our method improves coronary artery registration from three aspects. First, point cloud encoder learns the spatial geometric features of the points in the 3D coronary mask to effectively capture the vascular shape representation. Second, a vision transformer (ViT) is used to extract the image semantic information as a complementary condition of the geometric features to identify the bi-phasic correspondence of different vascular branches. Third, we design a transformer module to fuse the features of points and images to obtain the corresponding structural points in the two phases, and then use structural points to guide the coronary artery registration via thin-plate spline (TPS) method. We evaluate our method on a real-world clinical dataset. Extensive experiments show that our method significantly outperforms the state-of-the-art methods in coronary artery registration.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43990-2_74
SharedIt: https://rdcu.be/dnwMy
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents an approach to achieve systolic and diastolic registration of coronary arteries. The proposed method captures point-level geometric features extracted by PointNet and image-level semantic features extracted by ViT as enriched feature representations to obtain the corresponding structural points. And the TPS method is used to coronary artery registration guided by the structural points. The experimental result demonstrates the proposed method’s effectiveness.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors combine point level features extracted by PointNet and image features extracted by ViT to achieve more accurate coronary registration results. The combination of these features for registration is an innovative contribution of this paper. Additionally, the authors effectively highlight the detected structural points in their experiments and explore the impact of the pre-set number of points on the results. Such analysis is of great significance for understanding the proposed approach.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
A.The description of the proposed method is unclear in certain aspects. In the Eq. (4), “The third part constrains the two set of structure points to be close enough as possible.” For two images to be registered, it seems unreasonable to require the distance between the detected structural points sets to be as close as possible. Furthermore, while the paper highlights the importance of correspondence in structural points for successful registration, it does not clearly explain how the proposed method ensures such correspondence between points.
B.The dataset used in the experiment is too limited. Testing the proposed method on some publicly available datasets, such as the FIRE dataset [1], would be a useful way to strengthen the paper’s conclusions.
[1] C. Hernandez-Matas, X. Zabulis, A. Triantafyllou, P. Anyfanti, S. Douma and A. A. Argyros, “FIRE: Fundus image registration dataset”, J. Model. Ophthalmol., vol. 1, no. 4, pp. 16-28, 2017, [online] Available: http://www.ics.forth.gr/cvrl/fire/.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The dataset used in the article is non-public and the designed network structure is complex, therefore it would be difficult to implement it. It is expected to evaluate the proposed methods on some public datasets and make code available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
a. The description of the method is unclear in certain aspects. It is not clear how the method is used to extract image blocks along the vascular structure, or how the order is determined. Additionally, according to my understanding, k in the definition of V_p refers to the pre-defined number of structural points, and this should be clearly indicated. b. As mentioned above, the role of the third term in Eq.(4) is confusing. Its effect can be demonstrated by adding ablation experiments or mathematical proof. c. How is the comparison method defined? For instance, in VoxelMorph, is the binary image of the blood vessel used (optional in the original paper), and is the image initially aligned? d. It is necessary to validate the method on some publicly available datasets. e. The paper presents a complex network structure, thus it is expected to provide the number of parameters and registration time.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper has both strong and weak points. This paper combines point level features and image features for image registration, which is innovative, although the feature extraction methods are commonly used. My concerns on this work are mainly the lack of justification in some places. I think it can be addressed by experiments on more datasets or more detailed mathematical proofs.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This work proposed a deformable registration method for coronary arteries by combining two state-of-the-art neural network architectures. It is a hybrid approach, where both geometric as well as image semantic information are incorporated. The retrospective quantitative evaluation with clinical data demonstrate the effectiveness of the proposed method in comparison to other image-based or feature-based registration methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The novelty of this paper is majorly reflected by the combination of a PointNet-based with a ViT-based architecture to extract and represent the geometric as well as image feature, respectively. Due to the best of my knowledge, such a combined network architecture has been applied for the registration of coronary arteries for the first time. Furthermore, in the quantitative evaluation, authors used real clinical data, which is an additional plus point of this paper
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The major limitations of this work are the follows: 1. the related works are somewhat not comprehensive. As the authors proposed a hybrid technique, a brief overview of the existing hybrid registration methods is beneficial. 2. A short justification of the application of TPS as interpolation/warping technique is missing. Why the author chose to use TPS whereas it is a globally controlled method, leads to higher computational cost with the increasing number of corresponding points? Furthermore, TPS is less robust against outliers, which could potentially reduce the accuracy of the final registration result. Why a locally controlled method such as B-spline is not considered/chosen at all? 3. The authors have chosen seven state-of-the-art methods for the quantitative comparison. But the logic of chose these seven methods are not completely clear for me. Some of the selected seven methods (e.g., SyN, VoxelMorph etc.) are not primary designed for registration of vessel structures, therefore the lower performance of the them are not unexpected. In contrast, other more robust approaches considering the spatial locations and topology of the vessel trees (e.g., CPD, TMM, HdMM), that are not based on neural networks, are not selected. It would be valuable to see the comparison of the proposed methods to the most state-of-the-art vessel registration methods. 4. Why are the evaluation metrics CD and HD are not used in the second experiment? For instance, as HD is sensitive to outliers as well, it would be interesting to see HD for the case #8 in Table 2, especially as the authors stated “However, the performance decreases when it is further increased to 1024, indicating that dense structural points negatively affect the results, which is probably caused by the increasing number of outlier points.”
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors described the building blocks and architecture of the proposed network clearly. Implementation details such as training and evaluation schema as well as used hardware are also described. However, neither the process of data acquisition is described nor the data is published. The source code of the implementation is also not provided. In total, the reproducibility of the proposed method incl. the described evaluation schema are low.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
In general, the work is well motivated, structured and easy to understood. The proposed method is straightforward though effective to solve the given problem. This is a great achievement in my opinion. For the acceptance of MICCAI novelties in terms of sophisticated methods/algorithm, great reproducibility or/and evaluation schema/data is requested. The evaluation experiments, especially the selection of the baseline methods, are not sufficient to draw a comprehensive conclusion regarding the performance of the proposed network. I would suggest the authors to expand this work for a journal submission with more carefully structured literature review, dedicated description of the data acquisition process, clear logic of the quantitative comparison and comprehensive evaluations incl. more detailed analysis/discussions of the results. A further evaluation with CT images containing vessel/tubular structures could demonstrate the robustness and generalizability of the method.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work is well written and structured. It proposed a novel way to combine two existing architectures to solve a clinical relevant problem, although there is no fundamental new innovation on the network architectures themselves. And the evaluation is based on real clinical data, demonstrating the effectiveness of the proposed methods. However, the major limitations of this work (refer to 6) reduced the quality of this work greatly. The insufficient reproducibility reduced the quality of the paper further.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The paper describes a method for fusing incomplete coronary arteries segmented from the diastole and systole phases. It achieves this by combining a point cloud encoder with a transformer network. The method was trained with manually segmented CCTA datasets from 40 patients, and evaluated with 18.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The approach is novel and well described. The performance of the described work is compared to seven state-of-the-art methods, with favorable results for the described methods. An ablation tests shows the impact of various parameter choices and algorithmic components.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
With the step-and-shoot approach to CCTA, multiphase CCTA is becoming rarer. Is this paper still relevant in the near future?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
While the authors explain the overall system, the steps taken and most important aspects, the reproducibility of their work will be very much helped by making code and data available. From the text in the paper it is not clear whether the code will be publicly available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Generally: I am assuming that your method takes in the segmented coronary arteries. If this is correct, then it should be made much clearer that the segmentation of the coronary arteries is out of the scope of this paper.
- Introduction: “However, conventional reconstruction methods merely exploit the images obtained from the diastole that only reveals partial coronary arteries, which may cause diagnosis bias”: To which degree is it true that coronaries segmented from end-diastole always are always incomplete?
Minor:
- Abstract: “First, point cloud encoder…” should be “First, the point cloud encoder…”
- Introduction: “cardiac circle” should be “cardiac cycle”
- Equation 1 and equation 4: you are using x and y with different meaning in those two equations. Please try to use unique variable letters or names.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, the clinical application and validation in the paper is quite convincing.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper is about registering coronary arteries between the diastole and systole phases. The proposed method is a hybrid approach and it combines features extracted by PointNet and image-level semantic features. It has novelty and the results are more accurate. The reviewers have some concerns about the paper’s clarity (e.g., more method description and a more comprehensive overview of the existing hybrid registration methods are expected) and method evaluation (limited dataset, the inclusion of other vessel registration methods, and why evaluation metric CD and HD are not included).
Author Feedback
We thank the reviewers and meta-reviewer for their efforts and insightful comments. We address their concerns below.
Q1: Clarification on our method description, i) meaning of Eq.4, ii) implementation of VoxelMorph (R1, R3, MR) Eq.4 denotes the chamfer distance used in the three terms of Eq.3, while Eq.3 denotes the total loss. The first two terms of Eq.3 aim to encourage accurate vessel representation of the predicted structural points. The third term encourages accurate alignment of structural points between the two phases. Thus, Eq.3 ensures structural points to be employed for accurate registration. Fig.3 visually demonstrates these results. As for VoxelMorph implementation, we followed the original method, including coarse alignment of the initial image and applying constraints of the vascular binary mask.
Q2: Availability of public dataset (R1, R3, MR) Multiphase CCTA is a relatively new technique, thus no well-known public dataset so far. R1 mentioned the FIRE dataset, which is a fundus dataset. It cannot be directly employed to evaluate our proposed SPR-Net due to lack of vessel annotation. If necessary, we may ask cooperative doctors to annotate the vessel masks for the FIRE dataset. Furthermore, we plan to collect additional coronary data and make them a public dataset with the goal of accelerating the field.
Q3: Overview of the hybrid registration methods, implementation of CPD, TMM (R2, MR) Hybrid vessel registration methods have shown promising performance [1,2], such as CPD and TMM. We now implement CPD and TMM as recommended by R2, and the results further demonstrate the superiority of our proposed SPR-Net. In the final paper, we will cite relevant papers and add the results. …………..Dice………..CoDice……..CD………..HD CPD 46.73±2.88 52.69±2.73 6.20±2.43 8.54±2.16 TMM 54.76±1.98 60.21±2.14 4.11±1.87 3.35±1.41
Q4: Choice of evaluation metrics in Tab.2 (R2, MR) Because of the length limitation, we only provided Dice and CoDice in Tab.2. To further support the effectiveness of SPR-Net modules, both CD and HD will be added to the final paper.
Q5: Clarification on i) the effect of structural point number and ii) robustness of TPS (R2) The number of structural points is a key parameter that affects registration performance. Insufficient points may result in a loss of detailed vessel shape information, while an excessive number may introduce non-corresponding or distorted shapes in “deficient vessel parts” (Fig.3). Through extensive experiments, we determined the optimal number of structural points to ensure one-to-one correspondences between diastole and systole (Tab.2). With the abovementioned structural point correspondence, we employed TPS as the registration method due to its merits of preserving the continuity and smoothness of the global vessel shape [3]. B-spline suffers performance limitations, particularly in maintaining the integrity of vessel boundary and endings, and may produce excessive smoothing or folding [4]. We will provide a detailed discussion and comparison in the final paper.
Q6: Clarification on details, including i) application in multiphase CCTA and ii) incompleteness of diastolic vascular segmentation (R3) Multiphase CCTA can visualize coronary artery in multiple cardiac phases, which fits with our motivation and method and definitely has applications in the near future. As for the effect of incompleteness, [5] indicates that coronary vessel area varies with a ratio from -3% to 10.4% across the two phases. Thus, one phase data only reveals about 90% vessel tree, which potentially makes vessel lesions invisible, i.e., misdiagnosis. We will clarify these in the final paper.
Q7: Reproducibility of the method (R1, R2, R3) We will make the codes publicly available upon acceptance.
[1] 10.1007/978-3-319-46726-9_34 [2] 10.1016/j.media.2017.11.012 [3] 10.1109/TNS.2006.889161 [4] 10.1109/BHI.2012.6211615 [5] 10.1016/0002-8703(95)90234-1
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I have read the comments and rebuttal. This paper is about the systolic and diastolic registration of coronary arteries. Most of the concerns have been addressed in the rebuttal.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper is of high quality and addresses a very interesting topic (point-cloud based vessel registration) that has not received very much attention so far. The method is sound and substantially outperforms both voxel-based and basic image-based models. I recommend acceptance and an oral presentation but would like to stress the criticism of reviewer #2 that the comparisons are not sufficient. Vessel alignment in 3D has been widely studied in pulmonary/lung registration and hence a discussion of point-cloud based registration methods [1,2] from this area are necessary in the final paper (and are likely to perform much better than CPD) [1] Shen, Z, et al. “Accurate point cloud registration with robust optimal transport.” Advances in Neural Information Processing Systems 34 (2021): 5373-5389. and [2] Hansen, L et al. “Deep learning based geometric registration for medical images: How accurate can we get without visual features?.” Information Processing in Medical Imaging: 27th International Conference, IPMI 2021
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The proposed hybrid method for coronary arteries registration is technically sound and interesting, and the authors have cleared all the concerns in rebuttal. Therefore, I recommend accepting this paper.