List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Haosheng Su, Xuan Yang
Abstract
Non-uniformly spaced control points located on the interface of different objects are beneficial for constructing an accurate displacement field for image registration. However, extracting features of non-uniformly spaced control points in images is challenging for convolutional neural networks (CNNs). We extend a probabilistic image registration model using uniformed-spaced control points by employing non-uniformly-spaced control points. We construct a network to extract the image and spatial features of non-uniformly-spaced control points. Moreover, a variational Bayesian (VB) model using a factorized prior is employed to estimate the distribution of latent variables. In theory, we analyze the KL divergence between the posterior and the two separated priors. We found that the factorized prior has the advantage of decreasing the KL divergence, but too more factorized priors, such as the standard normal, might deteriorate registration accuracy. Moreover, we analyze the relationship between the uncertainty of the displacement field and the spatial distribution of control points. Experimental results on four public datasets show that our network outperforms the state-of-arts registration networks and can provide registration uncertainty.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43999-5_60
SharedIt: https://rdcu.be/dnwxd
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a probabilistic method for non-rigid image registration with application to cardiac registration. The method can be classified into the family of Krebs et al. 2019 [17] or diffeomorphic VoxelMorph 2019 [8]. The contribution of the authors relies on the use of non-uniformely spaced control points and the partition of the global prior into several independent priors. The results show a modest increment in accuracy measured in terms of the Dice coefficient.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The topic is interesting for Miccai community.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The main weakness of the paper is the clarity of the paper. It is hard to understand the problem setup and the improvements proposed by the authors to the baseline methods (KrebsDiff and VoxelMorphDiff).
-
The idea of using non-uniformely distributed points is not well motivated. I miss some ablation with a comparison with respect to uniformely distributed points.
-
The results show a modest improvement in terms of the Dice coefficient. The method does not compete with DalcaDiff on the quality of the Jacobians.
-
- Please rate the clarity and organization of this paper
Poor
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The datasets used for evaluation are publicly available.
The details given in the paper are confusing and I do not feel able to exactly reimplement the proposed method even taking VoxelMorphDiff codes as starting point.
For the evaluation, the authors provided a clear description of metrics and tendency. Statistical significance was stated when needed.
The time and memory complexity of the proposed method are not given.
The clinical significance of the method can be inferred from the introduction. However, the proposed method needs further validation in more different datasets for considering moving to clinical application.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The addressed problem is relevant to Miccai community and the approach selected by the authors seems to appealing with an interesting theory behind. However, it seems like the paper has been written in a hurry, and the flow of the algorithm is hard to imagine from the information given in the manuscript. The manuscript is not in good shape for the standards of Miccai conference.
I believe that, according to Miccai scoring, this is a paper with moderate to major weaknesses. In the following, the aurhors can find different appreciations that may help improve the quality of the manuscript.
Title. Variatonal methods usually refers to traditional methods. I suggest changing the title to Variational Bayesian methods. The word probabilistic could remind to probabilistic methods such as KrebsDiff or VoxelMorphDiff.
Introdution. “unsupervised registration methods based on Variational Bayesian”. Why not unsupervised registration based on Variational Bayesian methods?
Method. The authors directly state that the output of the algorithm is a function parametrized by network parameters z. It would be good to know what is the funcion representing (e.g. displacements, transformations…).
Figure 1. This figure is quite small. Indeed it would be more informative to represent the uniformely spaced control points in the same way as the non-uniformely spaced control points. Some key ideas on the benefits of using non-uniformely spaced points would be good.
Method. The term DVF has not been defined in the paper.
Method. Why use letter u for a pixel coordinate? I believe that a better option would be to use x.
Method. The term VAE (variational autencoder) should be defined in the paper.
Experiments. Dataset. Are the experiments given in 2D? Why not 3D?
Experiments. Registration Results. The naming convention of the authors method is too complicated. I suggest rethinking the presentation.
Figure 3. The source and target images of each registration pair should be stated in the subfigures. Otherwise, the reader may misunderstand the experiment. This also applies to Figure 4.
Figure 4. What is the meaning of 8000 12000 etc in the captions? Maybe the number of iterations?
Conclusion. C
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
As I said, I believe that, according to Miccai scoring, this is a paper with moderate to major weaknesses. The clarity of the manuscript is not according to the average quality of a Miccai manuscript. In addition, it is hard to understand the extend of the contribution of the proposed method to the state of the art.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
I am changing my decision to weak accept. It seems that the other reviewers do not mind the weaknesses of the paper and the contribution weights more in this case.
Review #2
- Please describe the contribution of the paper
The paper proposes a probabilistic image registration model based on variational Bayesian using non-uniformly spaced control points for cardiac image registration. The authors argue that non-uniformly spaced control points located on the interface of different objects are beneficial for constructing an accurate displacement field for image registration. However, extracting features of non-uniformly spaced control points in images is challenging for convolutional neural networks (CNNs). To address this challenge, the authors extend a probabilistic image registration model, and construct a network to extract the image and spatial features of non-uniformly-spaced control points.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Strengths:
- The paper proposes a novel approach to improve image registration accuracy by employing non-uniformly spaced control points.
- The authors address the challenge of extracting features from non-uniformly spaced control points using CNNs.
- The proposed method is not sensitive to location errors of control points.
- In the experimental setup of the author, there is a comprehensive explanatory framework that is good and reasonably sound.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Weaknesses:
- The author claims that their approach is not sensitive to location errors of control points, but it appears to be challenging to handle non-uniform selections from a methodological perspective, and this part is somewhat ambiguous..
- The evaluation is limited to two datasets only.
- The paper does not discuss potential limitations or drawbacks of using non-uniformly spaced control points in image registration.
- Is the result of the uncertainty maps sensitive to the control points of boundary of objects?
- If further research directions for the future can be provided, it would be commendable. Such as can the effectiveness of the method be demonstrated on a large clinical dataset?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility is average with some missing details, but the author declares to provide the training code and testing modes, which seems promising.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The authors demonstrate that employing nonuniformly spaced control points in a variational Bayesian image registration model improves registration accuracy. The control points are spaced on the contours of objects, and their intensity and spatial features are extracted using a network. They addressed the inherent disorder challenge in the control-points-based image registration model using CNNs, which can locate control points freely instead of only on grids. My major concerns are addressed in the weakness section. In addition, I am also interested in considering the generalization issue of this method, since NuPs are introduced.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is well clarified and organized, and the experiments are also well-conducted, which can demonstrate the points that they need to support. This is a well-completed article.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
After the author’s rebuttal, some of my questions were partially resolved, but I still maintain my previous judgment: 6 (accept).
Review #3
- Please describe the contribution of the paper
- employ the nonuniformly spaced control points in a va-riational Bayesian image registration model, to improve the registration accuracy;
- partition the global priors into several independent priors, which correspond to different control points;
- the proposed approach provides more available information about registration uncertainty at the boundary locations.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
the idea of considering non-uniformly spaced control points into the Bayesian-based image registration network seems novel; the results slightly outperform the existing registration methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- There are several places that the expressions are not well defined.
- There is no statistical tests for the experimental results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
More details are needed to completely reproduce the presented method.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Please improve the writing, and modify the errors. For example, CSRBFs seems is not the correct abbreviation for “compact radial basis functions”. In the 4th paragraph of Section 2.1, the terms of abbreviations LV,RV and MYO are not given. The last sentence at the end of Section 2.1 is not coherent.
- More details are needed to illustrate the presented network and the Figure 2. For example, it is still not clear what the FeatureNet is given that there is only sentence to describe the FeatureNet.
- There is no statistical tests between the results of the proposed method and those of the prior state-of-the-art registration methods. Without these information, it is hard for the readers to judge the novelty and improvements of the presented method.
- Lastly, the clinical relevance/significance should be elaborated or validated more.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, it is a paper that owns good merits and innovative values. However, the paper’s writing/mathematical definitions need to be improved. More details need to be added for both clear reading and reproducing. The experiments also need to be supported with more statistical tests.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #5
- Please describe the contribution of the paper
variational Bayesian (VB) model using a factorized prior to estimate the distribution of latent variables and work with nonuniformly placed control points.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors are considering a very interesting question which is the combination (or compatibility) of non-uniformly-spaced control points with deep networks, and account for the associated heterogeneity and variations. VB is also rational direction. Factorization of the latent is correct, though quite straight-forward.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
While the factorization into non-unform and uniform control sets is theoretically correct in the latent distributional sense, I have yet to see or be convinced of the benefit. Also, typically when one is motivated enough to go through the “hassle” of working with nonuniform spaced control points, there is not much reason to overlay that configuration on top of a regular (subset).
Another major deficiency is the less-than-convincing results. Improvement in the 10^-2 scale typically, smaller than the than or comparable to the std, questioning statistical significance. Also such small difference could easily be the results of tunning.
I am not completely sure how to interpret the results in Fig 3 - the major difference in DVF occured in the blood pool region which goes with flow and cardiac cycle - it should be considered in 3D and maybe the only way to know what it “should” look like is via tagging.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reasonable. I suppose the performance would somehow depend on how the nonuniform control points are placed, which is both practice and dataset dependent. it would be hard to reproduce or re-evaluate without a sharing explicit in association with the said dataset (e.g., ACDC) in public-domain.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
statistical analysis
further clarify the compromise, tradeoff, and dependency of the performance and uncertainty as control constallation placement.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is a very challenging problem, and despite the limitations, I feel the authors had made due diligence to present this work with some novel ideas. It certainly has potential to be better and benefit the community with further improvement and clarification.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper received a mixed review of both positive and negative feedback. The reviewers acknowledged the challenge posed by the problem of non-uniformly spaced control points in image registration and recognized a certain level of novelty in the proposed method. However, several significant concerns were raised, primarily stemming from the lack of clarity in the current manuscript. Additionally, questions were raised regarding the statistical significance of the experimental results, as they only demonstrated small improvements. During the rebuttal phase, the authors are strongly encouraged to thoroughly address all of the reviewers’ questions and concerns.
Author Feedback
Reviewer #1: Q1: The problem setup and the improvements proposed by the authors to KrebsDiff and VoxelMorphDiff. A1: In [11], the uniformly spaced control points (UPs) influence registration accuracy. Besides, commonly used priors in VAE are i.i.d., which cannot enforce each dimension of the variational posterior to correlate to each other in control-point-based image registration. We state our motivation clearly on page 2 of the manuscript. Like KrebsDiff and VoxelMorphDiff, our method is also a VAE-based method, which employed non-uniformly spaced control points (NuPs) to control DVFs and factorized the prior into global and local parts to constrain the posterior. The improvements of our method manifested in registration accuracy, and Table 1 lists the registration accuracy comparison results on two datasets. Another advantage of the VAE-based method is the uncertainty estimation illustrated in Figure 4. The uncertainty estimated by our method concentrates on the boundaries of objects, which provides more valuable information than NetGI [11]. We state it on page 7 of the manuscript clearly. We clearly stated our method’s motivation and improvements in the manuscript.
Q2: The idea of using non-uniformely distributed points is not well motivated. A2: Please refer to A1 to see our motivation for NuPs. NetGI used the UPs, and our method used NuPs. Table 1 shows that our method outperforms NetGI in respect of registration accuracy. The sixth and seventh lines in Table 2 show the advantage of factorized prior compared to NetGI. By combining the results of Table 1 and Table 2, the comparison of UPs and NuPs can be seen.
Q3: Experiments. Dataset. Are the experiments given in 2D? Why not 3D? A3: The resolutions of our dataset along the short axis are around 1mm, and that along the long axis are 5 to 13mm. Since the resolutions along the long axis are too coarse, it does not make sense to register in 3D. Therefore, most existing cardiac registration of short-axis cine-MR images is performed in 2D.
Reviewer #2: Q1: Non-uniform selections from a methodological perspective. A1: We trained a supervised learning U-Net to extract the contours of the heart (pNuPs), as stated in the first paragraph on Page 6. We collected all the points on the contour by randomly selecting one point and sampling the farthest point to acquire the nonuniform control points, as stated in the last paragraph on page 3. Results in Table 1 show that the performance of our method does not degrade obviously in respect of Dice and APD, which implies that our method is not sensitive to location errors of control points.
Q2: Potential limitations or drawbacks of using non-uniformly spaced control points in image registration. A2: Although our method is not sensitive to location errors of control points, location errors still deteriorate registration accuracy slightly.
Q3: Uncertainty maps sensitive to the control points of boundary of objects? A3: The uncertainty in regions where the NuPs are gathered is generally large, as section 2.2 on page 5 states. Since NuPs are located near the boundaries of objects, the uncertainty maps are sensitive to the control points of the boundary of objects.
Reviewer #5: Q1: Benefit of factorization into non-uniform and uniform control sets. A1: Priors with i.i.d are too simple to constrain the variational posterior, and the global distribution used in [11] might enforce each dimension to correlate too much, as stated in the first paragraph on page 2 of the manuscript. A small KL divergence can be obtained by factorizing the prior into two parts, which is favorable in increasing the ELBO in VAE to make the variational posterior approximate the real posterior better to improve the registration accuracy. Moreover, the UPs control the global DVFs, while NuPs control local DVFs finely, as stated in the last paragraph on page 4. The first four rows in Table 2 also show the advantage of factorization.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
All reviewers consistently recommend (weak) acceptance of this paper after considering the authors’ rebuttal. I am happy to support the reviewers’ recommendation. Additionally, I strongly urge the authors to meticulously incorporate all questions and suggestions provided by the reviewers into a revised version of the paper to ensure its readiness for final publication.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This is an interesting but somewhat borderline paper in many regards. On the positive side: 2/3 reviewers highlight the technical novel of varying non-uniform control points in diffeomorphic cardiac registration that yields reasonable improvements over simpler uniform models. However, the writing of the paper is not always deemed of high quality and the impact remains limited due to the evaluation on 2D slices.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Overall the variance had high variance in scores originally.
However, the rebuttal seems to have addressed some issues, whereas the whole process and the opinions of all reviewers have eased the worries of others.
Overall, reviewers now agree that the paper should be accepted. There was quite a lot of feedback, and I encourage the authors to put as much as they can in the camera ready, which should help the paper and its discussion at MICCAI