List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Ding Xia, Xi Yang, Oliver van Kaick, Taichi Kin, Takeo Igarashi
Abstract
Image registration is an essential part of Medical Image Analysis. Traditional local search methods (e.g., Mean Square Errors (MSE) and Normalized Mutual Information (NMI)) achieve accurate registration but require good initialization. However, finding a good initialization is difficult in partial image matching. Recent deep learning methods such as images-to-transformation directly solve the registration problem but need images of mostly same sizes and already roughly aligned. This work presents a learning-based method to provide good initialization for partial image registration. A light and efficient network learns the mapping from a small patch of an image to a position in the template space for each modality. After computing such mapping for a set of patches, we compute a rigid transformation matrix that maps the patches to the corresponding target positions. We tested our method to register a 3DRA image of a partial brain to a CT image of a whole brain. The result shows that MSE registration with our initialization significantly outperformed baselines including naive initialization and recent deep learning methods without template. Our dataset and source code will be published online.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_25
SharedIt: https://rdcu.be/cVRS7
Link to the code repository
https://github.com/ApisXia/PartialMedPreregistration
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
Presents a method for partial image registration of 3D rotational angiography (3DA) and CT angiography of the head. Their model combines several blocks to predict the location of patches in the template space and a transformation. They demonstrate improvement over 4 baseline method in measuring distance from ground truth anchor points.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Strong experimental results
- Large dataset with gold standard labels
- Good comparison with baseline methods
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Limited methodological novelty: combines several blocks from prior work
- Unclear training routine: is the entire dataset used in training?
- Time is not reported
- Requires labels
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper is challenging to reproduce. Their algorithm uses several blocks from existing work, and the authors modified these blocks without detail. For example, when describing AirLab, the authors list that they “modify the code as needed” to process partial volumes. While details of this are not needed in the paper, it would be helpful to release code. Further, the training routine and data split is not well described.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This paper presents a method that improves performance on the challenging problem of partial volume registration. However, my concern is with the limited novelty in this work, and incomplete experimental details.
Specifically, can the authors:
- describe why the method is not sensitive to the choice of template shape?
- Describe the data split used in training. From my understanding, all images were used in training, in which case the network is overfit to the data, with supervised labels. This seems like an unfair comparison, and I don’t believe this method would be feasible in practice.
- How sensitive is the method to the choice of number of image patches selected?
Since the method is the combination of several existing works, it is hard to recommend acceptance without methodological novelty.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper lacks methodological novelty. While the results improve on prior work, I am unsure about how the training was done, and seem convinced the model is overfit to the dataset. This would render the model unusable in practice.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The authors propose a deep learning-based method to perform the initial rigid registration of two images (especially when one image has a much smaller field of view) in order to provide an accurate starting point (based on translation only) for a second registration refinement. This effectively gets the second refinement registration out of many local minima. The proposed method utilizes a common template space with multiple sub-patches from the moving image to infer the initial transformation and then utilizes a standard registration approach for refinement. Experiments rigidly registering multi-modal 3D rotational angiography and CT angiography demonstrate the approach.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
• Comparison to both standard initialization techniques using traditional registration methods and to deep-learning methods is rigorous. • The proposed method outperforms the comparison method. • Evaluation on moving images with partial field of views (FOVs) stratified by FOV size is good (Fig. 4). • Cross-validation is rigorous for evaluation. • Ablation experiments demonstrate the importance of removing outliers (using RANSAC) in the inference results.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
• There are some model training hyperparameters that could use additional evaluation and/or analysis, e.g. patch size.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The dataset and imaging information used in this study are well described. The authors describe their training procedure in sufficient detail. Source code will be made publicly available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Sec. 2.1: There are a few Template-Space Patch Mapping (TSPM) parameters that would be interesting to test. Most interesting would be the patch size. 16^3 is used as input, but larger patches might offer more context that would improve performance. Different patch sizes should be investigated, or at the least provide a discussion of this at the end of the paper.
Sec. 3: Regarding your data splitting for the cross-fold validation experiments, please verify that your training/testing splits were stratified by subjects. You want to ensure that there was not data contamination between the two sets, e.g. the training and testing sets both had patches from the same subject.
Sec. 3: One of the challenges in this approach is that the small FOV images may come from very different parts of the template volume. I am curious if your dataset had 3DRA images that were uniformly distributed throughout the template volume. Or, did one particular location of the anatomy have more 3DRA images? Some discussion about this would be interesting. This might also be interesting to see if your failure cases were examples that were from locations poorly represented in your training data.
Grammatical/typographical: Sec. 2.1: “The rest space” -> “The remaining space”
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This is a well written paper with good experimental setup and validation of a novel registration initialization method based on deep learning. The authors provide a set of cross validation experiments to compare their proposed initialization strategy to conventional initializations and then demonstrate the effect these have on rigid registration in both standard (non-deep learning) and deep learning registration.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
I maintain my original scoring for this submission. Overall, this is a well written paper with good experimental setup and validation of a novel registration initialization method based on deep learning. While individual components may not be considered novel contributions, the application of these to this particular task is innovative and the authors demonstrate that their method is effective through a rigorous analysis.
Review #3
- Please describe the contribution of the paper
The paper introduces a method that aims to improve the initialization for other local-search registration in rigid multi-modal partial image registration. Instead of doing a direct matching, the 2 images are matched to a common template space. The method works for images of diverse sizes, and for partial volumes
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Alternative deep learning methods typically assume that data images the same whole region, that they are roughly aligned already or even that data has the same size or scale. This initialization step, however, performs well without requiring any of those.
- The proposed method represent a potentially very interesting addition as initialization for a variety of multimodal registration methods.
- The experiments section is quite good. It’s very detailed, covers a good range of experiments and enough competing methods
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- It would be interesting to see also execution time comparisons for the diverse registration methods
- While the experiments section in general is quite good, there is not enough technical data about the dataset to recreate a similar one
- There is no time performance indication.
- The Conclusions section is too short and it doesn’t really add much. It would be interesting to merge the Discussion subsection from the Experiments into the Conclusions section
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- Method is not publicly available
- Dataset is not publicly available, and there is not enough technical data about the dataset to recreate a similar one
- Method could be implemented from the provided description
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- In general terms, this is a solid paper. It introduces a novel method that is an interesting and robust alternative for being an initialization for other multimodal registration methods.
- The introduction and method sections are well detailed.
- The experiments section is quite good, providing extensive information about the experiments. However, it would be good to provide more technical data regarding the datasets that were utilized in the experiments.
- It would also be good to see time performance information
- The conclusions section is quite short, but I feel that most of its contents are actually located under the Discussion subsection from the experiments section. I’d aim to combine Discussion and Conclusions under the Conclusions section.
- The proposed future work, regarding both the performance improvements, as well as experimenting with deformable registration look like the adequate next steps for this work
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
- The paper provides a very interesting alternative initialization for multimodal registration methods, outperforming competing methods in a range of categories.
- I think it’s a good contribution to the field.
- I can’t find major faults on the method or the paper itself, which is quite solid
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Data-driven Multi-Modal Partial Medical Image Preregistration by Template Space Patch Mapping
This submission tackles the pre-registration of partial images. It learns an optimal initialization for standard registration approaches, by matching sets of small patches between an image and a template space with partial overlaps to a whole image. The evaluation is on partial angio and Ct with a private dataset, and shows better pre-registration than competing methods. The originality is that this approach tackles the initialization phase of modern registration algorithm, where partial images still pose a challenge, thus, despite its limited methodological novelty, it addresses a particular need with potential impact in the field. The reviewers have however indicated a few points to be addressed, such as methodological novelty (R1), effect of patch sizes (R2), hyperparameters and datasplits (R1,2), and reproducibility (R1,3), all to be addressed in a rebuttal.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
We sincerely thank all reviewers for comments.
META-REVIEWER #1 Thanks for summarizing. We’ll answer the main questions here
Q1: methodological novelty (R1) A1: It is true that some blocks are not new. However, we confidently claim novelties in the following aspects 1) We solved the partial image registration task, which is a particular need in the medical image area but is under-explored 2) Patch-to-template is new in 3D brain registration. Existing methods usually use entire volumes as input or patch-to-patch way. But, as we tested in Sec.3, they are inaccurate 3) Our model predicts only 3 variables (positions) while the traditional DL registration is 6, which improves efficiency and accuracy
Q2: effect of patch sizes (R2) A2: Thanks for the advice. We tested multiple sizes and 16 px is selected based on success ratio, e.g.. The smaller (12 px), are worse, due to too less information. The larger (20 px), cannot have enough patches being sampled. We will add detailed description in revised supp. materials
Q3: hyperparameters & data splits (R1, 3) A3: In training, batchsize is 32, without augmentation. In RANSAC, per iteration (total: 3000), we randomly picked 6 patches, error thres. is 10, and min. pick num. is 20 For data splits, we strictly divide testset & trainset. In detail, we split pairs of 3DRA & CT into 5 folds. Training/testing stage only uses patches from corresponding cases. Thanks for the suggestion, we will add these in the revised version
Q4: reproducibility (R1, 3) A4: We did not release code before submission for anonymity. We will release it as soon as paper is accepted. For data, we will release limited desensitized data. It is difficult to release the entire dataset soon due to ethical concerns (raw scan data might leak face geometry). We will keep efforts to prepare public dataset, but it takes time (get consent from patients)
Q5: time (R1, 3) A5: Thanks for advice. Cent. 16.580s w/o P. 19.773s Prop. 28.583s Atten. 21.202s Prop. w/o Ref. 24.180s Prop. w/o RAN. 20.427s Which is time to execute a complete pipeline Input: unregistered Dicom -> output: registered Dicom
Due to limited space, sorry about using abbr. for questions that can be answered above (Like M-A1)
Reviewer #1 [Weak] Q1: Lim… A1: M-A1 Q2: Unclear… A2: M-A3 Q3: Time… A3: M-A5 Q4: Req… A4: We considered the prediction of positions in canonical space as regression problem, which means there is no label (Sec. 3 in paper) [Reproducibility] Q5: The paper… A5: M-A4 [Constructive] Q6: des… A6: A template just serves as media for rough registration between 3DRA & CT volumes. The subsequent refinement does not rely on it. So, the choice of it is not a problem Q7: Des… A7: M-A3 Q8: How… A8: We used around 11100 patches for training without overfitting. But it is interesting to find how patch/case num affect performance. We will work on that in future Q9: Since… A9: M-A1
Reviewer #2 [Weak] Q1: There… A1: M-A2 [Constructive] Q2: Sec. 2.1:… A2: M-A2 Q3: Sec. 3:… A3: M-A3 Q4: Sec. 3:… A4: Different brain regions have almost the same probability. But Cerebellum is relatively more often collected, while the Front lobe is less often For failure cases, limited by the length of the article, we will add more to the webpage of our project Q5: Gram… A5: Thanks!
Reviewer #3 [Weak] Q1: It would… A1: M-A5 Q2: While… A2: Experts created the gt dataset using Avizo by these steps 1) Manually initialize a rough relative position & rotation between 3DRA & CT 2) Crop down CT to the size of 3DRA due to the automatic workflow of NMI in Avizo requires the input of 2 modalities has the same size 3) Manually verify the results Usually, it takes 10 min to register a case About how to train the model (data split), please refer to META-A3 Q3: There… A3: M-A5 Q4: The Con… A4: Thanks! [Reproducibility] Q5: Method… A5: M-A4 [Constructive] Q6: The exp… A6: M-A3 Q7: It would… A7: M-A5 Q8: The con… A8: Thanks! Q9: The proposed… A9: We also think so!
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Data-driven Multi-Modal Partial Medical Image Preregistration by Template Space Patch Mapping
The rebuttal has clarified the main concerns, despite its abbreviated format. The paper offers to tackle an original problem of partial registration, despite the limited methodological novelty. Recommendation is, therefore, towards Acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The submission introduces a framework for partial image registration of 3D rotational angiography and CT angiography of the head. The model combines several blocks to predict the location of patches in the template space and a transformation. Instead of doing a direct matching, the 2 images are matched to a common template space. The method works for images of diverse sizes, and for partial volumes. The authors demonstrate improvement over 4 baseline method in measuring distance from ground truth anchor points.
The submission introduces a novel registration initialization method based on deep learning.
The experimental section includes carefully set up experiments with large data sets.
The authors participated in the rebuttal. Although their format was unusual and took away space from full-sentences reviews, they managed to address the key concerns of the reviewers.
Some part of the data and the code will be released if submission gets accepted to support reproducibility efforts.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Practivally useful application with encouraging results. Clear and undecorated presentation.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR