List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Tony C. W. Mok, Albert C. S. Chung
Abstract
Registration of pre-operative and post-recurrence brain images is often needed to evaluate the effectiveness of brain gliomas treatment. While recent deep learning-based deformable registration methods have achieved remarkable success with healthy brain images, most of them would be unable to accurately align images with pathologies due to the absent correspondences in the reference image. In this paper, we propose a deep learning-based deformable registration method that jointly estimates regions with absent correspondence and bidirectional deformation fields. A forward-backward consistency constraint is used to aid in the localization of the resection and recurrence region from voxels with absence correspondences in the two images. Results on 3D clinical data from the BraTS-Reg challenge demonstrate our method can improve image alignment compared to traditional and deep learning-based registration approaches with or without cost function masking strategy. The source code is available at https://github.com/cwmok/DIRAC.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16446-0_3
SharedIt: https://rdcu.be/cVRSJ
Link to the code repository
https://github.com/cwmok/DIRAC
Link to the dataset(s)
https://www.med.upenn.edu/cbica/brats-reg-challenge/
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a deep learning-based deformable registration method for the pre-operative and post-recurrence brain MR registration. They use forward-backward consistency and inverse consistency to identify regions with absent correspondences and exclude them in the similarity measure. The proposed method is validated using the BraTS-Reg dataset.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
1 Pre-operative and Post-Recurrence Brain Tumor registration is an important research area. 2 The proposed strategy is sound
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1 1 The proposed method lacks novelty. It seems to be a combination of a few “tricks” from existing works. The forward-backward consistency is a widely used optical flow technique [19,21,28,29]. The inverse consistency is also a well-known constraint in the registration community.
2 Unfair comparison and incremental performance improvements. 1) Methods used in the comparisons are not designed for registering images with absent correspondence. MICCAI has a community dedicated to methodologies for pre-op and intra-op brain tumor images registration (CURIOUS’18, TMI), the authors should compare the proposed method to the best-performing methods in the CURIOUS challenge. 2) To the best of my knowledge, in Image-guided neurosurgeries, TRE (for landmarks near tumors) within 2mm is considered good. The TRE achieved by the proposed method is larger than 3mm, which might be too large to be clinically acceptable.
3 Lacks discussion potential. The proposed method doesn’t show clear innovations and contributions over the state-of-the-art methodologies. I’m doubtful that the MICCAI audience would be interested in this work.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Some important parameters, such as the threshold for the inverse consistency (in Sec2.2), might be data-dependent and have to be tuned.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
MICCAI audiences are likely not interested in papers with minor methodology novelty and incremental performance improvements. The authors may consider a workshop submission.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
2
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method doesn’t show clear innovations and contributions over the state-of-the-art methodologies. The performance improvement is incremental. It doesn’t seem to have any discussion potential at the conference.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
2
- [Post rebuttal] Please justify your decision
(1) The inverse-consistency is well known in conventional iterative registration. The forward-backward consistency has been used in optical flow community. This work uses both in image registration. I still think the originality of the proposed method is very limited.
(2) The CURIOUS is a multimodal registration challenge for pre-surgical brain (has tumor) and intra-operative brain (resected tumor). Many methods in CURIOUS challenge are developed to deal with the “absent correspondence”. As a result, to prove the practicality of this work, I still think it is fair to compare the proposed method to top methods in the CURIOUS challenge.
Review #2
- Please describe the contribution of the paper
The authors of the paper propose a novel method for deep learning based image registration of pre-operative and post-recurrence brain tumor MRI scans. The method automatically identifies regions where no correspondences can be established (e.g. due to tumor resection) and incorporates the resulting segmentations directly into the algorithm for masking the similarity measure and other parts of the loss function. Additionally, the method includes a constraint enforcing inverse consistency and is compared to several other algorithms showing superior registration results especially in near tumor regions.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper is very well structured and easy to follow
- The method is well classified into the state of the art
- The novel approach of estimating regions without correspondences is very interesting and well embedded into the overall approach
- No segmentations (e.g. of tumors) needed
- Strong evaluation with several compared algorithms (both conventional and deep learning based)
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The quality of estimated regions with absent correspondence could be evaluated in more detail, providing evidence for the proposition ‘The results demonstrate our method is capable of accurately locating the regions without valid correspondence’: How do the masks compare to automatically computed masks for pathological regions (that were used for ‘-CM’ baseline methods)? Can false-positive regions sometimes be a problem?
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The method is well described and the source code will be made publicly available after acceptance of the work. The data are not directly publicly available, but can be requested from the organizers of the competition. Therefore, the reproducibility is rated as good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- The last proposed method ‘DIRAC-D’ sounds promising, yielding slightly better results including additional MR modality as input. However, all other methods are only cmpared without this additional input
- It is still a bit unclear how the parameters alpha and p for the forward-backward consistency constraint are determined
- What are ‘successfully registered landmarks’? If there is a threshold applied, how is it determined?
- Typo on page 6: ‘…voxels are marked in m_{bf} and m_{bf}’, the second ‘m_{bf}’ should be ‘m_{fb}’
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors of the paper explain the challenging task of registration of images before and after tumor resection and propose an interesting and well explained novel method facing this problem. The paper is well structured and easy to follow, including an extensive evaluation that lacks only minor clarifications. Overall it is a very well written paper.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #3
- Please describe the contribution of the paper
This paper estimated the bidirectional deformation fields and located regions with absent correspondence, such as the tumor mask.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This work addressed the unsupervised deformable registration method for the preoperative and post-recurrence brain MR registration and tumor segmentation.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
It would be helpful to discuss the extraction of accurate masks in the proposed unsupervised learning scheme, considering the smooth regularization of registration fields. There lack some descriptions of the regularization term (Eq. 7) and mask threshold.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper has provided details about the models, datasets, and evaluation.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- It is interesting to use the bidirectional deformation field to define the mask of the tumor. However, it is suspicious to extract accurate masks in the proposed unsupervised learning scheme. It is unclear how to extract the exact boundary of the masks, considering the smoothness regularization of registration fields.
- Threshold \tau_{bf} is used to define the mask. It would be helpful to describe the threshold selection.
- The last term in Eq. 7 is used to regularize the mask to be small. Since the mask is represented by a vector or matrix, the appropriate norm is required.
- The proposed method was compared with those using cost function masking with the tumor core segmentation map. What did the cost function masking mean? Whether the prior mask of tumors was used in methods -CM. Table 1 showed that the proposed unsupervised approach outperformed the CM. It would be helpful to discuss the performance gain achieved compared with the CM.
- The proposed methods used invertible constraints for valid correspondence. Since the diffeomorphic registration provided invertible registration fields, using the existing diffeomorphic registration network as the backbone would be interesting.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper presented an unsupervised joint registration and segmentation framework, exploiting a forward-backward consistency constraint to estimate masks and registration fields. The proposed approach has been applied to the BraTS-Reg challenge dataset with performance gains over deep registration methods.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Contribution
The authors propose a deep learning-based deformable registration method for the pre-operative and post-recurrence brain tumour MRIs. They use forward-backward consistency and inverse consistency to identify regions with absent correspondences and exclude them in the similarity measure. The proposed method is validated using the BraTS-Reg dataset.
- Pre-operative and Post-Recurrence Brain Tumor registration is an important research area.
- The proposed approach involves a novel (according to two reviewers) approach for estimating regions without correspondences.
- The evaluation involved comparing against several algorithms (both conventional and deep learning based).
- The paper is well structured and easy to follow.
Weaknesses to address
- One reviewer had concerns regarding lack of clear innovations or novelty of the approach, because inverse consistency has been used in a number of previous works. Original contributions need to be emphasised more.
- This reviewer also had concerns about the other methods not being designed for registering images without clear correspondences, suggesting that the best-performing methods from the CURIOUS challenge should have been compared with instead.
- Perhaps justify whether the accuracies achieved are clinically acceptable by assessing the quality of estimated regions with absent correspondence in more detail and checking whether false-positive regions can sometimes be a problem.
- More discussion of the extraction of accurate masks in the proposed unsupervised learning scheme is needed, along with their dependencies on mask threshold and regularisation settings - which may be data dependent.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
7
Author Feedback
We thank all reviewers for their thoughtful comments. We categorize the major concerns (C) followed by responses(R) in the following paragraphs.
Reviewer#1 C1: Limited novelty. R: The reviewer may miss the most important points in this paper. First, our forward-backward (FB) consistency constraint can estimate smooth absent correspondence masks and the proposed inverse consistency does not penalize the inconsistency in regions with absent correspondence, which are different from the well-known constraint in the motion tracking and registration literature. Existing works leveraged forward-backward (FB) consistency to address the occlusion problem in motion tracking for 2D natural images, which focus on the small displacement between two successive frames and are not capable of solving the unique issues in brain tumour registration. Second, we propose a bi-directional large deformation image registration network, new FB consistency constraint, masked inverse consistency and an unsupervised learning paradigm from which we can address the large deformations and absent correspondences problems in the pre-operative and post-recurrence (PO-PR) brain MR registration without manual delineations, which is the first proposed to the best of our knowledge.
C2: Methods from the CURIOUS challenge should have been compared with. R: The CURIOUS challenge is about multi-modal registration of pre-surgical brain MRI to intra-operative ultrasound (iUS) scans, which is different from the PO-PR brain MR registration studied in this paper.
C3: In image-guided neurosurgeries, the accuracies achieved are not clinically acceptable/over the SOTA methods. R: The scope of our paper is not related to image-guided neurosurgery. Moreover, in this year’s ISBI BraTSReg challenge, the winner uses LapIRN and instance optimization for the challenge and we have included a comparison with LapIRN and its variant (-CM), which is dedicated to PO-PR brain registration, in Table 1 and Fig. 3, suggesting our method achieves SOTA accuracy among deep learning-based methods.
Reviewer#2 C4: The quality of estimated masks could be evaluated in more detail. R: The ground-truth regions with absent correspondence are not well-defined in the follow-up scans and are not necessarily equivalent to the pathological regions. For instance, the tumour core in the pre-operative scan and the resection cavities or recurred tumours in the follow-up scan, or the Edema presented in both scans may form a valid correspondence because of the similar spatial location and homogeneous appearance (in T1ce). Instead, we qualitatively evaluated the quality of estimated masks in Fig. 2 and supplementary material. The quantitative results in Table 1 and Fig. 3 demonstrated our masking strategy can improve the registration performance near the tumour regions, which is the main goal of the paper.
C5: How the hyperparameters for the FB consistency are determined? R: The alpha is determined by measuring the average FB error of the solutions in pathological regions using LapIRN. Our method is not sensitive to the choice of alpha since the first term in eq. 2 is adaptive to the registration complexity of each case and the FB error of regions with absent correspondence is much greater than that of the normal regions as shown in Fig. 2.
Reviewer#3 C6: It would be helpful to discuss the extraction of accurate masks. R: The ground truth of the absent correspondence regions is not well-defined. For details, please refer to the response in C4.
C7: Describe the mask regularization and threshold selection in eq. 7. R: The mask regulation in eq. 7 is the cardinality of the mask (Set theory). We will change the notation to the L1-norm. Thanks for the suggestion. For the threshold, please refer to the response in C5.
C8: What did the cost function masking (CM) mean? R: CM: Excluding the pathological regions in similarity measure during optimization [3,5]. Performance gains over CM are shown in Table 1.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After discussions about whether the validation was strong enough and whether it should include baselines that performed well in the CURIOUS challenge, Meta-reviewer 2 stepped in to explain that their baseline method had performed very well in the BratsReg challenge. Given that two reviewers are happy for the work to be accepted, with one reviewer (who wanted a comparison against CURIOUS methods) suggesting a rejection, I will go along with the majority.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
11
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
There was some discrepancy in review scores and a lively discussion among reviewers and area chairs. The paper definitely shows very promising results on the recent BRATS-reg challenge and proposes a solid solution for dealing with non-correspondence by masking the tumour region for the metric-loss and using inverse-consistency if no reliable image information is present. One review argues to extend the comparison to other algorithms that were used for CURIOUS - a related multi-modal challenge - but this is not a severe issue and can be left for future work (in fact some “CURIOUS”-algorithms may have already been applied to BRATS-reg)
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After extensive discussion, I believe the paper should be accepted.
I do believe that there are substantial outlying problems that the authors should try to address in the camera ready. Improve clarity as discussed, include a discussion of the curious challenge and the potential extent/utility of the method in different applications, details about the mask/thresholding aspect, etc.
Congratulations to the authors.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR