List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Rodrigo Santa Cruz, Léo Lebrat, Darren Fu, Pierrick Bourgeat, Jurgen Fripp, Clinton Fookes, Olivier Salvado
Abstract
The problem of Cortical Surface Reconstruction from magnetic resonance imaging has been traditionally addressed using lengthy pipelines of image processing techniques like FreeSurfer, CAT, or CIVET. These frameworks require very long runtimes deemed unfeasible for real-time applications and unpractical for large-scale studies. Recently, supervised deep learning approaches have been introduced to speed up this task cutting down the reconstruction time from hours to seconds. Using the state-of-the-art CorticalFlow model as a blueprint, this paper proposes three modifications to improve its accuracy and interoperability with existing surface analysis tools, while not sacrificing its fast inference time and low GPU memory consumption. First, we employ a more accurate ODE solver to reduce the diffeomorphic mapping approximation error. Second, we devise a routine to produce smoother template meshes avoiding mesh artifacts caused by sharp edges in CorticalFlow’s convex-hull based template. Last, we recast pial surface prediction as the deformation of the predicted white surface leading to a one-to-one mapping between white and pial surface vertices. This mapping is essential to many existing surface analysis tools for cortical morphometry. We name the resulting method CorticalFlow++. Using large-scale datasets, we demonstrate the proposed changes provide more geometric accuracy and surface regularity while keeping the reconstruction time and GPU memory requirements almost unchanged.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16443-9_48
SharedIt: https://rdcu.be/cVRy2
Link to the code repository
https://bitbucket.csiro.au/projects/CRCPMAX/repos/corticalflow/browse
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
In this paper, the authors propose an upgraded cortical surface reconstruction pipeline upon CorticalFlow. The efforts exist in three perspectives: a more accurate ODE solver, a sommther template generation routine, and a deformation process from white surface to pial surface.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This is a technical submission addressing a fundamental but important problem in brain morphometric analysis, cortical reconstruction. The major novelty lies in the improvement of pipeline and practical application scenario. The proposed work competes with several latest main-stream methods, and shows a strong performance gain.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
(1) Motivation of using fourth-order RK approximation is less discussed. The accuracy is theoretically expected to be better than Euler method, but isn’t RK4 an iterative method, too? Is backward Euler or RK2 a reasonable option (esp backward Euler)? Comparing with these popular methods is welcomed. (2) Concerns on the generalization. Is the proposed method sensitive to the inputs? For example, if model is trained with high-resolution MRI scans, what’s its performance on lower-resolution MRI scans (eg, train with ADNI2, test on ADNI1)? Such aspects are expected to be discussed. (3) What about the training efficiency? Testing efficiency is well demonstrated, but the training efficiency is not given. Please provide more information about the training process, such as convergence information, data preprocessing, epochs used to train, etc. As far as I know, RK4 converges slower compared with forward Euler, you have to set a larger time step for RK4, so I’m interested in its actual training efficiency in this task. (4) Not see in the submission if the code will be released to public.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The general idea is expressed clearly, but I didn’t see from the submission that the code will be released somewhere.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Besides the weaknesses mentioned in Sec 5, here some other questions: (1) About white-to-pial surface morphing, considering the thickness of the gray matter is itself not a “big” number, so a “small” deformation might be a large influence. I wonder if any constraints on the thickness are taken into consideration. (2) Question about registration of scans. FreeSurfer will do the registration, I wonder if your method will set a canonical template and mapping all inputs to this template.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I would like to hear from the authors about my questions. I have concerns on the feasibility of the proposed method.
- Number of papers in your stack
3
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Thanks for the rebuttal. Looking forward to the released code.
Review #2
- Please describe the contribution of the paper
This paper improves the cortical flow method in the following 3 aspects:
- Using the Runge-Kutta method to replace the forward Euler method to improve computation efficiency and the approximation accuracy.
- Using a genus zero smooth mesh templates to replace the convex-hull template in the cortical flow method.
- Using the predicted white surface to further predict the pial surface, which can build the vertex-wise correspondence between white and pial surface.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Accurately located the weaknesses of the cortical flow method and improved these weaknesses correspondingly.
- The paper organization and writing is quite clear, which makes the paper easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- These improvements are incremental.
- Although the weakness of convex-hull is explained, however, using the proposed smooth templates is dependent on the brain size. And the improvements contributed to this template is not discussed or validated.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The reproducibility of the paper is good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please justify and discuss whether the size of the brain have influences on the template generation. Also, if the test brain is larger than all the training brain, would the method still work? The current paper mainly validated the Runge-Kutta improvement, but not discussed the influence of the template improvement. The author claims the template improvement is a contribution, so it need some addition discussion to justify.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well written and the proposed method is easy to follow.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
The paper presents cortical surface reconstruction using neural nets. The underlying architecture follows what was originally proposed in CorticalFlow. The authors propose the RK4 ODE solver, cortical shape-like template, and pial reconstruction from the WM surfaces. The experimental results show improved performance over the original CorticalFlow and also compare with a single baseline method (PialNN).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed method extends [15]. The authors improves each component in [15] for better cortical surface reconstruction.
-
The RK4 ODE solver was used for accurate estimation of the solution to the flow ODE. It is well-known that RK4 can offer a more accurate solution than the Euler forward method.
-
The union of training volumes was used as an initial template. This approach may better capture deep sulci by placing the initial vertices close to the cortical tissue.
-
Pial reconstruction was done by employing the estimated white surface rather than a convex hull template. This idea has been widely adapted in the classic surface reconstruction pipelines and generally offers better CSF/GM boundaries.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
See below.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It seems reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
The motivation is great in the proposed template reconstruction. However, in Fig. 1, the actual template does not make a huge difference from the convex hull template in visual inspection. This is perhaps because the average across all the training samples. Although the authors show some examples in Fig. 1, it is still hard to see which regions have indeed a merit using this template.
-
The white to pial mapping is a good approach, but this idea is already adapted in many classic surface reconstruction pipelines (e.g., FreeSurfer, CIVET, etc.). Please acknowledge this strategy in the manuscript.
-
RK4 generally requires more time and memory requirement. It is a surprise that the computing burden is negligible on switching Euler forward to RK4. In particular, memory consumption presumably remains unchanged from Table 1. Could the authors discuss more about this?
-
One baseline method is insufficient for validation. I understand the dataset is overlap with [15], but Table 2 does not appear in [15].
-
Why do the inference time and required GPU memory differ from [15]?
-
The current evaluation is performed under the assumption that FreeSurfer is ground-truth. Nevertheless, the surface reconstruction quality can be validated on other metrics such as manual landmark placement, geodesic distance of the landmarks on the reconstructed surfaces, etc. As this is a conference paper, there might be no enough room.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
See above.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper presents an improved cortical surface reconstruction using neural nets based on the original CorticalFlow. Major improvements include a more accurate RK4 ODE solver, a smoother cortical shape template generation routine, and a deformation process from white surface to pial surface. Strengths:
- This is a technical submission addressing a fundamental but important problem in brain morphometric analysis, cortical reconstruction.
- The paper organization, motivation and writing is quite clear, which makes the paper easy to follow.
- The experimental results show improved performance over the original CorticalFlow and also compare with a single baseline method (PialNN). Weaknesses:
- Triangle self-intersection numbers have been greatly reduced, however, given the hundreds of thousands of triangles in each cortical surface, there are hundreds of self-intersections in each reconstruction cortical surface. Thus, its practical usefulness in neuroimage analysis is unclear.
- These improvements are incremental. The white to pial mapping is a good approach, but this idea is already adapted in many classic surface reconstruction pipelines (e.g., FreeSurfer, CIVET, etc.).
- Concerns on the generalization, e.g., images with different resolutions/quality or brains with different sizes.
- The current evaluation is performed under the assumption that FreeSurfer is ground-truth.
- Only one baseline method is used for validation.
- It is unclear on the code’s public availbility.
Please address these weaknesses in rebuttal.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Author Feedback
Q1-Concerns on the generalization (e.g., MRI resolution, quality, and brain sizes). CorticalFlow++ demonstrates effective generalization for scanner models and MRI resolutions. On the OASIS3 dataset, despite being trained only on ADNI MRIs of 1.5T, the model performs equally well on 1.5T and 3T MRIs of three different scanners (i.e. chamfer distance of 0.50, 0.55, 0.53 for BioGraph 3T, Sonata 1.5T, and TrioTrim 3T, respectively). Regarding brain sizes, the proposed template mesh is generated from a large dataset of affine registered MRIs to a brain MRI template and this same registration is performed at test time. Therefore, upon the convergence of this affine registration, a test brain MRI will fit inside of the template mesh and CorticalFlow++ will produce anatomically correct cortical surfaces. Q2-There are hundreds of self-intersections in the reconstructed cortical surface. The accuracy of the computation of self-intersecting faces depends on the mesh resolution. For cortical surface meshes composed of densely packed triangles, we observe a lot of false-positive self-intersecting faces. Also, using the same self-intersection pipeline one can find a similar rate of self-intersecting faces for Freesurfer meshes that are known to be self-intersection free. Regarding practical usefulness, since CorticalFlow++’s surfaces are accurate, regular, and present white-to-pial mapping, they can replace Freesurfer’s surfaces in many neuroimage tasks (e.g. cortical thickness analysis) while just requiring a very modest processing time. Q3-Motivation for using RK4 and its computational implications. The number of vertices for which we are solving an ODE does not allow us to use implicit methods such as backward Euler. The lower order RK scheme could be used (explicit Euler is RK1) at the cost of a higher approximation error. From our experiments, RK4 offers the best tradeoff between accuracy and computational cost. Also, we observe that RK4 is only 77.04 ms slower and consumes 2.62 MB more GPU memory than the Euler method for the same step size. This overhead is negligible compared to other components of the model (e.g. UNets). Q4-Training hyperparameters and efficiency. Similar to our baseline, we follow the data preprocessing steps from [23] and we train each deformation block for 70k iterations with a batch size of 3 MRIs using the ADAM optimizer with an initial learning rate of 0.0001. Regarding training efficiency, a training iteration of CorticalFlow++ with three deformation blocks is only 1.75 s slower and consumes 1.4 GB more GPU memory than its baseline. However, it provides an average reduction of 19.11% in terms of Chamfer distance and 56.77% in terms of the percentage of self-intersecting faces. It also reduces mesh artifacts as shown in Fig. 2. Finally, at test time, the computational overhead is negligible (0.54 s). Q5-Comparison to other baselines. Comparisons to other methods can be extracted from [15] since we use the same benchmark. We report only the best-performing methods due to the page limit. Q6-Code public availability. We will release the code, mesh templates, and trained models. Q7-The proposed template looks similar to the convex hull template. What are the performance improvements? The proposed template mesh does not present sharp edges like the convex-hull template. It is also “tighter” around the pseudo-ground-truth surfaces. Thus, the proposed template slightly improves the quantitative metrics (compare CorticalFlow and CorticalFlow + NEWTPL rows in Table 1) and suppresses mesh artifacts (see Fig. 2). Q8-Are thickness constraints taken into consideration? No thickness constraint is explicitly enforced. Even though, no white-pial surface crossing was spotted in our experiments. Q9-Why do the inference time and required GPU memory differs from [15]? In [15], the authors report the runtime and GPU memory usage only for the network prediction, while we report these metrics for the entire surface generation
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal clarified some questions. Although the meta-reviewer cannot agree with the rebuttal on the sel-intersection issues, it is still suggested to accept this paper with merits.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The rebuttal addressed key concerns on generalization and experimental validation. The reviewers reached consensus on acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
In light of the rebuttal, all reviewers have recommended weak acceptance.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4