Authors

Karthik Gopinath, Douglas N. Greve, Sudeshna Das, Steve Arnold, Colin Magdamo, Juan Eugenio Iglesias

Abstract

Surface analysis of the cortex is ubiquitous in human neuroimaging with MRI, e.g., for cortical registration, parcellation, or thickness estimation. The convoluted cortical geometry requires isotropic scans (e.g., 1mm MPRAGEs) and good gray-white matter contrast for 3D reconstruction. This precludes the analysis of most brain MRI scans acquired for clinical purposes. Analyzing such scans would enable neuroimaging studies with sample sizes that cannot be achieved with current research datasets, particularly for underrepresented populations and rare diseases. Here we present the first method for cortical reconstruction, registration, parcellation, and thickness estimation for clinical brain MRI scans of any resolution and pulse sequence. The methods has a learning component and a classical optimization module. The former uses domain randomization to train a CNN that predicts an implicit representation of the white matter and pial surfaces (a signed distance function) at 1mm isotropic resolution, independently of the pulse sequence and resolution of the input. The latter uses geometry processing to place the surfaces while accurately satisfying topological and geometric constraints, thus enabling subsequent parcellation and thickness estimation with existing methods. We present results on 5mm axial FLAIR scans from ADNI and on a highly heterogeneous clinical dataset with 5,000 scans. Code and data are publicly available at https://surfer.nmr.mgh.harvard.edu/fswiki/recon-all-clinical

Link to paper

DOI: https://doi.org/10.1007/978-3-031-43993-3_4

SharedIt: https://rdcu.be/dnwM5

Link to the code repository

https://surfer.nmr.mgh.harvard.edu/fswiki/recon-all-clinical

Link to the dataset(s)

N/A

Reviews

Review #2

Please describe the contribution of the paper

The goal of the authors is to create a method that performs cortical geometry reconstruction from clinical brain MR scans without regard to resolution or pulse sequence. This technique would vastly increase the data available for analysis with tools such as Freesurfer. The proposed technique predicts a signed distance map to the white matter and pial surfaces at 1mm independent of acquisition sequence and resolution.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- the construction of synthetic scans utilizing domain randomization seems to be an excellent approach to creating training data simulating lower resolution from high resolution inputs
- while full data analysis was lacking (see weaknesses), the qualitative results presented seem very promising.
- the technique presented would enable clinical data that is currently unusable for population analysis to be useful, rather than having to be discarded
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- the authors made no attempt to emulate common clinical pulse sequences and instead used gaussian mixture models from the input T1. Using GMM from, say, T2 or FLAIR sequences, may have improved the results
- the presented results are largely qualitative (Fig 3, 4, 5a). A table of regions and bland-altman plots of thicknesses would enable objective evaluation of the results
- the clinical dataset was not described as to resolution and pulse sequence. Perhaps input resolution could explain some of the wide variance in Fig 5b relative to 5c
- would have liked to see dice scores plotted against resolution and pulse sequence. Then the applicability of the algorithm to particular acquisitions could be judged
- the details of the CNN architecture were not presented
- an ethics statement on the clinical data is missing
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The checklist indicates that descriptions of algorithms, models, hyper-parameters, failure modes, etc are available. If provided with this information, the reproducibility of the paper should be high.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html

This paper lacked some important details. Results were largely presented as qualitative and compared to an existing technique. The existence in the clinical dataset of 1mm MPRAGE exams was a great opportunity to compare the gold standard to the output of the algorithm. A simple table or Bland-Altman plot would have demonstrated the degree of fidelity the CNN achieved. The plots of thinning (5b-c) only served to question if the model was useful. A short description of the CNN, training hardware, and training time would have been helpful to gauge reproducibility for other researchers.

Overall this paper was clearly presented and focused on one specific technique, that of inferring a signed distance function from arbitrary clinical scans. The focus and limited scope make this a good paper overall.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I liked this paper very much for it’s focus and applicability. The potential to use clinically acquired data in large scale neuro-analysis is important. However, I felt the paper lacked solid quantitative data to back up the claim of using previously precluded clinical data. I think the technique has merit and promise, but simply wasn’t evaluated in a thorough manner.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

This paper describes an to perform cortical reconstruction from MR brain images regardless of contrast or spatial resolution. The approach builds off of the SynthSeg and SynthSR methods that use domain randomization to improve robustness to contrast and resolution by simulating large amounts of training data with different properties. Instead of outputting a segmentation or T1 weighted image as in SynthSeg and SynthSR, a signed distance function (SDF) is generated representing the cortex. The method is evaluated by comparing cortical parcellation accuracy from 5mm FLAIR images to that obtained on 1mm MPRAGE, examining effect sizes in a study of cortical thickness in Alzheimer’s disease, and in an aging study using clinical quality MRI.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The ability to apply cortical reconstruction to any type of structural MRI is powerful.
- Tested on a large number of data sets.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The evaluation is missing a comparison to 1) running SynthSeg with the cortical reconstruction steps from FreeSurfer (replacing the initial segmentation steps), and 2) running SynthSeg but computing the SDF directly from the output segmentation and then running the proposed geometry processing.
- There is no real evaluation of the accuracy of the cortical surface. It would have been helpful, for example, to apply it to the 1mm MPRAGE data and compare it directly to FreeSurfer.
- The novelty is somewhat limited. Most of the steps are slight modifications from SynthSeg in the first step, and FreeSurfer in the second step.
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Excellent, code and data are publicly available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Although it is interesting to apply this approach to clinical data, I would have been more interested to see its performance in research quality multi-site data. The ability to get consistent measurements without additional harmonization steps would be very valuable.
- More of a comment than a criticism, an approach like this that can be applied to a wide variety of MR images could use some kind of confidence output to effectively warn the user about the quality of the reconstruction estimate. As noted, lower resolution FLAIR images led to likely inaccurate measurements in the frontal cortex, and the cross-sectional aging study had high variability.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

4
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

I appreciate the large amount of data that was processed in the evaluation but the novelty is somewhat limited for a MICCAI paper, and the evaluations seemed to miss the mark in terms of demonstrating that the results are accurate and superior to using SynthSeg.
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This study presents a method for cortical reconstruction, registration, parcellation and thickness estimation of MRI scans regardless of the resolution and pulse sequence. The method includes a convolutional neural network (CNN), which estimates the signed distance functions as a representation of the white matter and pial surfaces, and a classical geometry processing module to place the surfaces and allow parcellation and thickness estimation. The proposed method is evaluated on MRI scans from ADNI and a clinical data set from the local hospital.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed method seems reasonable for the task and achieves good results with accurate parcellation.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The paper lacks a detailed discussion/ comparison on the related work with respect to the proposed method
- The results produced on the clinical dataset, for thickness measurements showed variability compared to Freesurfer, and the data were noisy. Therefore, there are limitations for clinical translation and real world impact
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors provide code for this submission. This will help increase the reproducibility of the paper.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- The type of scans included in the clinical dataset are not shown. A breakdown of the sequences and parameters would have been useful to provide a clear setting about the experimental protocol
- The authors provided some details of some of the downstream tasks (e.g. group study between AD subjects and elderly controls etc), but important information is missing and the paper did not deliver clear information about the tasks.
- Some more work in the presentation of the results would have helped the reader to more clearly interpret the findings
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

This method is interesting, targeted towards a highly needed task, and some of the results look effective.
Reviewer confidence

Confident but not absolutely certain
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
- interesting approach of construction of synthetic scans using domain randomization
- clinical sequence parameters could be incorporated as mentioned by reviewer 1
- details about CNN and data need to be added
- reviewers have made sme suggestions about improving validation
- reviewer 2 has concerns about related works.

Author Feedback

N/A

back to top

Cortical analysis of heterogeneous clinical brain MRI scans for large-scale neuroimaging studies