List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Jianwei Zhang, Yonggang Shi
Abstract
Cortical thickness is an important biomarker associated with
gray matter atrophy in neurodegenerative diseases. In order to conduct
meaningful comparisons of cortical thickness between different subjects,
it is imperative to establish correspondence among surface meshes. Conventional
methods achieve this by projecting surface onto canonical domains
such as the unit sphere or averaging feature values in anatomical
regions of interest (ROIs). However, due to the natural variability in cortical
topography, perfect anatomically meaningful one-to-one mapping
can be hardly achieved and the practice of averaging leads to the loss
of detailed information. For example, two subjects may have different
number of gyral structures in the same region, and thus mapping can
result in gyral/sulcal mismatch which introduces noise and averaging in
detailed local information loss. Therefore, it is necessary to develop new
method that can overcome these intrinsic problems to construct more
meaningful comparison for atrophy detection. To address these limitations,
we propose a novel personalized patch-based method to improve
cortical thickness comparison across subjects. Our model segments the
brain surface into patches based on gyral and sulcal structures to reduce
mismatches in mapping method while still preserving detailed topological
information which is potentially discarded in averaging. Moreover,the
personalized templates for each patch account for the variability of folding
patterns, as not all subjects are comparable. Finally, through normality
assessment experiments, we demonstrate that our model performs
better than standard spherical registration in detecting atrophy in patients
with mild cognitive impairment (MCI) and Alzheimer’s disease
(AD).
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43904-9_6
SharedIt: https://rdcu.be/dnwGJ
Link to the code repository
N/A
Link to the dataset(s)
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents a personalized patch-based method for cortical thickness comparison without using the correspondence through spherical registration.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper gives a novel method of comparing cortical thickness directly using cortical 3D surface information, avoiding the dense matching of different cortical topographies.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
It is a good trial to work on original surface information. But the whole work needs more detailed verification and validation in terms of both methodology and experiments. The computation of patch is unclear. Comparison is limited.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
More experimental details are needed, such as how many times of running in terms of computing the final rates in Table 1?
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Some questions are listed as follows:
Page3: “[?]”
Eqn (2) : x \in V, not v
The computation of patch is unclear. How to select the center vertex, how many Voronoi cells or patches, or center vertices in total?
Comparison is only on the method of different data representation, spherical surface mapping. Lack of description of and comparison to the state-of-the-art results, including other traditional or mapping methods.
Why to select 200 nearest patches for each patch, and use the top 50 patches, not other numbers? Maybe the last 20 patches in the 50 ones are not similar or meaningful any more.
z-score: what kind of patch would be abnormal? Lack of definition and verification of z-score value and its corresponding abnormality.
It is a good trial to work on original surface information. The whole work needs more detailed verification and validation in terms of both methodology and experiments.
Table 1: the rates are usually computed on average of like many times of running. Here, lack of details.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The advantage of the proposed method was not validated with state-of-the-art methods. In addition, the work is lack of algorithmic and experimental details.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
3
- [Post rebuttal] Please justify your decision
The authors’ rebuttal didn’t solve the problems I mentioned. The paper needs major revision. I keep my original decision.
Review #2
- Please describe the contribution of the paper
This work proposed a novel personalized patch-based method for brain atrophy detection by matching segmented patches based on gyral/sulcal label, location and shape similarity and constructing personalized template set for abnormal detection. Experimental results demonstrate that the proposed method is more effective at detecting brain atrophy.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of the paper is a novel formulation for patch-based method for brain atrophy detection. Conventional method uses spherical brain mapping to find correspondences between brain patches, which introduce significant mismatches. The proposed method match segmented patches based on gyral/sucal label, location and shape similarity and construct personalized template set, which improves the effectiveness.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
It will be more helpful to explain the computational cost for constructing personalized template. For each patch, a personalized set of template is chosen, the time complexity should be analyzed.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The algorithmic details and the mathematical formulae are well explained. The testing data sets are public available, so the work is easy to be reproduced by a graduate student.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The paper is well written, some details can be added to further improve the writing:
The computational complexity for the segmentation, constructing the personalized templates can be further explained. The consistency and the stability of the segmentation can be further analyzed. The formulae and the intuition for shape index can be explicitly given. The reference to the fast marching algorithm is missing.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Conventional brain atrophy detection heavily depends on spherical registration. Since the cortical structures varies from person to person, conventional methods suffer from big mismatches. This work introduces the personalized template method, which mitigate the problem and improves the effectiveness. The proposed method is promising.
The paper is well written, all the concepts, algorithmic details, mathematical formulae are well explained. It is easy to follow and reproduce. The experimental results are convincing.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper highlights the significance of cortical thickness as a biomarker for Alzheimer’s disease and the difficulties in comparing cortical thickness across different individuals due to variations in cortical topography. Conventional mapping techniques, such as projecting onto canonical domains or averaging in anatomical regions of interest, may result in loss of detailed information and noise. To address these challenges, the authors propose a personalized patch-based approach that minimizes mismatches in mapping and preserves topological details, resulting in improved detection of atrophy in patients with mild cognitive impairment and Alzheimer’s disease compared to standard spherical registration.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The paper tackles the gyral/sulcal mismatch problem that commonly occurs in cortical thickness comparison. This is an important problem in neuroimaging data analysis.
- The proposed method matches gyral and sulcal patches to their respective regions and selects a personalized set of templates for each patch. This ensures that only comparable data are measured together, increasing sensitivity in brain atrophy detection while reducing noise introduced by mismatching.
- The proposed method was evaluated on a portion of ADNI dataset by showing normality assessment and binary classification between CN and non-CN subjects.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The patch merging process in the proposed method seems order dependent and non-deterministic due to the greedy algorithm used to select initially extracted patches.
- Some of the methodological designs are unclear due to the lack of explanation.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
While the proposed method appears to be reproducible, it would be helpful to include certain important information, such as the runtime and parameters used for sulcal skeleton extraction, to ensure that others can easily replicate the results.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The paper presents a method to tackle the problem of gyral/sulcal mismatch in cortical thickness comparison, which is a significant issue in neuroimaging data analysis. By matching gyral and sulcal patches to their respective regions and selecting personalized templates for each patch, the proposed method improves sensitivity in brain atrophy detection while reducing noise. However, some aspects of the method require clarification, as outlined in the comments below.
Major concerns:
- The patch merging process in the proposed method seems to have order dependency on selecting the initially extracted patches, leading to non-deterministic resulting patches due to the algorithm’s greedy approach. It would be helpful if the authors could provide some comments on the stability of the algorithm and its sensitivity to the order of the patches.
- Could the authors please provide a rationale for their choice of the chi-square distance as the similarity metric in this work, as it is currently unclear in the manuscript.
- Section 3.1 is somewhat difficult to follow, and the description of the personalized templates in Section 2.3 raises some questions. For instance, how was the sphere-based z-score calculated? What does “the spherically matched vertices” mean? In Fig. 3, the sphere-based map also uses patches, but it is unclear what “sphere-based score” means in this context.
- While the proposed method appears promising, the success of the approach may depend on the consistency of patch extraction, which can be challenging in cases where sulcal delineation fails, leading to patch under-segmentation. To further justify the proposed method and demonstrate its robustness, it would be valuable to validate it on a multi-scan dataset such as https://www.nitrc.org/projects/multimodal/ and demonstrate its ability to handle different levels of noise. However, given the page limit, it may not be feasible to include such additional validation in this conference paper. I suggest the authors consider conducting these experiments in future studies.
Minor concerns:
- It would be beneficial to see how the proposed method performs with different bin sizes and varying numbers of patches with the largest similarity scores.
- The manuscript requires editing to address several typos (e.g., “sulca”) and spacing issues between sentences. Also, there is a reference missing on page 3 that needs to be included.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the paper proposes a promising new approach to analyzing cortical folding patterns using a patch-based method, there are some areas of concern. However, I believe that the merits of the paper outweigh its weaknesses.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
The authors developed a method for segmenting cortical regions and comparing the cortical thickness of those regions between groups. The proposed methodology segments the cortex, combines nearby segments, compares segments to similar segments in congnitive normal subjects, and then creates a personalized template set for each subject.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors offer an interesting methodology of comparing surface regions between subjects without requiring surface registration. The authors’ methodology seems to offer increased predictive accuracy of MCI.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The authors validate their methodology by comparing the abnormality in cortical thickness (CT) measurements between FreeSurfer and their approach in MCI and AD subjects. However, they do not explain why they think abnormality is a good validation measure. AD has been reported to be associated with decreased CT (see https://www.sciencedirect.com/science/article/pii/S2213158216300936#f0010, for example), not just abnormal CT. Therefore, using abnormality of CT rather than CT decrease for validation is a confusing measurement choice.
The authors compare their results to only one method for surface registration, Freesurfer, which was developed in 1999. While Freesurfer is still used, many other methods for surface registration have been developed since then and have demonstrated improved performance over FreeSurfer: https://www.sciencedirect.com/science/article/pii/S1053811914004546?via%3Dihub https://www.sciencedirect.com/science/article/pii/S1053811917308649?casa_token=9fZJQMB3U1QAAAAA:_8SV7zk2DP_0x0g7DOq0ePMFpyWhENId2BnmkzrMQAfTd3qLWgakm6M2GPtztbOzRI63uEa0MJY https://www.sciencedirect.com/science/article/pii/S105381191500097X https://www.sciencedirect.com/science/article/pii/S1361841518308016?casa_token=32kltDYTxNYAAAAA:Cgux7q7gS1YQujyXYI9HbbOct17SL_7bXOSPBPtcZ-1lAVdB0_y1PETGouTtfXYtySYjBW05qCM https://www.sciencedirect.com/science/article/pii/S1053811920306479 https://ieeexplore.ieee.org/abstract/document/5223581?casa_token=IUP50-mqzmIAAAAA:eyv6WyScERwF2ahpviXkSikFf5X9rDfc60Nzj4QF_4jwydGB3uQ7ODggjc2A6Qk8GfGf0ty2lAI
Some are also easily accessible, such as FSL’s MSM (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/MSM) and Spherical Demons (https://sites.google.com/site/yeoyeo02/software/sphericaldemonsrelease).
Without comparisons to more modern approaches to surface registration, it is difficult to assess whether the proposed methodology offers any improvement in over spherical registration techniques as the authors claim.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors seem to detail their methodology to an extent that the results should be reproducible. However, the authors do not intend to share the code used to derive their normalized cortical measures.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Page 3 has a missing citation: “[?]”.
Why did you take the absolute value of the patch based z-score? AD should have reduced cortical thickness (https://academic.oup.com/psyrad/article/2/3/113/6835627), but it is difficult to tell based on your validation if your method produces reduced cortical thickness in these subjects or just abnormal cortical thickness.
Could you show the results of the comparison in z-scores between your method and FreeSurfer for the CN subjects? Does your methodology simply produce more abnormal CT measurements in all subjects?
In Table 1, providing other metrics such as sensitivity, specificity, and AUC in addition to accuracy would help reinforce the improved predictive ability of your methodology.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the idea behind using a technique that avoids surface registration is somewhat interesting, the paper failed to convince me that their methodology offered improved performance over surface registration techniques due to authors’ choice to use abnormality instead of a standard z-score. The authors also did not convince me that avoiding surface registration is even necessary, therefore, calling into question the need for this research.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors introduce a novel method for evaluating cortical thickness abnormalities without using cortical surface spherical registration. Registration based methods can often suffer from misalignment. The novel method is interesting, but the paper could be enhanced with further validation. The evaluation of results is rather short and a discussion of the patches with abnormalities compared to previous AD research is lacking.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposes a method for matching gyral and sulcal patches to their respective regions as a personalized template. This is a crucial problem regarding accurate cortical thickness and assessment of brain atrophy. On the other hand, some major critiques from reviewers regarding method descriptions and experiments need to be discussed in the rebuttal. However, it should justify how the provided experiments are sufficient to validate the proposed method. In a similar context, the rebuttal should include answers to those critiques: The motivation of this method over state-of-the-art surface matching methods (refer to examples suggested by Reviewer 4)?; Details of experiments should be clarified, including the z-score definition (suggested by both Reviews 1 and 4); How to ensure the robustness of patch extraction (reviewer 3)? How can we tell if the proposed method outperforms Freesuffer (and other surface-based methods).
Author Feedback
First, we would like to thank all reviewers for their time and valuable comments on our work. Following the suggestion of the meta-reviewer, we respond below to major reviewer comments and denote reviewers by R1, R2, R3, R4 and RM for meta-reviewer:
- Motivation of proposed method over SOTA surface matching. (R4, RM)
The main motivation of our research is the widespread presence of gyral/sulcal mismatch in existing surface matching methods that focus on establishing pointwise one-to-one correspondences between cortices. While mathematically elegant, these methods cannot account for topographic mismatch of sulcal/gyral pattens in cortical anatomy that naturally exist between subjects. As shown in figure 1, the misalignment of sulcal/gyral regions is a common issue and can occur in about 30% of the gyral and sulcal regions based on our analysis of the ADNI3 dataset. Because of different cortical thickness distributions in sulcal/gyral regions, this is no doubt a serious problem that can significantly affect our power in the accurate detection of brain atrophy in Alzheimer’s disease. While we focus on our comparison with FreeSurfer, this is a common challenge for all previous diffeomorphic surface mapping methods. Our method represents an alternative strategy to identify patch-based and anatomically more meaningful correspondences for brain atrophy detection. Our results provide a promising starting point for this line of research and pave the way for more thorough evaluation in future work.
Robustness of patch extraction process. (R3, RM) We want to clarify that the patch-merging process is not order dependent. The VCs can be considered as nodes in a graph and edge between nodes has the same length as geodesic distance between VCs’ centers. Our algorithm starts from a VC to merge neighboring VCs if the connecting edge length is less than a threshold. After that, it performs breadth-first search until all neighboring edges of the merged component is bigger than the threshold. To continue this process, our algorithm starts from a random unvisited VC and repeats the process until all VCs are visited. The whole process is essentially a connected component search. Therefore, different starting VC will yield the same merging result.
Z-score definition and results for CN. (R1, R4, RM) We want to clarify the use of the absolute z-scores in our work. While the decrease of cortical thickness (CT) is often considered as a biomarker of brain atrophy, abnormal increase of CT is also commonly observed for multiple reasons. For example, white matter degeneration can alter the white matter surface boundary and hence cause an abnormal increase of CT. We thus used the absolution value of z-scores to account both abnormal decrease and increase of CT. Because brain degeneration also exists in CN subjects, our method can also enhance the detection of abnormal cortical thickness changes (increased absolution z-scores) in CN, but less so than MCI and AD subjects as demonstrated by the enhanced SVM-based classification performance in Table 1.
Performance comparison and relation to other methods. (R1, R4, RM) As mentioned in question 1, gyral/sulcal mismatch is common among all surface registration methods, therefore, our method is an alternative way of utilizing the registered coordinates from any surface registration methods. In our experiments, we used FreeSurfer spherical registration results to demonstrate that our method can be applied to improve the correspondence between surfaces and the detection power for brain atrophy. Similar to the combination of our algorithm with FreeSurfer, we can seamlessly integrate with other surface mapping methods by using the registered spherical coordinates from those methods for the search of matched patches across subjects for enhancing patch-based brain atrophy detection.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The proposed method presents cortical thickness abnormality assessment by not relying on cortical surface spherical registration, which has novelty. Reviewers addressed most of the review critiques in the rebuttal, such as details of patch extraction and motivation for using Z-score. This paper has value in neuroimaging studies and the MICCAI community.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper proposes a method to address topographic mismatches between gyri and sulci by constructing a personalized set of templates for each patch. The paper is aims to address a challenge in neuroimaging analysis. However, it suffers from two main weaknesses. First, there is lack of description of and comparison to the state-of-the-art results, including other traditional or mapping methods. The baseline that has been used is not appropriate. Direct comparison with freesurfer results in an fsaverage would more appropriate. Second, the reported results that quantify classification accuracy of an SVM for both sphere- and patch-based features are very poor, greatly limiting the interest for the proposed approach. These concerns were also raised by reviewers and were not addressed satisfactorily. Taken together, these greatly limit the interest for the paper.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The proposed method does not give significant performance improvement and relies on FreeSurfer based surface correspondence. These flaws do not warrant acceptance of the paper in its current form.