List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yuqian Chen, Fan Zhang, Chaoyi Zhang, Tengfei Xue, Leo R. Zekelman, Jianzhong He, Yang Song, Nikos Makris, Yogesh Rathi, Alexandra J. Golby, Weidong Cai, Lauren J. O’Donnell
Abstract
White matter tract microstructure has been shown to influence neuropsychological scores of cognitive performance. However, prediction of these scores from white matter tract data has not been attempted. In this paper, we propose a deep-learning-based framework for neuropsychological score prediction using microstructure measurements estimated from diffusion magnetic resonance imaging (dMRI) tractography, focusing on predicting performance on a receptive vocabulary assessment task based on a critical fiber tract for language, the arcuate fasciculus (AF). We directly utilize information from all points in a fiber tract, without the need to average data along the fiber as is traditionally required by diffusion MRI tractometry methods. Specifically, we represent the AF as a point cloud with microstructure measurements at each point, enabling our adoption of point-based neural networks. We improve prediction performance with the proposed Paired-Siamese Loss that utilizes information about differences between continuous neuropsychological scores. Finally, we propose a Critical Region Localization (CRL) algorithm to localize informative anatomical regions containing points with strong contributions to the prediction results. Our method is evaluated on data from 806 subjects from the Human Connectome Project dataset. Results demonstrate superior neuropsychological score prediction performance compared to baseline methods. We discover that critical regions in the AF are strikingly consistent across subjects, with the highest number of strongly contributing points located in frontal cortical regions (i.e., the rostral middle frontal, pars opercularis, and pars triangularis), which are strongly implicated as critical areas for language processes.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_17
SharedIt: https://rdcu.be/cVD4Z
Link to the code repository
N/A
Link to the dataset(s)
http://www.humanconnectomeproject.org/
Reviews
Review #1
- Please describe the contribution of the paper
The authors present a new approach for predicting neuropsychological assessments scores from brain MRI data using a point representation of individual white matter fiber tracts. The approach uses a point-based siamese network adapted from PointNet. Their approach further enables the identification of areas in the input point cloud that are critical for the prediction. They evaluate their approach against classic along-tract analysis as well as a simple analysis of the tract specific mean values of the used feature maps. Their results are in line with previous research and they claim better performance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed point cloud based analysis is innovative and provides additional spatial information compared to other fiber representation methods.
The proposed WCRL algorithm identifies critical areas that are both consistent across subjects and with previous research regarding language performance as measured by TPVT scores.
The results hint that the presented approach might yield more information and thereby enables better predictions, but this is not confirmed using statistical tests.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
No statistical testing is performed. Therefore, the performance of the model cannot be evaluated properly. It is just shown, that the MAE is slightly lower compared to the baseline. In Table one, at least the standard deviation should be reported to display stability of the predictions. A box-plot with significant differences indicated would be best.
The authors compare their approach against relatively simple regression models. Why where theses baselines chosen and not more advanced approaches such as random forests or neural networks?
The authors missed to discuss their method and its limitations. It is also not discussed that classical along tract analysis also enables localization, albeit only along the tract.
The authors do not correct the density of their tractograms, e.g. using SIFT. Therefore, tract densities will not be representative and some regions will be over- and others under-estimated.
Incorrect definition of loss function: The definition of the loss function seems to be displayed incorrectly and should probably be changed to Loss=L_pre+w×L_ps
It is not explained, why the authors introduced a weight for L_ps and why it was set 0.1
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
By using the openly available data from the Human Connectome Project and a brief description of the statistics of the subjects used, reproducibility seems to be possible. In addition, the settings and software framework are described in Section 2.5. The authors’ answers on the reproducibility checklist confirm the given reproducibility. It would be beneficial, if the code would be provided on github.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The Lossfunction and Table 1 should be fixed, or explained in more detail. All in all one should explain the choice of tuned hyperparamters more deeply and discuss the applied method and its limitations in more detail.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty of representing microstructure measurements of white matter trajectories as point clouds weighed most heavily in the decision to accept the paper. The performance of the method compared to the baseline cannot be evaluated properly, as no statics are published.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
This work proposes a novel framework for predicting neuropsychological scores, which represent fiber bundles as point clouds, and then preserve point information about diffusion measurements and enable efficient processing using point-based deep neural networks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Looks like this is the first work to predict individual cognitive performance based on microstructure measurements of the white matter fiber tracts. - Overall, the paper is well written, and the experiments are well designed.
- Also, this work shows some promising results and is good for clinical application.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The motivation and evaluation of the study are limited in most clinical applications.
- The explanation from the results about neuroscience seems a bit far-fetched.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It should be easy for authors to provide source code for reproducibility analysis.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- How to keep the smooth information of a continue streaming when representation with a point cloud?
- Unclear why only left AF be used in this work. It is necessary to use both AF across hemisphere, or have a enough reason.
- Is there any way to automated select the best weight of difference loss w, and the number of input points?
- In table 1, the difference in MAE across methods looks very small, but significantly improved in r. It is very interesting. Needs to include more discussion about that.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
New issue in the dMRI field. Promising in clinical analysis.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors have almost addressed my previous concerns and may significantly improve the quality of the manuscript. Although there is still some weakness, which could be accepted.
Review #3
- Please describe the contribution of the paper
This work presents a deep network for neuropsychological score prediction based on point cloud representation of white matter tract features. The proposed network is trained by using a Paired-Siamese cost in addition to the MSE loss. The results of prediction compare favorably to traditional measures and a 1D network defined on tractometry. Based on this network, a critical region localization algorithm is proposed to detect the informative anatomical regions relevant to language processing.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The idea of using deep point-cloud network for neuropsychological score prediction is new. The proposed paired-Siamese cost for training the network seems an interesting formulation. The proposed method outperforms traditional methods, and it does not require a very deep architecture, making its interpretation easier. The proposed critical region localization algorithm effectively detects the regions relevant to language processing.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The point cloud representation does not include shape information, such as curvature, critical points etc. Hence, the network may not be able to detect alternations due to subtle shape variations. The motivation of using a point cloud based model is not sufficiently justified by its novelty. This work may be better motivated if critical region localization on fiber streamlines is a main task of interest. The discussions and conclusions drawn regarding the critical regions are weak based on the limited quantitative results of the critical region localization. These discussions can be reduced significantly. The intersection region figure in Fig. 3 is not clearly presented. What do the three colors (green, red, yellow) mean? Besides, the surface of white matter-gray matter interface may be a more proper choice for visualizing the intersection. In the caption of Fig.2, the synonyms are not properly written. Lpre -> L_{pre}, Lps ->L_{ps}, (MSE -> (MSE) The definition of Loss under Eq. (1) is wrong. Minor grammatic errors can be found. For example, 4 lines above Section 4, “our method is non-invasively localizing…”
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This work is fairly easy to implement given the background on white matter analysis and deep learning.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
This work may be improved by presenting more quantitative results on the critical region localization or considering a weaker conclusion about it.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The point-cloud deep network and the paired-Siamese loss are new for the task of neuropsychological score prediction. The network is simple and highly interpretable. The point-wise feature representation learned by the network is used to derive the critical region localization algorithm, which is shown to be able to detect the functional regions corresponding to the neuropsychological measure.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This work proposes a novel network for the prediction of neurocognitive scores based on a point cloud representation of the left arcuate fasciculus. There is consensus about the technical novelty of this work. There is some concern about the lack of statistical analysis in the experiments for comparison with previous methods. In addition, there is a lack of comparison with more advanced machine learning methods.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
4
Author Feedback
We thank the AC and reviewers R1-R3. We first give our responses to two main overall points requested by the AC, followed by our responses to other comments from each reviewer.
To address the “lack of comparison with more advanced machine learning methods,” we will add a new comparison method (random forest as suggested by R1). We have performed this experiment and the results show that our proposed method (r = 0.358) outperforms random forest (r = 0.134). We also clarify that a neural network method (suggested by R1) has already been included for comparison (AFQ+1D-CNN).
To address the “lack of statistical analysis in the experiments for comparison with previous methods,” following R1’s suggestion, in Table 1 we will report the standard deviations for each method, which are found to be stable across methods. In addition, we have investigated why the difference in MAE across methods is small but largely improved in r (asked by R1 and R2). We find that several baseline methods give predictions narrowly distributed around the mean, producing reasonable MAE but low r. We will clarify and discuss this in the paper.
R1 and R2 asked about the choices of parameters. The parameters including number of epochs, batch size and number of points are set according to conventional settings in deep learning pipelines for processing point clouds (Qi et al. CVPR 2017). The weighting parameter in the loss function, which is used to regulate the contributions of the main loss L_pre and the additional loss L_ps (asked by R1), is set to 0.1. This setting is commonly adopted as the weight of additional losses (Wang et al. MICCAI 2020; Fu et al. MICCAI 2020). We will further clarify this in the paper.
We acknowledge that classical along-tract analysis also enables localization though our point-wise localization can provide more detailed information (R1) and we will discuss this in the paper. We understand that SIFT has been widely used for improving connectome-based analysis (R1), while the current study focuses on microstructure of a certain white matter tract. We believe there should be minimal effects with or without density correction, but we agree with the reviewer that this is an interesting topic and will be discussed in our paper. We will further discuss our method and its limitations (requested by R1), including several points suggested by other reviewers. For example, we will further clarify and discuss that the left AF is dominant for language, rather than the right AF, so it was chosen in our study (R2), and we will acknowledge that by representing the whole tract as a point cloud, we did not utilize the continuous streamline information, which will be further investigated in the future (R2).
R2 and R3 have concerns about the discussion of results in relation to neuroscience, and R2 asked about the clinical utility of the method. While there is no ground truth to validate our identified results, multiple studies have shown the identified regions are critical in language processing, supporting our results. However, we agree with the reviewers’ comments and will soften the conclusions and reduce the discussions about the “limited quantitative results of the critical region localization” as recommended by R3.
For the purpose of this study, our point cloud representation utilized point spatial coordinates and microstructure measurements as point features but no direct shape indicators (R3). However, point clouds with spatial coordinates encapsulate shape information and have been applied to shape classification (Qi et al. CVPR 2017). We will further investigate the performance of including more explicit shape information. To justify our motivation of using a point cloud based model (R3), we will further clarify that critical region localization on fiber streamlines is a main task of interest. We will update Fig. 3 as suggested by R3.
We apologize for the typo in the loss function (R1-R3) and will further check and fix writing errors.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
While there is still some remaining questions about the statistical analysis, the consensus about the technical novelty among the reviewers make this work acceptable.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Overall, reviewers support the presentation of this work on the point cloud based prediction of neuropsychological scores from dMRI at MICCAI. Even though requests for statistical significance tests and a comparison to more advanced methods are only partly addressed in the rebuttal, I believe that acceptance of this work should not rely on them.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
6
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
I agree that the statistical analysis is helpful, but overall this is a good paper.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
8