List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Tiantian He, Elinor Thompson, Anna Schroder, Neil P. Oxtoby, Ahmed Abdulaal, Frederik Barkhof, Daniel C. Alexander
Abstract
Computational models of neurodegeneration aim to emulate the evolving pattern of pathology in the brain during neurodegenerative disease, such as Alzheimer’s disease. Previous studies have made specific choices on the mechanisms of pathology production and diffusion, or assume that all the subjects lie on the same disease progression trajectory. However, the complexity and heterogeneity of neurodegenerative pathology suggests that multiple mechanisms may contribute synergistically with complex interactions, meanwhile the degree of contribution of each mechanism may vary among individuals. We thus put forward a coupled-mechanisms modelling framework which non-linearly combines the network-topology-informed pathology appearance with the process of pathology spreading within a dynamic modelling system. We account for the heterogeneity of disease by fitting the model at the individual level, allowing the epicenters and rate of progression to vary among subjects. We construct a Bayesian model selection framework to account for feature importance and parameter uncertainty. This provides a combination of mechanisms that best explains the observations for each individual. With the obtained distribution of mechanism importance for each subject, we are able to identify subgroups of patients sharing similar combinations of apparent mechanisms.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_45
SharedIt: https://rdcu.be/dnwNQ
Link to the code repository
N/A
Link to the dataset(s)
https://www.humanconnectome.org/study/hcp-young-adult
https://portal.conp.ca/dataset?id=projects/mica-mics#
Reviews
Review #2
- Please describe the contribution of the paper
This paper provides a model for neurodegenerative misfolded protein local accumulation and spatial spread across the brain that combines two previously articulated ideas: 1. that local accumulation and spatial spread can be captured by a differential equation that operates on graphs where nodes represent brain regions and edges represent inter-regional connectivity; 2. that local accumulation may depend on local graph metrics such as node centrality and degree, for various neurobiological reasons. This paper puts those two concepts together into a combined model; allows the dependency of local accumulation on graph metrics to vary by individual; and provides uncertainty estimates on model parameters because parameter estimation is all Bayesian.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The main strength of the paper is that it extends simulation models of misfolded protein accumulation and spread in a seemingly reasonable and nontrivial set of ways. Combining the dominant graph-based accumulation and diffusion model with individual-specific connectivity-baed accumulation modifiers seems like a good idea as a way to model more of the heterogeneity that we know underlies the progression of these neurodegenerative diseases. Also, providing uncertainty estimates for model parameters seems like a good idea in general, as a way for clinical applications to figure out how much they should trust individual-level predictions about accumulation and spread.
The methodology seems to be pretty clearly described.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The biggest problem is the validation experiments, which are unclear in their design and not that well motivated. ADNI participants who provided longitudinal pairs of tau PET scans were included, leading to the obvious conclusion that this method would use each individual’s baseline scan as the initial condition, and we would see how well the model generates the follow-up scan, given that initial condition plus simulation. Instead, only a slim subset of such ADNI participants were selected for reasons that were unclear; their tau PET data was edited / modified to force all this data to conform to an “epicenter” model where tau accumulation was required to initiate at one of 8 selected epicenters; local tau PET positivity was determined in an ad hoc fashion; and the tau PET values were scaled in an arbitrary fashion to provide simulation initial conditions. In addition it is unclear whether the modeling is used to generate just the follow-up scan, or baseline and follow up together; or what. Together, all of the data massaging and cherry-picking undermines the premise that this method does a better job of capturing the heterogeneity in temporal progression of tau in AD; it seems like a lot of the heterogeneity has been edited out of the data set by the authors.
Another weakness is the use of a priori population-level estimates, rather than individual-level DTI or fMRI data, to calculate inter-regional brain connectivity. Such connectivity varies substantially from person to person. The use of generic connectivity data again undermines the premise that this method handles inter-individual heterogeneity better than other methods.
Also, while estimation of model parameter uncertainty is a selling point of the method, the results don’t show those, nor do we have a clear sense of why the authors think those are useful to have.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The data used in the study comes from a widely known public domain source (ADNI), which is good for reproducibility. As mentioned above though, many critical details of the evaluation are unclear, thus reducing the ability to reproduce results from these descriptions alone.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
See comments above.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method seems to be an advance that meaningfully pushes forward models of neurodegenerative disease. Uncertainty about the experimental validation tempers this enthusiasm however.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
In this paper, authors explore the problem of pattern of pathology progression in neurodegenerative diseases over the example of Alzheimer’s Disease. They quote prior literature that models progression and spread of the diseases as a linear model that considers the contribution of the two. Highlighting the limitation of the prior literature in considering the interaction between these two aspects, they propose a bayesian model that also accounts for the interaction between these two aspects. In their analysis, they compare their propsed method to a baseline method that only considers combination of the two aspects while ignoring their interaction. They demonstrate improvement of prediction with the proposed model across subjects and at individual level. They also demonstrate that structural centrality and segregation plays a more highlighted role in the prediction.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The study is conducted well, with a novel method as it also evaluates interaction between location and spread of the disease together. Experiments point to the intended result. And the paper is well written.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
I only have two small comments regarding the manuscript, as noted below.
- The first paragraph of the introduction makes bold statements about the state of the art, without giving any citation. Please cite relevant literature to give birds eye view to the reader.
- Coloring scheme of Figure 3.A is not explained, hence not conveying a message properly to the reader. Figure 3.B is too small and the message that it conveys was unclear to me.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Authors provided details of data processing and how the models were trained. I would assume, one could repeat the experiments by following the steps explained in the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Provided above in the weaknesses section.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Paper addresses a relevant problem by using a robust method. It is written well, and conducted experiments support their claims.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This paper considers modeling the evolving pattern of neurodegeneration pathology. A dynamical model is proposed for the entire process of spontaneous pathology appearance and pathology spread along brain connectivity pathways. This allows individualized estimation of pathology progression in terms of epicenters, diffusion rate, etc. The proposed model also provides mechanistic profile (contributions of multiple network topology metrics) estimation, which is combined with the rate of production of the local production process terms. Variational Bayes approximation with sparsity constraints is used in the fitting of this model. The result shows that the model fitting is improved with the new model compared to a baseline model that does not account for the mechanistic profiles and only assumes a constant local production rate. The result is also used in clustering of patients based on the top two relevant network topology metrics in the individual profiles.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed estimation framework allows more flexible individualized estimation of epicenters, diffusion rate, local production rate, and weights of mechanistic profiles. The variational Bayes inference method potentially allows feature importance and uncertainty analysis. The result is potentially impactful also as a tool of patient clustering based on pathology progression patterns, which is especially relevant given the complex hypotheses around different types of neurodegenerations.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The performance of the proposed model is not sufficiently evaluated. No cross-validation or independent sample is used for validating the prediction performance. The performance is only quantitatively evaluated by the Pearson correlation between the predicted value and the observed value on the same dataset. This not only provides possibly biased evaluation of precision, but may also flavor the more complex model (the proposed one with mechanistic profile terms in comparison to the baseline with constant local production rate) by the nature of this metric.
There is no comparison in performance with state-of-the-art models, including the network diffusion model and the Garbarino et al. (2019) model that are listed as the major comparators in the introduction section.
The subgroup identification method seems arbitrary (using top two metrics out of the five candidates, with the largest weights) and only discovered six subgroups that cover a part of the full cohort (only 84 out of the 110 participants are labeled).
Uncertainty estimation seems available based on the methodology section, but the performance is not evaluated.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The description of the model and implementation is clear for reproducibility purposes.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Pearson correlation based on the same data might be improved by cross-validation based predictive R-squared.
The use of t-test can be better justified with roughly symmetric distribution of the predictions.
Alternative clustering methods such as tree-based methods might be considered to provide more powerful subgroup identification.
The result claims that some low performance subjects are significantly improved after using the proposed method; this might be better supported with illustrations of the fitted individual trajectories.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper proposed an interesting and potentially impactful framework for modeling the heterogeneous progression of neurodegenerations. The motivation, methods, and results are clearly presented. However, despite several promising properties of the proposed framework, the data experiments provide only limited justification on the performance of the proposed method and the advantages compared to state-of-the-art alternative methods.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The presented work is important to the computational clinical neuroscience community. The approach is sound and interesting. There is a consensus that the experimental validation could be improved to better validate and illustrate the advantages of the proposed framework, such as the parameter uncertainty. Choices regarding use of population level estimates of connectivity matrices, local tau PET positivity, epicenter selection should be clarified. Comparisons with state-of-the-art methods are missing and would be desirable.
Author Feedback
We thank the reviewers for the comments.
R2 Regarding initialization, we use epicentres rather than the baseline scan for two reasons. First, we want to explore the mechanism of the full process of disease progression from very early stages, i.e. prior to the baseline scan, from which we simulate the baseline scan and the follow-ups. The baseline scans are typically already at relatively advanced disease stages so contain diffused patterns. Second, the pattern of tau distribution at baseline and follow-ups are similar in ADNI due to the narrow time gap (within 4 years). Thus, the follow-up can be easily predicted from baseline even using some simple models. It is also common in literature: Lee et al Neuron 2022 use a single region with most people being tau positive as the epicentre. Vogel et al Nature Medicine 2021 identify 4 epicentres of the disease progression and they are all included in our epicentre candidates. We’ll clarify these in section 3.1 and list all candidate epicentres in the supplementary.
To answer “only a slim subset of participants was selected”, this is not “cherry picking”, but alignment with the task. We exclude the subjects with one scan since we cannot estimate the trajectory slope and subject location simultaneously. And we include only Abeta-positive subjects with at least one region being Tau-positive to focus on the people with the potential to accumulate and aggregate them in abnormal amounts and pathological forms. We’ll clarify this in Section 3.1.
We agree that using the cohort-level connectome is a limitation and will mention it in the discussion, but individual connectomes generated from ADNI have higher false positive/negative rates due to the lower MR image qualities. Furthermore, Powell et al J Alzheimers Dis 2019 show that there’s no significant difference between the prediction of the spreading model using personalized and population-level connectome generated from ADNI.
R3 Regarding the missing citations and unclear expression of Fig3, we’ll add references and captions in the camera-ready version.
R4 Regarding the lack of model validation, the main aim of this paper is to uncover the underlying mechanism of disease progression by estimating the parameter distribution and contribution of different metrics on the individual level. Cross-validation is challenging due to heterogeneity– all the parameters vary among subjects and thus it’s hard to apply the model trained on one subject to a new subject from the validation set. But we will validate the cohort-level information derived from the six subtypes that our model has identified using external datasets in future work.
For the model evaluation metric, R correlation is the metric used in previous papers on network spreading (Raj Neuron 2012, Raj Cell Reports 2015) to focus more on the match of the distribution of disease patterns across different regions regardless of the data range. Nevertheless, we’ll appreciate the advice to include other metrics which penalize model complexity in Discussion but leave it for future work since the sparsity structure of our model, designed to avoid overfitting, complicates parameter counting for standard information criteria.
We did have a comparison to the original network diffusion model, which is defined as beseline. Regarding the missing comparison to Garbarino’s method, it is difficult as their method is not a generative model like ours but instead matches the derivative of the trajectory with the linear combination of different mechanisms. But we have some ideas of making the comparison doable by extending their model and will mention it as future work.
Regarding subtyping using the top 2 contributions in Fig4, this is intended only as an initial demonstration of the potential to derive subtypes from our modelling framework, and it already gives a relatively clear distinction in the main feature patterns.
Finally, we’ll add potential ways of evaluating uncertainty in the Discussion.