Paper Info Reviews Meta-Review Author Feedback Post-rebuttal Meta-Reviews Back to top
List of Papers By topics Author List
Paper Info Reviews Meta-review Author Feedback Post-Rebuttal Meta-reviews

Authors

Kai-Cheng Chuang, Sreekrishna Ramakrishnapillai, Lydia Bazzano, Owen Carmichael

Abstract

Time-varying Granger causality refers to patterns of causal relation-ships that vary over time between brain functional time series at distinct source and target regions. It provides rich information about the spatiotemporal structure of brain activity that underlies behavior. Current methods for this problem fail to quantify nonlinear relationships in source-target relationships, and require ad hoc setting of relationship time lags. This paper proposes deep stacking networks (DSNs), with adaptive convolutional kernels (ACKs) as component parts, to ad-dress these challenges. The DSNs use convolutional neural networks to estimate nonlinear source-target relationships, ACKs allow these relationships to vary over time, and time lags are estimated by analysis of ACKs coefficients. When applied to synthetic data and data simulated by the STANCE fMRI simulator, the method identified ground-truth time-varying causal relationships and time lags more robustly than competing methods. The method also identified more biologically-plausible causal relationships in a real-world task fMRI dataset than a competing method. Our method is promising for modeling complex functional relationships within brain networks.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_26

SharedIt: https://rdcu.be/cVD48

Link to the code repository

N/A

Link to the dataset(s)

N/A

Reviews

Review #1

  • Please describe the contribution of the paper

    The paper focus on effective brain causality. More specifically on some issues related to Granger causality which are the limit of linearity and poor performance with task-based fMRI data. The non-linearity is addressed by estimating the regressive coefficient by deep stacking networks. The idea is tested on a synthetic dataset and on a real dataset.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The non-linearity in Granger causality is a known-issue therefore the paper focuses on a relevant topic. The paper is well-written.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1. The proposed method only test on real world data fMRI sequence of a known task. It lacks of a more practical application.

    2. Topology network to test the causality is unclear in the synthetic data (it seems just couple-triple)

    3. The case in exam for the real data is more suitable for other tools as DCM.

    4. There is no discussion or analysis for indirect connections

    5. The use of convolutional networks to estimate source-target relationship is new though already proposed for other causal physiological systems (Antonacci et al. PeeerJ 2021).

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    The synthetic dataset is probably reproducible, the other dataset will be probably distribute after acceptance

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    1. There is a growing interest on convergent cross mapping (CCM) (Siguhara et al. Science 2011) as an alternative tool to address non-linearity in Granger causality and estimation of different lags. Moreover, Granger and Siguhara causality (as well as dynamical causal model (DCM)) have been criticized to mere temporal correlation tools, and perturbation based approached should be more relevant. These aspects should be mentioned.

    2. In several papers investigating convergent cross mapping and Granger causality, it has been observed that the amount of noise added during the creation of the synthetic data (e.g. using the STANCE tool) can deteriorate the estimation of the causality directionality. In the paper, it is reported the defined relationship but not this critical detail. This is not a small thing, as you might be using relatively clean data with your ground-truth and slightly noisy data in reality, with the latter leading completely wrong causality estimation. How sensitive is the tool about this aspect? Or for the simulated data the reported noise (0.1) is already big, and therefore your simulations are even noiser than real data?

    3. It is not clear the network topology of the synthetic data. The STANCE has some kind of DMN simulation and other nucluei, but in the presented paper it seems to be a simple couple-triple relationship. The authors are asked to clarify this. Ideally you should have networks with at least 5 nodes as in (Smith et al. Neuroimage 2011, or Crimi et al. Neuroimage 2021). Couple analysis is completely a different story than multivariate.

    4. This also raises the further question about indirect causality. With convergent cross mapping a time series should be able to reconstruct even indirect causality (Ye et al. Nat SciRep 2015), is the same for your case given the conditional estimation and non-linearity? Or would you need a propagator settings as described in (Crimi et al. Neuroimage 2021).

    5. It is not clear why the authors use up to 5 lag causality. BOLD signal is very low-temporal resolution, generally causality study with those data don’t go beyond 3 lags due to this (depending on the data 1 lag is the max useful). Other times the order of lags is estimated by the Akaike information criterion, but I haven’t seen this here. I cannot grasp what you are trying to convey in Fig.5 regarding lags and inhibition/excitation, please clarify better.

    6. One of the advantages of Granger causality is that is computationally less demanding of DCM and therefore more suitable for brain-wide analysis rather than few ROIs. Yet, you use it in a scenario with 5 areas and task-based. Then, why are we even using Granger causality? We could stick with DCM, which is a physiological tool ideal for task-based studies and less prone of finding temporal correlation rather than causality.

    7. “the method captures richer information than prior methods” is clearly an overstatement. You haven’t proved the superiority compared to CCM for nonlinearity, or to propagators for indirect connections. Without saying that since you focus on task-based with few areas justifications against DCM has to be made.

    Minors: some capitalization in the bibliography is lost (“granger”, “keras”…), notation in Sec2.1 could be improved e.g. “time point of Yt into Yt…” Then there is \hat{Y}, please clarify or rephrase, for example at the second row of this section.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    The use of neural networks might be novel, but already used by other people. Moreover, the autoregressor coefficient for the linear case are already something like like machine learning predictor. If we focus on the non-linearity aspect, a comparison to Siguhara causality is necessary. The reported experiment is just an analysis for which more suitable tools exists, a further practical application is missing.

  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    3

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Not Answered

  • [Post rebuttal] Please justify your decision

    Not Answered



Review #2

  • Please describe the contribution of the paper

    The manuscript proposes a nonlinear Granger Causality estimation model by reconstructing the target time series based on the nonlinear modeling of source time series by a neural network. In addition, potential temporal lags between the source and target time series are modeled by the adaptive convolutional kernels (ACK) for identifying the real time lag. The proposed model was evaluated on both synthetic and real fMRI data.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    It is intuitive and novel to substitute the linear regression in Granger Causality analysis into non-linear functions, such as the stacked Deep Stacking Network used in this work. Using a network will not only improve the performance of fitting the underlying relationship between time series (thus obtaining more faithful GC estimation), but also has the potential of extending the scope of modeling (like the ACK operator introduced in this work).

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    1) In addition to what the author has claimed in the Introduction section, “One prior method limited the time lag to exactly one time point to reduce computational complexity [9], while the other methods required the user to specify the time lag a priori.”, there have also been works where different time lag setting (e.g., lag of 0, 1, and 2) will be tested independently to order to account for the possible different time lags when estimating GC. In the proposed model, the ACK filtering is conceptually similar to the mentioned approach.

    2) The possible typo in the sentence in page 3, “First, CNN-ACKs 1 and 2 are trained to transform previous time points of 𝑌_𝑡 into 𝑌_𝑡, and 𝑍_𝑡 into 𝑌_𝑡…” makes the reviewer difficult to understand all the latter descriptions.

    3) In the experiment results of “Real-World Task fMRI Dataset”, are the six connectivities listed in Supplemental Fig. 1 all of the non-zero causalities, including time lag k Granger causality?

  • Please rate the clarity and organization of this paper

    Good

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Implementing the network proposed in this work and performing the reproducible study is feasible (although complex) given the Fig. 2 and section 2.1. It will be better if the author can provide the source code for generating the synthetic data as well for further reproducible experiments.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    1. In the original Granger Causality modeling, the corresponding statistics of the significance can be estimated for testing whether incorporating X can “significantly” improve the reconstruction of Y. The reviewer would suggest performing similar investigation to derive the statistics of Granger Causality in the current non-linear setting, if possible.

    2. While the idea of using a fixed number (six) 1*2 filter to model the time-varying causal relationship at specific time lags is intuitive and working, the reviewer suggests improving this approach into a more integrated framework.

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    5

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    This paper proposed a novel perspective of utilizing a traditional effective connectivity estimation method (Granger Causality analysis). Using network as a non-linear approximator can improve the field of cognitive neuroscience and neuroimaging at various aspects in a similar way of this work.

  • Number of papers in your stack

    3

  • What is the ranking of this paper in your review stack?

    1

  • Reviewer confidence

    Very confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    Not Answered

  • [Post rebuttal] Please justify your decision

    Not Answered



Review #3

  • Please describe the contribution of the paper

    The paper proposes an extension to the conditional time-varying granger causality to the nonlinear setting. The target signal Y_t is modeled as a time varying kernel (learned by a neural network) convolved with X_t after accounting for covariates Z_t. Results are shown on synthetically generated data, fMRI simulated data showing that the method can recover the true lagged GC coefficients. Results on real-world task fMRI are also presented identifying an additional causal relationship from fusiform gyrus to occipital gyrus.

  • Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

    The presented method is novel and applicable to real world datasets. The problem considered has a long history and is relevant to the community.

  • Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

    Some papers are not included in the discussion, please cite and compare (if applicable). https://www.nature.com/articles/s41598-021-87316-6 https://arxiv.org/abs/1802.05842

    I am still a little confused about the optimization. Is the optimization performed purely based on the minimization of the MSE between the predicted Y_t and the actual Y_t? In that case isn’t the problem underdetermined? Is there only one possible kernel that can match the data? How is this problem solved in the proposed framework?

    What is Z_t in the synthetic dataset?

    How does the model perform in cases where there are bi-directional recurrent connections?

    How are the p-values evaluated?

    What is the performance of the model under non-Gaussian (or more generally non-iid) noise?

    What is the performance with increasing dimension of the data and covariates?

    How does this compare to other approaches for nonlinear connectivity estimation, such as convergent cross mapping?

  • Please rate the clarity and organization of this paper

    Satisfactory

  • Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

    Code are not included, but there are sufficient details for reproducing the results.

  • Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

    See the weakness part

  • Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

    4

  • Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

    Since this problem has a long history and the corresponding methods are applied broadly by the community, it’s important to investigate the properties of the model in detail to help the practitioner understand the limitations and strengths of the proposed method.

  • Number of papers in your stack

    4

  • What is the ranking of this paper in your review stack?

    2

  • Reviewer confidence

    Somewhat Confident

  • [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

    4

  • [Post rebuttal] Please justify your decision

    I don’t see any of my comments addressed in the rebuttal, hence I’m keeping my score unchanged. I still think that critical steps are needed to validate the proposed method, especially given how critical the context is. Functional, effective, and causal connectivity literature has a long history and there are well established issues for various proposed methods. It’s important to understand the limitations of the proposed technique using ground truth experiments and validations as a function of the parameters in the simulation/model.



Primary Meta-Review

  • Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

    This paper proposes a nonlinear Granger Causality estimation model by reconstructing the target time series based on the nonlinear modeling of source time series by a neural network. The method is intuitive and novel, and shows solid performance improvement. The reviewers raised some valid concerns that could be addressed to further improve the quality of the paper: (1) convergent cross mapping (CCM) (Siguhara et al. Science 2011), dynamical causal model (DCM), and perturbation based approaches are not considered or discussed. (2) Lack of consideration or analysis of indirect causality. (3) Does this model extend beyond fMRI?

  • What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    1



Author Feedback

  1. Reviewers ask about the novelty of our method compared to prior methods. The key difference between our method and convergent cross mapping (CCM), dynamic causal model (DCM), perturbation based, and neural network based approaches (PMID: 34084917, 33837245, 33705309) is that our method estimates time-varying causal relationships, including time-varying time lags, in a seamless fashion, via post-hoc analysis of CNN-ACK parameter estimates. To the best of our knowledge, our only time-varying competitor is CCM applied within a sliding temporal window (DOI: 10.1007/s11071-021-06362-x, PMID: 31639776). This sliding window approach has limitations: 1. The user must specify the size of the sliding window a priori, with no clear guidelines for how to do so (PMID: 34586519, 25234118). 2. CCM is vulnerable to inaccurate estimation of fit when the temporal window is short (e.g., less than 500 time points, DOI: 10.1007/s11071-021-06362-x). 3. The nonreverting continuous dynamics problem of CCM may lead one to infer false causality (DOI: 10.1101/2020.08.03.233692). To our knowledge, the only obvious time-varying extension of DCM or perturbation methods is sliding temporal windows, with the same limitations as above.
  2. Reviewers asked about the analysis of “direct” and “indirect” causality. The conditional Granger causality we estimate between A and B is exactly the “direct causality” between A and B after accounting for the “indirect causality” that goes from A, to C, to B. As such, our method automatically identifies indirect causality and separates it from direct causality. Indirect causality is often viewed as a nuisance phenomenon (PMID: 21232892, 31794821), but our method does identify it. We note that CCM (PMID: 26435402) uses ad hoc thresholding to identify indirect causality, while our method uses formal statistical testing.
  3. Reviewers asked about the breadth of applicability of our method. Our method can be applied to any data type with multiple time series, both biomedical (EEG, MEG, fNIRS, dynamic PET…) and non-biomedical (economic data, geoscience measurements), following prior applications of non-time-varying Granger causality (PMID: 20132895, 29542141, DOI: 10.1016/j.physa.2015.02.017, 10.5194/gmd-10-1945-2017).
  4. Reviewers stated that the synthetic data networks may have been too small. Since submitting this manuscript, we validated our method on two 5-node networks with bi-directional causalities, generated from mathematical formulas (PMID: 16927098); as well as a 33 ROI network derived from real data. The purpose of the current manuscript is to establish proof of concept on well-controlled and small networks, but we are willing to include the 5-node data in this paper.
  5. Reviewers asked why our method estimated time lags up to 5 time steps. We believe prior studies focused on time lags of 3 or fewer only because the methods did not have the complex machinery required to capture complex causal relationships, or for computational reasons. We chose to assess up to 5-step time lags to determine if our more complex method could capture causal relationships of longer duration.
  6. Reviewers asked us to clarify Figure 5. The magnitude and direction of the lag-1 causal relationship at each time point is represented by one parameter with a value in [-.5,.5]. The ground-truth value of this parameter at each time point is depicted visually via the color bar labeled “Ground Truth.” Three methods estimated the value of this parameter at each time point; those estimates are also depicted as color bars under the time series. Our method gave parameter estimates closer to the ground truth; our method’s color bar looks more like the ground truth than competitors did.
  7. Reviewers asked about the noise level in simulated data. Our simulated data had a noise level (0.1) in the same range (0.01-0.1) as real-world fMRI data (PMID: 15862224, 17126038).



Post-rebuttal Meta-Reviews

Meta-review # 1 (Primary)

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The authors have addressed the major concerns raised by reviewers in terms of its novelty, comparing to existing methods (DDC, DCM, etc.) and direct. vs. indirect causality. Though R3 has remaining concerns on the optimization process, implementation details, and notations, I still think the strength of this work overweight the weaknesses, and this work will bring new insights and applications to fMRI data analysis as well as other time-series data. Thus I recommend acceptance of this paper, with suggestions to the authors to include clarification and address R3’s questions in the final version.

  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Accept

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    6



Meta-review #2

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
    • The method applied non-linear granger causality dicovery for fMRI data
    • The R2 raised a valied point: identifiability, which is the most important in causal discovery , and some other technical quesitons regarding noise and p-value estimation
    • I found the rebuttal convincing, I vote for accept
  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Accept

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    na



Meta-review #3

  • Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.

    The rebuttal clarifies the concerns raised by the reviewers. Given the generally positive reviews on the novelty and quality of the work, I recommend to accept the paper.

  • After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.

    Accept

  • What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

    3



back to top