Authors

Javid Dadashkarimi, Amin Karbasi, Dustin Scheinost

Abstract

Connectomics is a popular approach for understanding the brain with neuroimaging data. Yet, a connectome generated from one atlas is different in size, topology, and scale compared to a connectome generated from another atlas. These differences hinder interpreting, generalizing, and combining connectomes and downstream results from different atlases. Recently, it was proposed that a mapping between atlases can be estimated such that connectomes from one atlas (\textit{i.e.}, source atlas) can be reconstructed into a connectome from a different atlas (\textit{i.e.}, target atlas) without re-processing the data. This approach used optimal transport to estimate the mapping between one source atlas and one target atlas. Yet, restricting the optimal transport problem to only a single source atlases ignores additional information when multiple source atlases are available, which is likely. Here, we propose a novel optimal transport based solution to combine information from multiple source atlases to better estimate connectomes for the target atlas. Reconstructed connectomes based on multiple source atlases are more similar to their ``gold-standard’’ counterparts and better at predicting IQ than reconstructed connectomes based on a single source mapping. Importantly, these results hold for a wide-range of different atlases. Overall, our approach promises to increase the generalization of connectome-based results across different atlases.

Link to paper

DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_37

SharedIt: https://rdcu.be/cVD6T

Link to the code repository

https://github.com/dadashkarimi/carot

Link to the dataset(s)

https://www.humanconnectome.org/study/hcp-young-adult/document/900-subjects-data-release

Reviews

Review #2

Please describe the contribution of the paper

This paper proposed a all-way optimal transport algorithm that combines information from multiple source atlases to estimate a connectome close to the target one. As an application, predicted connectome was used to predict IQs and yielded a equivalent performance as the target one.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The proposed algorithm is novel. An all-way optimal transport algorithm was proposed based on single-source one. The information integration ability of the proposed algorithm was well demonstrated by comparison with single-source one.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
1. It is a bit weird that the predicted connectome has a better IQ prediction performance than the original one. Isn’t the algorithm was designed to predict a connectome as close as the original one? That is, if the original one has a bad IQ prediction performance, the predicted one is expected to have a bad one, too. If the predicted one has a better performance, as shown in Fig. 4, this connectome is a ‘hybrid’ connectome, rather than a predicted one.
2. The authors may provide some in-depth discussion on the performance.
Please rate the clarity and organization of this paper

Very Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors didn’t provide that source code. Datasets and atlases used are publicly available.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

For the first comment in Q5, the authors may consider some connectome-related metrics to evaluate the performance, such as graph metrics (degree, clustering coefficient, etc.) For the second comment in Q5, some discussion may be provided. For example, why other connectomes only have a prediction accuracy around 0.5 while Craddock has it as high as 0.75. Also, it seems that Craddock always yields the best single-source performance. Does the choice of atlas strongly affect the performance?
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The novelty of the proposed algorithm.
Number of papers in your stack

6
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #3

Please describe the contribution of the paper

This paper addresses a problem when you have a large rsfMRI dataset processed to form a connectome using one atlas, and you want to compare with a connectome generated with another atlas. Normally, you would have to reprocess your dataset with the second atlas in order to make a comparison. This problem exists because there are several widely used atlases used in connectomics research. Previously it was proposed to use optimal transport algorithm to transform connectome based on a single atlas into a connectome based on a second atlas, using just a subset of “training” subjects processed with both atlases. Here, the authors show that a connectome can be predicted better if the training dataset is processed with multiple existing atlases.
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

The paper is very well written. It addresses an interesting problem, for which only one solution seems to have been proposed so far. The solution is a natural extension of the single atlas to target algorithm. The evaluation is comprehensive and shows that the proposed multi-atlas strategy predicts the true connectome much better than single-atlas. Particularly impressive are the IQ prediction results because they show that synthetic connectomes generated using multiple atlases are just as good at predicting IQ as the true connectomes.
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

The only drawback is that it is somewhat unclear how practically useful this method will be. It still requires one to process some subset of the target dataset with the target atlas, and at that point, once you’ve set up all the scripts, doing it on the full dataset might just be a matter of computing power. But still, the method is elegant, so this is not a major weakness.
Please rate the clarity and organization of this paper

Excellent
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

Ok
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

See above
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

7
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

The positives outweigh the negatives. An interesting method with thorough evaluation and something different to complement all the CNN papers at MICCAI.
Number of papers in your stack

4
What is the ranking of this paper in your review stack?

1
Reviewer confidence

Somewhat Confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Review #4

Please describe the contribution of the paper

The authors explore the optimal transport method to estimate the mapping of brain connectomics form multiple source atlases, unlike widely used single source atlas, to a target atlas improving better estimation. For this a paired time-series is taken from ‘k’ different source atlases with a linear combination of ns regions and compared to the target atlas distribution (as distance minimisation of two pdfs). An iterative Sinkhorn algorithm is used to solve the equation using existing Optimal Transport toolbox [11].
Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.

1) The paper is well written and easy to follow 2) Using optimal transport for multiple source atlases is interesting 3) Results show clear improvement in correlation values with target ground truth 4) Provided details on parameter sensitivity is worth noting
Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.

1) The paper lacks insight into how many number of source atlases are optimal and what is the effect of increasing or decreasing these numbers 2) More details on choice of regions could help readers. What are ns and nt regions and how these are included?
Please rate the clarity and organization of this paper

Good
Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance

The authors have agreed to most of the reproducibility questionnaire.
Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html

1) It is hard to understand from abstract how the authors have validated their approach and what improvement they get. I think briefly mentioning the dataset and a sentence with key finding would help reader. 2) Authors could replace training and testing word with 80% for optimal ‘parameter estimation’ which was then applied on 20% remaining data for measuring the efficacy of the method. This will help readers not to be confused with the deep learning works. 3) Authors should take special care with regard to anonymity and not put the acknowledgment during submission of double blind reviews.
Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making

6
Please justify your recommendation. What were the major factors that led you to your overall score for this paper?

Use of multiple source atlases to better generalise the mapping of target atlas is interesting and the results show promise in the proposed method. Also, the experiments are done thoroughly along with the parameter sensitivity test.
Number of papers in your stack

8
What is the ranking of this paper in your review stack?

3
Reviewer confidence

Very confident
[Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed

N/A
[Post rebuttal] Please justify your decision

N/A

Primary Meta-Review

Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.

weaknesses were mainly found around the discussion, which can be easily adressed.
What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).

1

Author Feedback

N/A

back to top

Combining multiple atlases to estimate data-driven mappings between functional connectomes using optimal transport