List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Sixtus Dakurah, D. Vijay Anand, Zijian Chen, Moo K. Chung
Abstract
Cycles or loops in a network embed higher-order interactions beyond pairwise relations. The cycles are essential for the parallel processing of information and enable feedback loops. Despite the fundamental importance of cycles in understanding the higher-order connectivity, identifying and extracting them are computationally prohibitive. This paper proposes a novel persistent homology-based framework for extracting and modelling cycles in brain networks using the Hodge Laplacian. The method is applied in discriminating the functional brain networks of males and females. The code for modeling cycles through the Hodge Laplacian is provided in \url{https://github.com/laplcebeltrami/hodge}.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16431-6_31
SharedIt: https://rdcu.be/cVD5d
Link to the code repository
https://github.com/laplcebeltrami/hodge
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper proposes a novel method to identify 1-cycles in brain networks. The algorithm starts with decomposing the network into MST and non-MST. Then a 1-skeleton with only one cycle is composed by adding one edge from the non-MST into MST. The unique 1-cycle can be identified by the zero eigenvalue of the Hodge Laplacian. The number of 1-cycles is the same as the non-MST edge number. The authors then propose that these 1-cycles forms a basis system over the collection of all possible 1-cycles, and thus can be used to discriminate networks with different topology. The authors validate the proposed method on simulated networks, against geometric measures 1-norm, 2-norm, infinity-norm and Gromov-Hausdorff distance.
Contribution: The authors provide a new mathematical framework to extract cycles using the Hodge Laplacian over simplicial complexes.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors propose a novel algebraic method to identify and represent 1-cycles in a weighted graph. The simulation experiment shows high accuracy in discriminating networks with different topology.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
** Although we appreciate the algebraic methodology proposed by the authors, the motivation of the proposed method is not clear. Computationally, simply using the non-MST edge and the MST will give you a cycle. This way of computing a cycle basis with a MST is very efficiently. It is not clear why we need the Hodge Laplacian and its eigen decomposition. Plus recomputing the Laplacians and their eigen vectors for each non-MST edge will be extremely expensive.
** There are already methods like persistent homology to differentiate networks with different topology. What is the benefit of the proposed method over persistent homology? Some empirical comparison with persistent homology features might need to be provided to prove the proposed method is better and necessary.
** More details in the validation experiment need to be provided. In equation (6), the authors define the difference between two groups with the same nodes. However, they do not mention how to calculate the difference between two different networks, like the difference between group 1 and group 3 in Fig.2. In such case, how to match the nodes between two groups is not trivial, even when the two groups have the same number of nodes.
** There are no baselines in the application experiment. Although we can see the proposed method can differentiate the male and female brain network successfully, the authors did not provide any baseline to compare with. Such baselines can include the other metrics used in validation and also methods like persistent homology. For persistent homology, here is a very relevant work that can be considered to be a baseline,
T. Songdechakraiwut, L. Shen, and M.K. Chung. Topological learning and its application to multimodal brain network integration. Medical Image Computing and Computer Assisted Intervention (MICCAI), 12902:166–176, 2021.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The description seems reasonably clear. But I believe the code needs to be publicly available to ensure full reproducibility.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
See above.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors do propose some interesting novel ideas. But the motivation is not completely clear at this moment. Also comparing to other topology-based baselines is necessary. I will be happy to improve my scores if the concerns are addressed.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #2
- Please describe the contribution of the paper
The quantitative analysis of structural and functional brain connectivity by network (graph-based) measures helped to understand their complex properties. This submission proposes to extend current methods by including loops in the analytic assessment of networks. Authors introduce the Hodge Laplacian as a generalization of the graph Laplacian in order to identify and quantify 1-cycles (loops) in functional connectivity. Functional MRI data acquired in the resting state were used here, from a large, publicly available data base.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The topic of this submission is within the focus of MICCAI, and of potential interest to a broader audience. The text is understandable for a reader with knowledge in network methods for modeling structural and functional data. Interesting on-going work is presented here.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Unfortunately, one potential major error was found (#1).
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
There are two questions about the validity of the statistical analysis (see item 8).
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
- p.6, Table 1: It is understood that p-values are shown here, and it is guessed that the \pm refers to a standard deviation, likely, from repeating the simulation. If this guess is correct, than, well, contents of this table are incorrect. It is not viable to perform this kind of arithmetic on p-values. Even more, p-values are not Gaussian distributed. Please, revise.
- Suppose Table 1 was corrected. Please, describe exactly how you demonstrated that “the proposed method … outperformed all (other) measures.” How do you compare two (or more) sets of p-values?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work presents a novel and useful extension of a relevant analytical method in medical image analysis.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
Not Answered
- [Post rebuttal] Please justify your decision
Not Answered
Review #4
- Please describe the contribution of the paper
In this paper, the authors propose a 1-cycle basis in brain network analysis of fMRI-based graphs. The 1-cycle basis is extracted based on Hodge Laplacian using persistent homology framework. The authors first prove (Theorem 1) that the collection of 1-cycles (the maximum homology group for a graph is 1) spans the kernel L1 or the 1-th Hodge Laplacian of the graph. Then using this theorem, they extract the coefficients of 1-cycles across all graphs in the between-subject study. In the end, they also introduce an alternative to GH and other distance measures to compute the difference between brain networks, named statistics T. To validate the efficacy, they analyze the common 1-cycles between females and males in the brain network of normal subjects.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well-written and addresses all the necessary theoretical aspects, both Hodge Laplacian theory and persistent homology. Their novel 1-cycle basis helps to extract brain multi-way interaction between regions as opposed to conventional connection-based analysis. This is even more fruitful for analysis of DTI based networks which are more sparse and can be more discriminative in normal-patients comparison.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The authors could have better demonstrated the motivation for cycles in the brain network by providing more evidence in the neuroscience literature. Although this feature can be discriminative, the intuition behind finding some cycles with carriable length may not be very clear from a medical analysis standpoint. Also, there could have been a comparison to 0-cycles features based on the same approach to see the significance of 1-cyles are more important and meaningful.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Table 1 in the paper is not well tabulated as there are no bold numbers to see the difference. The fact that just positive connections are chosen to construct the final network should be explained more, as those connections are important and some studies may use the absolute value. There is a paper related to the cycles using the notion of persistent homology which is not mentioned in this paper as a related paper in medical brain network analysis ” A Univariate Persistent Brain Network Feature Based on the Aggregated Cost of Cycles from the Nested Filtration Networks”.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This work proposes a novel theoretically sound approach to identifying 1-cycles in the brain network as a strong feature. This novel approach is proven based on an important theorem in the paper. The synthetic data and real-world data suggest the efficacy of the proposed method to extract 1-cyles in the brain network. This idea can be extended to any graph semantic graph analysis beyond medical image analysis. The paper is also clearly well-written and cohesive in all parts.
- Number of papers in your stack
1
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
8
- [Post rebuttal] Please justify your decision
Great work with novel ideas and solid experimental results. It will certainly inspire much new research in our community.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper provides an algorithm for identifying 1-cycles in brain connectivity network in order to incorporate these in the downstream analysis. While the reviewers appreciate the theoretical contributions of the paper, they also have several concerns. For the rebuttal, the authors should in particular address:
- the motivation for why cycles are relevant features, and why they are not sufficiently captured by existing work such as persistent homoloty
- the concerns of Reviewer 1 regarding relationship and comparison to alternative methods
- the question from Reviewer 2 concerning the p-values
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
5
Author Feedback
We thank reviewers who complemented our work as “novel”, “theoretically sound”, “well-written” and “broad appeal”. However, Reviewer 1 misunderstood our paper. Misunderstandings can be easily addressed by additional edit and code distribution with publication.
R1: …not clear why we need the Hodge Laplacian … recomputing the Laplacians and their eigen vectors for each non-MST edge will be extremely expensive.
When a non-MST edge is added to the MST, a cycle is formed, but we do not precisely know which edges constitute the cycle. The eigen decomposition of Hodge Laplacian can precisely identify only the edges that contribute the cycle. Since one edge added to MST is extremely sparse, computation is extremely fast. The CPU time for identifying all the cycles for 400 subjects took about 12 seconds in a desktop.
R1: There are already methods like persistent homology to differentiate networks with different topology.
Existing persistent homology methods can differentiate networks of different topology, but they cannot localize the connections contributing to the difference. Our method can explicitly identify and model all the cycles and then match them across networks at the edge level, which was never done before. This is the main contribution.
R1: More details in the validation experiment … they do not mention how to calculate the difference between two different networks, like the difference between group 1 and group 3 in Fig.2.
We used the data augmentation published in Songdechakraiwut et al (2021, MICCAI). Additional explanation will be given in revision.
R1: There are no baselines in the application experiment.
There is no ground truth in real brain data, so it is unclear which method produces the correct answer. Thus, we only applied the 4 baselines in the simulation with the ground truth. We will add the result of 4 baselines to real brain networks in supplementary.
R2: Table 1: It is understood that p-values are shown here, and it is guessed that the \pm refers to a standard deviation, likely, from repeating the simulation. …contents of this table are incorrect. It is not viable to perform this kind of arithmetic on p-values. Even more, p-values are not Gaussian distributed.
Regardless of the distribution, we can always compute the standard deviation of p-values in showing the variability of p-values. This is a common practice in literature (Songdechakraiwut et al., 2021, Vidaurre et al., 2019; Phillips et al., 2011). However, we will add an additional metric in the table: the rates of false positives and negatives in the revision.
R2: …describe exactly how you demonstrated that “the proposed method … outperformed all (other) measures.”
We will tone down our expression. Our method only outperforms 4 other baseline methods.
R4: …the intuition behind finding some cycles with carriable length may not be very clear from a medical analysis standpoint.
The cycles are important for information propagation and feedback loops. The larger the cycle, the more inefficient the information transfer around the cycle. The cycle length can be a new biomarker for quantifying the disease progression such as AD (Chung et al., 2019, Farazi et al, 2020).
R4: …there could have been a comparison to 0-cycles features based on the same approach….
The same method can be applied to 0-cycles using the graph Laplacian (0-th Hodge Laplacian). This will be added in the supplementary material.
R4: …positive connections are chosen to construct the final network should be explained more…
Negative connections are still not fully understood. Many works have excluded negative correlations and advocated the use of positive connections alone (Buckner et al., 2009; Meunier et al., 2009). We will add additional explanations.
R4: … paper related to the cycles … ” A Univariate Persistent Brain Network Feature Based on the Aggregated Cost of Cycles from the Nested Filtration Networks”.
We will include it in the revision.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
In the rebuttal, the authors were asked to address the following:
- the motivation for why cycles are relevant features, and why they are not sufficiently captured by existing work such as persistent homology
- the concerns of Reviewer 1 regarding relationship and comparison to alternative methods
- the question from Reviewer 2 concerning the p-values
However:
- they do not address why cycles are not sufficiently captured by existing work such as persistent homology
- they quote small samples size as a reason not to compare with GNNs, but first of all, GNN parameters are shared between nodes, not just between graphs, so a higher number of nodes also helps prevent overfitting. Moreover, if GNNs truly were not able to learn a model on this data, why not showcase this to strengthen the paper?
- They do not address the validity concern of reviewer 2 concerning statistical analysis of p-values.
In other words, the authors have not really answered any of the questions asked, and I therefore recommend rejection.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Reject
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The paper presents an elegant framework for modeling higher order interactions in network using a persistent homology type approach. The idea of using the eigen decomposition on the Hodge star operator (Laplacian) for detecting cycles in brain networks is clever.
All reviewers agreed on the methodological novelty of the paper. The authors response in the rebuttal was satisfactory.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
he key strength of this paper is to provide a new mathematical framework to extract cycles using the Hodge Laplacian over simplicial complexes. Although there are still some concerns, I think it is novel enough for acceptance of this paper. I would strongly suggest the authors revise the final version of this paper to integrate all useful comments.
- After you have reviewed the rebuttal, please provide your final rating based on all reviews and the authors’ rebuttal.
Accept
- What is the rank of this paper among all your rebuttal papers? Use a number between 1/n (best paper in your stack) and n/n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1