List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Sara Arabyarmohammadi, German Corredor, Yufei Zhou, Miguel López de Rodas, Kurt Schalper, Anant Madabhushi
Abstract
Quantitative immunofluorescence (QIF) enables identifying immune cell subtypes across histopathology images. There is substantial evidence to show that spatial architecture of immune cell populations (e.g. CD4+, CD8+, CD20+) is associated with therapy response in cancers, yet there is a paucity of approaches to quantify spatial statistics of interplay across immune subtypes.
Previously, analyzing spatial cell interplay have been limited to either building subgraphs on individual cell types before feature extraction or capturing the interaction between two cell types. However, looking at the spatial interplay between more than two cell types reveals complex interactions and co-dependencies that might have implications in predicting response to therapies like immunotherapy. In this work we present, Triangular Analysis of Geographical Interplay of Lymphocytes (TriAnGIL), a novel approach involving building of heterogeneous subgraphs to precisely capture the spatial interplay between multiple cell families. Primarily, TriAnGIL focuses on triadic closures, and uses metrics to quantify triads instead of two-by-two relations and therefore considers both inter- and intra-family relationships between cells.
The TriaAnGIL’s efficacy for microenvironment characterization from QmIF images is demonstrated in problems of predicting (1) response to immunotherapy (N=122) and (2) overall survival (N=135) in patients with lung cancer in comparison with four hand-crafted approaches namely DenTIL, GG, CCG, SpaTIL, and deep learning with GNN.
For both tasks, TriaAnGIL outperformed hand-crafted approaches, and GNN with AUC=.70, C-index=.64. In terms of interpretability, TriAnGIL easily beats GNN, by pulling biological insights from immune cells interplay and shedding light on the triadic interaction of CD4+-Tumor-stromal cells.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43987-2_77
SharedIt: https://rdcu.be/dnwKA
Link to the code repository
https://github.com/sarayar/TriAnGIL
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents a new approach named TriAnGIL, to quantitatively characterize the spatial arrangement and relative interplay of multiple cell families in pathological images. The features extracted from the graph constructed by the method have shown to be predictive of response after immunotherapy and able to determine long or short survival groups.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Studying cells interplay in the tumor microenvironment is really important
- The method outperforms other approaches even GNN on the presented experiments
- The results can be easily interpreted
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- No clear description on how the model is trained according to the constructed graph
- Many typos in the text
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The method for constructing the graph is clearly described and could be reproduced. But the usage of the features extracted from the graph is missing. No code publicly available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The only reproach I can make is that the method of construction of the graph is very clearly described, but nothing is said about the use of the features extracted from this graph. How are they used? By which algorithm? How is the relevance of such or such feature evaluated? A section on these questions should be added to make the complete understanding of the method clearer and the results more reproducible.
There are also many typos in the text, which should be proofread in detail.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The method is really interesting and have many benefits compared to GNN: explainability and understandability for medical doctors, efficiency on the experiments shown in the paper.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors developed a method to use triangle constellations (three cell types interacting) in IF images of Lung cancer to predict patient therapy response.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors have an innovative idea of using triangular intersections of three cell types, which is one step more than the usual binary cell interactions in that field.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Unfortunately, the newly introduced information does not seem to be helpful. The major features are from binary interactions of two cell types. Also, the authors do not seem to get significant better results than the GNN (sota).
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The data set seems to be proprietary (it is not provided). Therefore, reproducibility is poor.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Page 2: Many studies has only looked at…
- Page 2: Immune an cancer cells
- Page 3:”Once the different cell families are identified (Figure 2-B), a list is generated for all possible sets comprising of membership from three [9, 14] different cell families”: I do not understand this sentence and a rephrasing might be better. I got it later, though. Maybe: a list is generated of all sets of cells from maximal three cell types.
- Page 3: Figure 2-B is Figure 1-B
- Page 6: what is S_t-hat. and S_v-hat?
- Page 6: bilogical.
- Page 6: “it enables bilogical insights that a DL model might not be able to provide” Respectfully, I don’t agree. A DL model might even get much more biological insights than a hand-crafted feature approach can possibly get. It is more the human interpretation, which is difficult. So, replace “insights” with “interpretation”.
- Although the method seems interesting, the results indicate that the triangulation is not needed. The most important features were binary connections (stroma, cd4 and stroma tumor). These relations have been known already. Especially the interplay of a triplet of cells does not seem to be important?
- In the second experiment, the method seems to replicate what GNN already do. Is there any significant improvement over GNN?
- Reproducibility is poor as the data set does not seem to be available.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Although the idea seems to be interesting, I cannot see positive results in this experiment. The SOTA GNN is performing equally well as the Triangulation, and a feature analysis reviels that triangular features are not contributing much. Further, data set nor code are not provided.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
4
- [Post rebuttal] Please justify your decision
Thank you for the answer to the review. However, I could not change my overall opinion. The triangular relationship does not seem to provide significant results. Morevorer, the topic of n-tupel relationships between cell types is a known area of research of multiplexed images, such that the novelty here is moderate.
Review #4
- Please describe the contribution of the paper
This paper proposed an approach called Triangular Analysis of Geographical Interplay of Lymphocytes (TriAnGIL) which can capture the higher-order interactions (more than 2 interactions) between multiple cell types (extracted from immunofluorescence images) and hence is a useful and accurate approach for predicting immunotherapy response for cancer patients. TriAnGIL captures the interaction between multiple cell types by constructing heterogeneous subgraphs and quantifying triadic closures, thereby considering both intra and inter-family relationships between the different cell types. TriAnGIL is used for predicting response to immunotherapy and overall survival in patients with Non-Small Cell Lung Cancer (NSCLC).
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-The motivation behind developing the TriAnGIL algorithm is clear and the relevant previous works have been referenced. -The TriAnGIL algorithm proposed in this paper is novel as it is a better approach compared to other methods which analyze spatial interactions between cell types, as TriAnGIL can handle the interaction between multiple cell types as opposed to the other methods which could only handle interaction between a maximum of two cell types.
-TriAnGIL outperformed several handcrafted approaches such as DenTIL as well as deep learning based approaches with graph neural networks (GNN). -Since TriAnGIL uses triadic interactions, it is more interpretable compared to other approaches and hence could be easily used to find some insights on the triadic interaction between cell types that leads to a better immunotherapy response or overall survival prediction. - Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-In section 3.2 part 1, for the 126 features that are extracted: is it in the absence of one family or summed over all 3 cases where each case is the absence of one family. Please clarify this.
-The steps in figure 1 should be made in accordance with the text, i.e., first the proximity graph in 1.C-C1 and then Delaunay triangulation in 1.E-E1 -In section 3.2 part 2, how is the Delaunay triangulation done? Please explain briefly. Is it constructing a fully connected graph? -The process of feature extraction should be clearly explained both from the proximity graphs and Delaunay triangulation. -Are the triangular interactions calculated from the unpruned Delaunay triangulation (1-C1) or the pruned Delaunay triangulation (1-C2)? An ablation study should have been done comparing these two approaches. -In Algorithm 1, for the index j: shouldn’t the index be Del(i,j) in the inner loop? Is the index j used for the 3 types of vertices. Please explain. -How are the features extracted in the GetTriangleFeatures() function?
-For the experiments, a comparison with some well-known GNN algorithms such as GraphSAGE (“Hamilton, Will, Zhitao Ying, and Jure Leskovec. “Inductive representation learning on large graphs.” Advances in neural information processing systems 30 (2017).”) should have been performed to have an idea of the baseline GNN algorithms on heterogeneous graphs. -In section 4.3 and 4.4, for the results: how is the interpretability done to identify the features? Are the features (such as number of edges for IO or shorter triangle edges for survival) identified using gradient-based class activation mapping function or other methods. Please explain. -A reference or brief explanation of C-index should be provided. -When training the TriAnGIL algorithm, what are the parameters that are trained. A comparison between the number of parameters of TriAnGIL and the GNN should be provided. Since, the number of patients in training cohort is small: it is required to explain why the model does not overfit in this case. - Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
-This paper can be replicated.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Please refer to the weaknesses section for additional comments and questions. In addition to those, I have the following questions and comments: -In section 3.2, 5th line – there is no Fig. 2-B. I think, Fig. 1-B is referred. Please correct the figure reference. -In section 3.2 part 1, after equation 1, the statement should be removing its edges from G1 and not A.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The strengths in this paper slightly outweigh the weaknesses. This paper has novelty as it introduces an algorithm that considers the higher-order interaction between different cell types (more than 2 interactions) which cannot be done with other comparison methods. Also, this method is highly interpretable.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
Based on the rebuttal, I stick to my decision of weak accept.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper presents a method to construct graphs using higher-order (>2) relationships between different cell types (extracted from different tumor/immune multiplex immunofluorescence markers). The features extracted from the graph were shown to be predictive of treatment response as well as overall survival for two different non-small lung cancer patient cohorts, assembled from 5 different institutions. This is an interesting study with exhaustive evaluation against standard methods used for spatial biomarker quantification in multiplex immunofluorescence images. There are some concerns from reviewers that need to be addressed, i.e. [R1/R2] will the code for all the methods including TriAnGIL be released for at least partial reproducibility of the paper in case users have their own relevant datasets (assuming the datasets can’t be released)? [R1/R3] further elaboration of the GetTrianglesFeatures() function and how these features are used and salient features selected in the two experiments via LDA/mRMR, etc? [R3] strategies used to avoid overfitting of GNNs on such a small cohort? brief description of the delaunay triangulation computation methods and c-index, [R2] briefly elaborate on the results in the context of the importance of triplet relationships and advantages over GNNs?
Author Feedback
Our paper received scores of 6, 4, 5 (R1-R3). R2 assigned the lowest score, and we focus primarily on R2. The concerns mainly stemmed from a lack of full comprehension of the information presented in Section 4, where we thoroughly explained 1) how the features were used, 2) importance of triangulation features, and 3) advantages of TriAnGIL over GNNs. 1) Triangulation information is not helpful as the triangular features did not contribute to the results: We disagree. TriAnGIL is adaptable and captures multiple interaction types simultaneously (binary and triadic) by focusing on triplets. In experiment 1, the binary interactions were predictive of immunotherapy response. But, in experiment 2, the triadic interaction features were selected by LASSO as the top features for predicting overall survival. This was explained in Section 4.4 and Figure 2, where we showed how the CD4-stroma-tumor triangulations were different between long- and short-survival patients. In fact, it is a significant strength that TriAnGIL is versatile, can generalize to multiple domains and can discover what types of spatial relationships are predictive of outcome or response. 2) How were the features used and salient features selected in the two experiments: As mentioned in experiment 1 (Section 4.3), once the features were extracted from every triplet using TriAnGIL method, we performed mRMR feature selection and selected 2 top features (most correlated with immunotherapy response) and trained a LDA classifier using the training set and validated it on the validation set. In experiment 2 (Section 4.4), we used LASSO to select the 10 top features (most associated with the risk of death) to calculate risk scores for every patient. We then trained a Cox PH model [30] to analyze overall survival. Interestingly, we found that the interplay of tumor-stroma-CD4+ cells was predictive in both experiments. The use of feature selection strategies helped avoid overfitting with TriAnGIL, as also the use of dedicated and independent training and validation sets 3) Elaborate the advantages of TriAnGIL over GNN: TriAnGIL outperformed transformer-based GNN in both experiments, although the difference was not substantial in experiment 2. However, TriAnGIL boasts several additional advantages over GNN: 1) As mentioned in Section 2 and 4.4, TriAnGIL is highly interpretable (mentioned by all three reviewers) while GNN is opaque, so interpreting its results is a challenge in the medical domain since physicians need to understand the reasoning behind AI-driven predictions, fostering trust, facilitating clinical validation, ensuring patient safety, and promoting ethical accountability. 2) Compared to transformer-based GNN, TriAnGIL is less resource-intensive, does not require substantial computational power and very large quantities of data points to be trained effectively. 3) TriAnGIL has a few parameters (<10) and is therefore more robust to overfitting. In contrast, GNN has ~4 million parameters, and is prone to overfitting. In this study, we utilized node dropping and dropout techniques to avoid overfitting with GNN, but it remains less reliable in real-world scenarios compared to TriAnGIL. 4) Elaborate GetTriangleFeatures() function: As mentioned in Section 3.2, this function is the last step of Algorithm 1 where we quantify the triangles that were picked out using previous steps. GetTriangleFeatures() calculates a set of 33 features (included in Table 1 in Supplementary) to measure basic triangle properties, including area, perimeter, and longest side. 5) Will you release the codes for reproducibility and describe the Delaunay triangulation and c-index: We will share all the codes publicly on Github, add 1-line explanation (with reference) to describe well-known terms of c-index and Delaunay, and to verify that GNN is a valid baseline. We will also provide a list of LASSO features with their corresponding weights in the supplements and correct all typos upon acceptance.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
All the questions were addressed by the authors.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This work tries to tackle an important clinical problem. The method and results sound reasonable. The rebuttal properly addresses the major concerns of the reviewers.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The responses of the authors during the rebuttal process is not convincing enough. We hope that the constructive remarks will help you to improve the work for any future submission.