List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Yingli Zuo, Yawen Wu, Zixiao Lu, Qi Zhu, Kun Huang, Daoqiang Zhang, Wei Shao
Abstract
The tumor-infiltrating lymphocytes(TILs) and its correlation with tumors play a critical role in the development and progression of breast cancer. Existing studies have demonstrated that the combination of the whole-slide pathological images (WSIs) and genomic data can better characterize the immunological mechanisms of TILs and assess the prognostic outcome in breast cancer. However, it is still very challenging to characterize the intersections between TILs and tumors in WSIs because of their large size and heterogeneity patterns, and the high dimensional genomic data also brings difficulty for the integrative analysis with WSIs data. To address the above challenges, in this paper, we propose an interpretable multi-modal fusion framework, IMGFN, that can fuse the interaction information between TILs and tumors with the genomic data via an attention mechanism for prognosis predictions of breast cancer. Specifically, for WSIs data, we use the graph attention network (i.e., GAT) to describe the spatial interactions of TILs and tumor regions across WSIs. As to genomic data, we use co-expression network analysis algorithms to cluster genes into co-expressed modules followed by applying the Concrete Autoencoders to select survival-associated modules. Finally, a self-attention layer is adopted to combine both the imaging and genomic features for the prognosis prediction of breast cancer. The experimental results on The Cancer Genome Atlas(TCGA) dataset suggest that the proposed IMGFN can not only achieve better prognosis results than the comparing methods but also identify consistent survival-associated imaging and genomic biomarkers correlated strongly with the interaction between TILs and Tumors.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_22
SharedIt: https://rdcu.be/cVRrE
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper presents a multi-modal framework that fuses the interactions between TILs and tumors with a graph attention network and the genomic data by concrete autoencoders for prognosis predictions of breast cancer.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The graph attention network can characterize the interaction between TILs and tumors by weighting graph edges.
The experimental design is comprehensive, especially integrating several scenarios using imaging only and using genomic data only.
The correlation between particular genes from the concrete autoencoders and TIL and tumors is demonstrated.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Why is clinical or demographic info not integrated into the framework? Please check the rest of the minor comments in section 8.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Yes.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
What is the magnification of the WSI?
The meaning of gamma, lambda, t, and beta in ImQCM should be explained.
Were there any duplicate patches across the two categories for the selected 200 patches with the largest tumor or TILs ratio?
The red/blue number labels in Fig. 2(b) patient A seem to be inverted. The black bounding box in Fig. 2(b) Patient B appears off.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It seems to be the first method that combines TILs information with genomic data for breast cancer using WSIs.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper analyses the role of immune cells, lymphocytes, in their role in survival of cases with tumours
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper proposes a new methodology that outperforms 9 other methodologies, this is a good performance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper could be much better written. See comments below.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Public data and adequate description of the methodology
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper is fairly well-written and the few suggestions that I have a on style and clarity: 1) The font in the figures is TINY! I suggest that the authors print one of their figures in Letter or A4 and try to read the figures. I have a 27 in monitor and had to maximise to be able to read. In some cases (fig 3) just making the figure larger would help, but it would be better to increase the font when the figures are produced. In Fig 1, the jet colormap for the 2 step is not good, better to change to greyscale so that the contrast between classes is clear. Again, try to print in a printer that is not colour to notice this would not print well. 2) The captions are just too short. Every caption should help to make the figure self-explanatory so that a reader can turn to the figure and understand it without having to refer back to the text, i.e. which are groups 1 and 2 in Fig 3? Add some insight in the caption, e.g., “it should be noticed that …” 3) With the tables, it is easy to get lost in the numbers. Please use bold to highlight the best results. Also use the caption to indicate which is the proposed method, this is normally added as the last one, especially since in the text all other methods are numbered 1-9, but in the table these appear in positions 2-10 4) The majority of the cases are censored, thus very few (10%) are used. What is the implication of discarding so many? What is the criteria to censor? 5) there are many typos and bad English (“patients, 46 of the them are non-censored patients…”)
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Good and interesting paper
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
Fusing imaging features with the genomic features has been used for the prognosis prediction of breast cancer in existing previous works. The contribution of this work is to use an attention mechanism for fusing imaging features with the genomic features.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The proposed approach is well motivated. Experiments are carefully done. Multiple neural networks such as U++, graph attention network, concrete autoencoder, and co-expression network, are used to process and combine imaging and genomic data. For that it became an end-to-end deep learning based framework.
Clear analysis of the results are done. Comparisons with existing approaches are done.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Nothing
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Datasets and experimental setup are clearly mentioned.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Two minor Comments:
- put space after commas and before ‘(‘. [e.g., in page7, methods(i.e.,OSCCA, DGM2FS,SALMON)]
- make labels in Fig. 2 and Fig. 3 clearer by bigger fonts.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Fusing imaging features with the genomic features by an attention mechanism is a novel part of this work.
The proposed approach is well motivated. Experiments are carefully done. Multiple neural networks such as U++, graph attention network, concrete autoencoder, and co-expression network, are used to process and combine imaging and genomic data. For that it became an end-to-end deep learning based framework.
Analysis of the results are clearly mentioned. Comparisons with existing approaches are done.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The paper presents a multi-modal framework that fuses image features with a graph attention network and genomic data by autoencoders for prognosis predictions of breast cancer. The introduction proposes a well motivated study. Multiple advanced neural networks such as U++, graph attention network, concrete autoencoder, and co-expression network, are integrated in the methodology and show novelty. Comprehensive experiment and analysis have been conducted to compare with existing SOTA method and demonstrate superior performance. The paper still need improvement in style and figure caption.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
1
Author Feedback
N/A