List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Xingcan Hu, Li Xiao, Xiaoyan Sun, Feng Wu
Abstract
Glioblastoma (GBM) is the most aggressive malignant brain tumor. Its poor survival rate highlights the pressing need to adopt easily accessible, non-invasive neuroimaging techniques to preoperatively predict GBM survival, which can benefit treatment planning and patient care. MRI and MRI-based radiomics, although effective for survival prediction, do not consider brain’s functional alternations caused by tumors, which are clinically significant for guiding therapeutic strategies aimed at inhibiting tumor-brain communication. In this paper, we propose an augmented lesion network mapping (A-LNM) based survival prediction framework, where a novel neuroimaging feature family, called functional lesion network (FLN) maps generated by the A-LNM, is achieved from patients’ structural MRI, and thus are more readily available than functional connections measured with functional MRI of patients. Specifically, for each patient, the A-LNM first estimates functional disconnection (FDC) maps by embedding the lesion (the whole tumor) into an atlas of functional connections in a large cohort of healthy subjects, and many FLN maps are then obtained by averaging subsets of the FDC maps such that we can artificially boost data volume (i.e., FLN maps), which helps to mitigate over-fitting and improve survival prediction performance when learning a deep neural network from a small sized dataset. The augmented FLN maps are finally fed to a 3D ResNet-based backbone followed by the average pooling operation and fully-connected layers for GBM survival prediction. Experimental results on the BraTS $2020$ training dataset demonstrate the effectiveness of our proposed framework with the A-LNM derived FLN maps for GBM survival classification. Moreover, we identify the survival-relevant brain regions that can be traced back with biological interpretability.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_29
SharedIt: https://rdcu.be/dnwNu
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This study applies lesion network mapping to the task of overall survival classification from pre-treatment MRI data of Glioblastoma patients. The lesion network maps are derived from resting-state fMRI data from the GSP project. The lesion network mapping is benchmarked against classical ML techniques and DL approaches, outperforming the baselines. Furthermore, the authors identify relevant brain regions for each survival group by aggregating GradCAM saliency maps.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The study is well-motivated by prior research (both methodological and clinical)
- The experimental setup is described very well and is sound
- All experimental settings and tools used are sufficiently detailed that I would be able to implement it myself; good reproducibility
- The evaluation is based on multiple metrics and considers relevant baselines
- A clear benefit is demonstrated
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
No major weaknesses
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The data, experiments and evaluation are documented very well.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The authors motivate their proposed approach thoroughly considering both clinical and methodological advances from related research. The writing style and organization of the text makes it easy to follow the motivation, method, experimental setup and the study contains a clear link to a clinical interpretation in the end. The baselines selected make sense and the whole experimental setup is sensible and described in detail. I especially appreciated the clean evaluation with many baselines, reasonable parameter search space, and multiple metrics. The figures are well-prepared and contribute to the understanding of the method and results.
Details:
- You mention that FCs from rs-fMRI may suffer from the mass effect. Would it be an option to use the inverse of the transform of your SyN registration to warp your seed region to get a more accurate mapping to the corresponding area in the healthy population?
- How many components did you use for the PCA you incorporated in your comparison studies?
- Since you did a cross-validation, please consider adding the standard deviation across folds and runs such that the readers are able to also gauge the robustness/consistency of the methods. If that does not fit into the existing tables, adding it in a supplementary document would be appreciated.
- In the introduction, where you list the three steps for your approach, 1) reads like you did the manual segmentations, while in 2.1 you mention that the expert segmentations are available from the BraTS dataset. Please clarify if you did the segmentation or used the provided ones from BraTS (unless you are the BraTS segmenters/organizers of course ;-))
- In case you follow-up on the idea (probably out of scope for this MICCAI paper): I would find it very interesting to see also similiar maps as in Fig. 2 for Voxel-based lesion-symptom mapping based on the three surival time groups for a) your FLN maps, and b) the basic whole tumor maps. This would maybe give some indication on 1) how the pure lesion load/distribution compares to the connectome disruptions related, and 2) how GradCAM compared to established classical lesion-symptom mapping techniques.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed method is very clearly described, motivated, reproducible, and shows an improved performance when compared against competitive baselines.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
7
- [Post rebuttal] Please justify your decision
I raised few concerns, but they were fully addressed by the authors. I still believe the paper is novel enough in applying this technique to GBM data.
Review #3
- Please describe the contribution of the paper
The authors of the paper propose leveraging the alterations in the brain functions caused by glioblastoma to predict the survival time of the patients. In particular, the authors propose indirectly measuring the functional alterations from the readily available structural MRI rather than less common fMRI for more robust applications in clinical settings. The authors validate the model through analysis of the survival time prediction as well as identification of relevant regions of the brain for the prediction and the corresponding biological relevance.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The novel framework proposed by the authors shows promising results using feature family previously unused and achieves those results without using some of the commonly used and proven features such as radiomic features widely in use. I believe the model proposed by the author has a lot of room for improvement as using existing features, such as demographics information and radiomic features may further improve model performance.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The quantitative comparison of different prediction models on the BraTS 2020 dataset lacks information on standard deviation and/or confidence interval. A statistical significance test would have been helpful in asserting the argument the A-LNM model with FLN maps outperformed the traditional machine learning models.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The data used were clearly described in their composition and the subjected imaged, and the preprocessing applied are clearly outlined. The feature generation is clearly described with details of the methodology, and the deep learning model used is noted. The authors note that the code will be released upon acceptance.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
I believe this paper is solid work both in terms of novelty as well as its analysis. Other than providing additional statistics in Table 1 and 2 for quantitative comparisons, I cannot find area to improve in the scope of submission to the conference. The future direction stated by the authors regarding combining FLN maps and radiomics features seems to be a promising extension of the existing work.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I believe the authors provide a novel framework for predicting GBM patients’ survival time with functional brain changes acquired from structural MRI. There is merit in the clinical settings as the paper addresses concern about the shortage of functional MRI in clinical settings for GBM patients. Furthermore, the method proposed in the paper has potential and can contribute to other GBM survival time prediction frameworks because it provides additional features to predict survival on top of the existing, widely used radiomics features.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
- The paper is based on survival prediction using MRI and fMRI data.
- The work proposed Augmented Lesion Network Mapping (A-LNM) to derive better features and predication for survival of patients.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The manuscript is based on predication of survival for GBM patients.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The manuscript is difficult to follow for reading.
- There should be discussion on LNM for readability.
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
- May be reproducible given hyper parameters
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- The manuscript is difficult to follow.
- There should be discussion on FLN maps and how its generated.
- While going through the manuscript, it is difficult to follow the notion of augmentation in LNM, which is represented as A-LNM.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
3
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The work does not seems to have significant technical contribution. It is difficult to follow the manuscript.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors have given justification based on query raised. However, the concern is the technical contribution and novality of the work proposed in the manuscript.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
I agree with the reviewer#4 that most details of the framework is missing, like LNM and FLN. Besides ResNet, it seems the authors did not propose any methodologies in this paper. The authors should clarify their technical and methodology contribution in the rebuttal.
Author Feedback
We appreciate the meta-reviewer (MR) and reviewers (R) for their constructive feedback, according to which we will reinforce our paper in the version if we have the chance. We clarify main concerns as follows.
R1&R3: We thank the reviewers for positive comments. [PCA Components] We kept those PCA components explaining over 95% variance ratio in the comparison studies. [Statistical Analysis] We conducted paired t-test on the results of our A-LNM and the other models used. Our A-LNM achieved p<0.05 consistently, showing statistically significant improvement in performance. Standard deviations to all the results were also calculated. We will add them in the revision. [Segmentation] In Introduction, we presented our A-LNM with respect to a general GBM dataset, not specific to BraTS 2020. In Methods, we focused on BraTS 2020, and used the manual expert segmentation labels provided by BraTS 2020. [Others] We referred to the mass effect of tumors just for FC estimation from patients’ fMRI. In our A-LNM, we effectively avoid it. As suggested, we will check the difference of survival-related brain regions identified by pure lesion overlapping, voxel-based lesion-symptom mapping, FLN overlapping, and GradCAM. Compared with the former two schemes, the latter two FC based schemes are likely to identify healthy regions besides regions in tumor.
R4&MR: [Hyperparameters] The hyperparameters of our model were given in Sec. 3.1, while the competing models followed the instructions as in the references to achieve the best performance; see Sec. 3.2. [Details of LNM, FLN, and A-LNM] We clarify that LNM and A-LNM are approaches, and FLN maps are features produced by LNM or A-LNM. Specifically, for each patient, LNM requires embedding the lesion/tumor into a normative functional connectome and computing FC between the lesion and the rest of the brain from fMRI of all N healthy subjects in a large cohort. The resulting N functional disconnection (FDC) maps caused by the lesion on brain network are averaged to produce an FLN map for the patient. In our A-LNM, we partition the N FDC maps into M disjoint subsets of equal size and average FDC maps in each subset. As a result, we obtain M FLN maps for each patient. Such data augmentation helps mitigate over-fitting and improve the performance when training a deep prediction model. We will highlight the above details in the revision. [Technical Contribution] We would like to emphasize that the primary contribution of this paper is in exploring novel neuroimaging features, i.e., A-LNM derived FLN maps, which are not only more readily available than FC estimated from patients’ fMRI, but also more discriminative than other widely used features for survival prediction. First, compared with previous studies [7,16,24] using patients’ fMRI to estimate FC, our A-LNM indirectly estimates FC abnormalities (i.e., FLN maps) caused by the lesion using the tumor segmentation label and fMRI from healthy subjects in a large cohort. It can overcome two efficiencies of the previous studies: 1) incorrect FC estimation from patients’ fMRI due to mass effect and physical infiltration of tumors; and 2) low statistical power due to limited sample size, as patients’ fMRI data are not routinely collected for cancer clinical practices. In addition, our A-LNM improved the survival classification accuracy by 10.5% to 18.5% over LNM; see Table 1. We also validated the effectiveness of A-LNM derived FLN maps by comparing with other types of features (i.e., clinical, biophysics and radiomics features, and MRI images); see Table 2. We just used 3D ResNet to show the effectiveness of our A-LNM, and developing other methods seems unncessary in our opinion. Altogether, to our knowledge, this paper is the first to apply and improve LNM to study the relationship between functional alternations caused by tumors and survival time, which is clinically significant for therapeutic strategies aimed at inhibiting tumor-brain communication.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After reading the rebuttal, especially the last section, I do not think this paper have sufficient technical contribution and convincing results. If the major goal is to exploring novel neuroimaging features, the results should focuse on comparing to other features that are widely used. However, in this paper, the authors compared 4 types features+different classification models, whereas the proposed features used another model - resnet. It is not clear what conclusions can achieve when using different models for different features.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Overall this is a well-written paper with minor concerns from the reviewers. The rebuttal has successfully convinced R4 to increase the review score. It would be a good fit for publication at MICCAI.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper proposed a lesion network mapping to the task of overall survival classification from pre-treatment MRI data of Glioblastoma patients. Most of reviewers agree with the clinical use and interest to the MICCAI field, although the authors have not provided a very convincing rebuttal to address the key questions and concerns on the technical and methodology contribution. I recommend acceptance of this paper.