List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Shouyu Chen, Xin Guo, Jianping Zhu, Yin Wang
Abstract
Three-dimensional medical images are crucial for the early screening and prognosis of numerous diseases. However, constructing an accurate computer-aided prediction model is challenging when dealing with volumes of different sizes due to numerous slices (native nodes) in a single case and variable-length slice sequence. We propose a Global Semantic-guided Dual-stream Graph model to address this issue. Our approach differs from the existing solution that aligns volumes with varying numbers of slices through downsampling. Instead, we leverage global semantic vectors to guide the grouping of native nodes, construct super-nodes, and build dual-stream graphs by incorporating the sequential association of each volume’s unique slices and the feature association of global semantic vectors. Specifically, we propose a shared global semantic vectors-based grouping method that aligns the number of nodes and the semantic distribution of nodes among different volumes without discarding slices. Furthermore, we construct a dual-stream graph module that enables Graph Convolutional Networks (GCN) to make clinical predictions from computer tomography (CT) volumes through the natural sequence association between native nodes and, simultaneously, the latent feature association between semantic vectors. We provide interpretability by visualizing the distribution of native nodes within each group and weakly-supervised slice localization. The results demonstrate that our method outperforms previous work in diagnostic (96.74%, +2.81%) and prognostic accuracy (84.56%, +1.86%) while being more interpretable, making it a promising approach for medical image analysis scenarios with limited fine-grained annotation.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43904-9_45
SharedIt: https://rdcu.be/dnwHq
Link to the code repository
N/A
Link to the dataset(s)
http://ncov-ai.big.ac.cn/download?lang=en
Reviews
Review #1
- Please describe the contribution of the paper
A graph-based neural network model to handle classification of COVID-19 CT images that have variable numbers of slices is proposed. By aligning/grouping slices to semantically meaningful nodes and solving a graph classification task.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper addresses an important and common problem in medical imaging and specifically CT classification, namely, how to handle a variable number of slices. In particular, a novel unsupervised slice grouping method is developed and combined with a graph classifier. The method quantitatively outperforms previous works, and the paper is easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- What is the difference between the prognostic and diagnostic probabilities?
- It seems the paper is solving a classification problem (predict whether a given CT volume represents a patient that has COVID-19). Why is this problem referred to as weakly supervised, rather than supervised?
- More details about the comparator method [21] are needed. In particular, [21] is a 3D CNN method, and it would be helpful to highlight that 3D CNNs might require use of more-memory intensive and large-parameter models, and why might a graph-based approach outperform a 3D CNN (that includes all slices anyway).
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It appears all the details for reproducibility are included.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- It would be helpful to see how classification accuracy varies across volumes by number of slices.
- Has the semantic grouping of the nodes been evaluated by a clinician? While the anatomy from the examples in the Supplementary material (Fig. 1-3) seems matching, it would be helpful to obtain clinical feedback.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Overall, the paper is well-written, addresses an important problem, and introduces graph-based grouping of nodes to address frame number variability in CT.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
After reading reviews and author feedback, I believe the paper offers an interesting an important innovation over previous work and its experimental results are well validated. It would be a valuable contibution to the conference.
Review #3
- Please describe the contribution of the paper
A method to perform weakly-supervised volume-level classification combining convolutional feature extractor with a graph neural network encoding relationships between slides. In order to support a variable number of slices, each element in the graph is a super-node associated to an homogeneous set of slices. Compared to previous solutions, the proposed model incorporates two graph modules, one at the volume and one at the slice level, and groups graph nodes into super-nodes based on sematic similarity. Experiments are conducted on COVID-19 prognosis from chest CT.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Architectures combining CNN and GCNN represent an elegant solution to the problem of integrating information across multiple slices and locations in order to classify an entire volume, while selecting the key slices for the classification. While experiments are limited to COVID-19 prognosis, application can be easily extended to other diseases
- Ablation studies indicate a performance advantage in introducing two adjacency matrices (local and global)
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Progress is incremental with respect to previous work (Liu, C., Cui, J., Gan, D., Yin, G.: Beyond covid-19 diagnosis: Prognosis with hierarchical graph representation learning. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 283–292. Springer (2021)). Differences are minor from an architectural point of view and performance increase is limited (ACC from 93.93 +/-0.41 to 96.74 +/- 0.64).
- Although the paper is generally well written, the methodology section is not perfectly clear; in particular, the motivation for introducing a dual stream GCNN is not clear, as well as which components are trained end-to-end and which on the other hand are pretrained.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Key hyper-parameters are clearly state and the experiments are conducted on public datasets. The authors plan to release the code - reproducing the architecture and training loop solely from the description would be challenging.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Major remarks
- Are super-nodes constructed while training, in an end-to-end fashion, or are they constructed in a pre-processing phase, prior to training the GCNN?
- How were hyper-parameters specific to the proposed method selected? For instance, what is the rational for setting the number of super-cluster K=6 and NG= 2.
- Section 3.3 and 3.4 refer to several tables/figures in the supplementary material, making it difficult to follow.
Minor remarks
- In the introduction, the sentence “CT slices have intrinsic non-Euclidean associations” would benefit from additional details and/or a supporting reference -(Introduction ) Unclear sentence: One of the major challenges is achieving consistency training in GCNs while preserving the integrity of the slice facts. (what are the slice facts? and what is meant by consistency in training (I presume))
- In Equation 3 and should not be formatted in italics
- (Section 2.3) sentence not clear: Precisely, the semantic vectors ci and cj are concatenated and input to a link predictor, which is constructed by a 2-layer MLP, then output θ(i,j). (is the link predictor that outputs θ(i,j)?)
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Even though the contribution is smaller (the paper extends a previous MICCAI paper), the methodology may be of interest to many researchers working on CT analysis.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
I stand by my previous assessment. If accepted, many of the points addressed in the rebuttal could/should clarified in the final submitted manuscript.
Review #4
- Please describe the contribution of the paper
This work propose a Global Semantic-guided Dual-stream Graph model to make clinical predictions from computer tomography (CT) volumes. They provide interpretability by visualizing the distribution of native nodes within each group and weakly-supervised slice localization.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed method is better than some SOTA methods by developing a dual-stream base graph module incorporating the local slice sequence and global semantic knowledge.
- The experiment evaluates the effectiveness of their proposed method through comparison and ablation experiments.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- In classification, why you don’t choose famous graph-based methods like GAT or GraphSAGE as the baseline. The table doesn’t have the sensitivity and specificity score. Do you conduct k-fold cross validation?
- The explanation of slices localization comparison between normal and CP case are too short. Is there any evidence to support your finding and result?
- In ablation study, it is unclear whether the result of ablation study is for diagnosis task or prognosis task. Why the table doesn’t have the ACC and AUC score?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The proposed method should be reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
see above
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The classification result is unclear and the comparison of interpretation result are limited.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
5
- [Post rebuttal] Please justify your decision
The authors have answered my concern and I decided to increase my rating to weak accept, based on all reviewer’s comments and the responses of the authors.
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This paper proposed to combine GCNs with CNN for classification of COVID-19 CT images. There are several concerns. First, previous work already has a sound performance on this task and thus the progress seems incremental. Second, some method details could be added as suggested by all reviewers.
Author Feedback
We thank the reviewers for their invaluable feedback on this paper. We address the common questions (CQ) first and then specific concerns each reviewer highlights. CQ1: Explanation regarding “CT slices have intrinsic non-Euclidean associations” and “why might a graph-based approach outperform a 3D CNN”. CA1: We found lesions are not necessarily local, nor did they necessarily have a stable adjacent pattern. Therefore, the mutual correlation of the slices is not well described and processed by Euclidean data (such as 3d pixel grids) and models (such as CNNs with fixed receptive fields). The advantage of GNNs over CNNs is the long-distance modeling between slices. We retain all the slices because systematic sampling [7] removes slices and disrupts the data’s integrity, lowering the performance. Reviewer #1 Q1. Difference between prognostic and diagnostic probabilities? A1. The former comes from the 1 output node + sigmoid activation, while the latter comes from the 3 output nodes + softmax. Q2. Why is this problem referred to as weakly supervised rather than supervised? A2. When localizing the slices, y is still a volume-level category label without providing any positional information about slices, so the model needs to explore fine-grained positional information from the coarse-grained y. Q3. Details about the comparator method [21]. A3. [21] first train a segmentation net using segmentation masks. Then, a fixed-size 3d tensor is cropped from the lung region, which is transformed into a 4d tensor and input into the 3d classification net. For an explanation of the advantages of graph-based methods, please refer to CA1. Reviewer #3 Q1. Progress is incremental concerning previous work [7]. A1. The only similarity is that we use the SLICE importance method in [7] to determine GROUP importance. Our unique intuition is that the systematic sampling [7] disrupts the data integrity, so our model adopts a completely different approach from [7] regarding graph node and edge construction and model design. Q2. The motivation for introducing a dual-stream GCNN is unclear. The training manner of each component? A2. Motivation: We aim to train GCN on variable-length volumes datasets. Therefore, we group the native nodes into super nodes H. Then, we have semantic vectors and native nodes to depict the adjacency relationships between nodes in H. Inspired by the multi-resolution model design in CNNs; we explicitly model the global and local adjacency relations. Only the feature extractor is pre-trained on ImageNet, while all other modules are trained end-to-end. Q3. Are super-nodes constructed while training, in an end-to-end fashion? A3. Yes. Q4. How to select hyper-parameters? A4. The selection of hyperparameter values is mainly based on experience and constraints in the formula. Q5. More details about “CT slices have intrinsic non-Euclidean associations”. A5. Please refer to CA1. Q9. Is the link predictor that outputs θ(i,j)? A9. Yes. Reviewer #4 Q1. Why not use (1) GAT or GraphSAGE as the GNN baseline, (2) sensitivity and specificity scores, and (3) k-fold cross-validation? A1. They are only functionally similar to the GCN backbone rather than our complete model. We choose a mature GCN-based work [7] and their AUC and ACC metrics for comparison. We report the avg, std (or 95% CI) scores of 20 experiments to reflect the model’s average performance. Q2. More evidence to support slice localization? A2. In addition to the results and analysis in Section 3.2, we provide visualizations of localization for NCP and CP cases in Figures 4 and 5 of the supplementary materials. Q3. In the ablation study, whether the result is for diagnosis or prognosis task is unclear. Why doesn’t the table have the ACC and AUC scores? A3. We report the ablation results for diagnosis, prognosis, and localization tasks, respectively, in Tables 1 and 2 of the supplementary materials. The ‘D-‘ and ‘P-‘ in Table 1 indicate diagnosis and prognosis.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Previous concerns are addressed in the rebuttal and all reviewers now approve for this work.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The work proposed a graph-based neural network model for clinical predictions from computer tomography (CT) volumes. The authors have been responsive in the rebuttal. Two of three reviewers have raised their scores. It is recommended for publication in MICCAI 2023.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
All reviews agree with the acceptance of this paper, and the authors should carefully revise their final version to address many of the points raised by reviewers.