List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Zheyao Gao, Yuanye Liu, Fuping Wu, Nannan Shi, Yuxin Shi, Xiahai Zhuang
Abstract
Staging of liver fibrosis is important in the diagnosis and treatment planning of patients suffering from liver diseases. Current deep learning-based methods using abdominal magnetic resonance imaging (MRI) usually take a sub-region of the liver as input, which could miss critical information. To explore richer representations, we formulate this task as a multi-view learning problem and employ multiple sub-regions of the liver. Previously, features or predictions are usually combined in an implicit way, and uncertainty-aware multi-view learning methods have been proposed recently. However, the methods could be challenged to capture cross-view representations, which can be important in the accurate prediction of staging. Therefore, we propose a reliable multi-view learning method with interpretable combination rules, which can model global representations to improve the accuracy of predictions. Specifically, the proposed method estimates uncertainties based on subjective logic to improve reliability, and an explicit combination rule is applied based on Dempster-Shafer’s evidence theory with good power of interpretability. Moreover, a data-efficient transformer is introduced to capture representations in the global view. Results evaluated on enhanced MRI data show that our method delivers superior performance over existing multi-view learning methods.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43904-9_18
SharedIt: https://rdcu.be/dnwGV
Link to the code repository
https://github.com/key1589745/Multi-view_liver
Link to the dataset(s)
N/A
Reviews
Review #3
- Please describe the contribution of the paper
Authors have proposed to use multiple views of liver MRI images for fibrosis staging instead of the common single view detection. Their method uses overlapping patches of liver image as multi views combined with an uncertainty aware multi-view learning method that estimates the uncertainties of sub-views using an evidential network and combines the predictions according to Dempster-Shafer’s evidence theory. They have evaluated their method by comparing to other multi-view learning methods in the literature.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Novel method for liver fibrosis staging: using multiple patches of image, using an uncertainty-based multi-view learning method
- Clear figures with informative captions that are self-contained.
- Good explanation of the method’s details
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Could benefit from a minor language polish.
- Abstract could be organized better; motivation, objective, method.
- There is some overlap between the material in different sections; e.g. first paragraph of page is something that should be in methods section.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The data provided about the networks structure and hyperparameters are enough to reproduce the code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- It would be interesting to see how other methods of slide aggregation would work out. For instance, you could assume a multi channel data as input to some CNN-based netwrork.
- Authors could reorganize the paper so that Introduction talks only about previous work and literature, Methods talks about the current method and so on.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors have used an innovative approach to add more interpretability and better performance to liver fibrosis staging using multiple patches of MRI images.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This paper proposes a multi-view learning method for the staging of liver fibrosis. The method is uncertainty-aware, which provides good interpretation. The results and their uncertainty from the local and global views are calculated and aggregated with uncertainty-aware models and combination rules. Performance gain can be obtained compared with single-view methods or previous multi-view methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well-written and well-organized. The proposed method appears to be reasonable, practical, and of novelty. The effectiveness of the method can be well demonstrated by the conducted experiments. The quality of the paper is satisfactory and is worth-well of being accepted.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Not much weakness, my concerns and questions to the authors are listed in Q9.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Open-source is encouraged.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
Page 4, “Subjective logic serves as a principle that transforms the vector e”, what the vector e is should be described here.
-
“For opinions from K local views and one global view, the combined opinion could be derived by applying the above rule for K times, i.e., D = D1 ⊕ · · · ⊕ DK ⊕ DGlobal.” Is it the same for (D_1 ⊕ D2) ⊕ D3 and D_1 ⊕ (D2 ⊕ D3)? Why not use the average of combined opinions between every two views?
-
The local and global views are all images, so why do you use CNN for local views rather than Vision Transformer for views of both levels?
-
The descriptions and the formulations in Section 2.1 is a little bit confusing and can be improved in clarity.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well-written and well-organized. The proposed method appears to be reasonable, practical, and of novelty. The effectiveness of the method can be well demonstrated by the conducted experiments. The quality of the paper is satisfactory and is worth-well of being accepted. The idea and the method are inspirable and useful to me, and I’m looking forward to seeing the published version of the paper and the open-source code.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #1
- Please describe the contribution of the paper
A framework of multi-view learning for liver fibrosis staging was proposed in this study. The idea of multiple views is similar to image interpretation methods used by doctors. Shifted patch tockenization were employed to generate multiview images and locality self-attention facilitated the global representation modeling. Evidential networks, subjective logic, and combination rules were used to improve the performance of the proposed framework.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The idea of multiple views is similar to image interpretation methods used by doctors. Shifted patch tockenization were employed to generate multiview images and locality self-attention facilitated the global representation modeling. Evidential networks, subjective logic, and combination rules were used to improve the performance of the proposed framework. The description about the above modules were well explained and is easy to understand. The proposed method was also compared with some recent models.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- External data should be used to validate the proposed method to confirm its generalization.
- Some details of dataset are needed including the size of MRI images.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
I think the proposed method is reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- External data should be used to validate the proposed method to confirm its generalization.
- Some details of dataset are needed including the size of MRI images.
- Is it possible to divide S1, S2, S3, and S4 instead of S4 vs S1-3 or S1 vs V2-4?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelity of the method and the quality of the paper.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
An evidential multi-view framework is designed to stage fibrosis with MR images. All three reviewers and the AC consistently recognize the novelty of the proposed method. Overall, the method is designed under clear application-oriented motivation. The experimental part could be enhanced, e.g., by including more implementation details, and more analyses to justify interpreterbility.
Author Feedback
Dear Area-Chairs,
We would like to thank the meta-reviewer (M-R) and 3 reviewers (R1, R3, R4) for their very constructive and thoughtful comments, which have greatly improved our manuscript. We have summarized several main comments and given corresponding responses.
- Regarding experiments: Q: M-R stated more analyses could be included to justify interpretability. R: The proposed framework involves Ad-hoc interpretation modeling [Feng-Lei Fan et al. 2021] by designing and deploying interpretable modules into a deep learning model to address its black-box nature before training. Specifically, our method applies Dempster’s combination rule, which is interpretable, as the decision aggregation module in a multi-view learning model. With the proposed module, the trained model could explain which view of the input image contains more decisive information for liver fibrosis staging through uncertainties. In the experiment, we evaluated the quality of explanations by comparing with annotations from experienced physicians. According to Fig. 3, the critical signs of liver fibrosis are consistent with estimated uncertainties, which indicates that the proposed method could explain which input view is more important. We will add a subsection in Section 3.3 to further justify the interpretability.
Q: M-R stated more implementation details could be included and R-1 expected some details of the dataset. R: We will include more data acquisition and implementation details in Supplementary Materials and a link for the implementation code in the camera-ready manuscript.
Q: R-1 stated external data should be used to validate the proposed method to confirm its generalization. R: To our best knowledge, there is no public Gd-EOB-DTPA-enhanced MRI dataset for liver fibrosis staging. We will collect datasets from other medical centers to validate generalization in our future research.
Q: R-1 was wondering if is it possible to divide the classes as S1, S2, S3, and S4 instead of S4 vs S1-3 or S1 vs V2-4? R: For evaluation, staging cirrhosis (S4 vs S1-3) and identifying substantial fibrosis (S1 vs S2-4) is clinically important [27]. It is difficult to discriminate between S2 and S3 based on MRI since the difference is subtle. Typically, the gold standard is acquired through the analysis of liver biopsy [Won Hyeong Im et al. 2022]. For training, our experiment demonstrated that two binary classification models (84.4, 85.5 in ACC) perform better than a multi-classification model (75.4, 82.5 in ACC).
- Regarding method: Q: R-4 was wondering whether the fusion operator “⊕” satisfies commutativity and associativity and why not use the average of combined opinions between every two views? R: First, the fusion operator satisfies commutativity and associativity in our method, by setting \alpha = e+1 [10]. Second, the algebraic average of combined opinions, e.g., [(D1 ⊕ D2) + (D1 ⊕ D3) + (D2 ⊕ D3)] / 3, is not a valid opinion operation. The Averaging Fusion (AF) for opinions is derived based on the bijective mapping between the belief and evidence [10]. AF could be an option for the combination rule.
Q: R-4 was wondering why not use Vision Transformers for both local and global views. R: For local views, we expect the evidential networks to focus on detailed features. However, Vision Transformers without convolution-like structures require a large amount of data to learn the local connection [15]. Therefore, convolutional networks with locality inductive bias would be a better choice for local views.