List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Ke Yan, Xiaoli Yin, Yingda Xia, Fakai Wang, Shu Wang, Yuan Gao, Jiawen Yao, Chunli Li, Xiaoyu Bai, Jingren Zhou, Ling Zhang, Le Lu, Yu Shi
Abstract
Liver tumor segmentation and classification are important tasks in computer aided diagnosis. We aim to address three problems: liver tumor screening and preliminary diagnosis in non-contrast computed tomography (CT), and differential diagnosis in dynamic contrast-enhanced CT. A novel framework named Pixel-Lesion-pAtient Network (PLAN) is proposed. It uses a mask transformer to jointly segment and classify each lesion with improved anchor queries and a foreground-enhanced sampling loss. A patient branch further aggregates information from the whole image and predicts image-level labels. A large-scale multi-phase dataset is collected containing 939 tumor patients and 810 normal subjects. 4010 tumor instances of eight types are extensively annotated. On the non-contrast tumor screening task, PLAN achieves 95% and 96% in patient-level sensitivity and specificity. On contrast-enhanced CT, our lesion-level detection precision, recall, and classification accuracy are 92%, 89%, and 86%, outperforming widely used CNN and transformers for lesion segmentation. We also conduct a reader study on a holdout set of 250 cases. PLAN is on par with a senior human radiologist, showing the clinical significance of our results.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43904-9_8
SharedIt: https://rdcu.be/dnwGL
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a novel framework that segments and classifies liver tumors in CT images.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The paper is well-written and organized. It also fits well the conference. The methods utilized in the research have been described in great detail across several sections, making it easier for readers to understand the technical nuances. The results and figures presented in the paper are appropriate and effectively showcase the findings of the study. Additionally, the references cited are sufficient and provide an adequate foundation for the research. Overall, the paper represents a significant contribution to the field and is a valuable addition to the conference.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The paper would benefit from a more comprehensive comparison with existing literature, as well as a more detailed discussion of the advantages and disadvantages of the proposed method.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
This paper meets the reproducibility requirements. It gives details about the implementation that allows us to reproduce the experiments.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
This paper implements correctly the methodology. It acquires the image, process, and classify. The experiments use well-known metrics and describe well the overall method performance. The comparison with the pathologist’s diagnosis and the visual classification of the exams demonstrates its real-world capabilities. The only suggestion would be to compare these results with the other works in the literature.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The well-founded approach presented in this study utilizes cutting-edge technologies to tackle a well-known problem, resulting in excellent results. Therefore, it is deemed a major contributing factor for the overall score.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper presents a segmentation and classification network based on a mask transformer for liver tumor using CT images. So, a screening and preliminary diagnose (non-contrast CT) and a later segmentation and classification (dynamic contrast-enhanced CT) can be performed. Both a pixel and patient levels of diagnose are included in this network.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- Implementation of a versatile network that can be used with contrast and non-contrast CT images.
- Good approach with clinical relevance (patient-level diagnose)
- (from my point of view) High potential impact in tumor screening (non-contrast CT)
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Not sure that lesion-level results will be good enough
- Clear poor performance (recall) for small radius. How can this results for <10mm be improved in a future?
- Reduced dataset. I perfectly know restrictions in this point but I would strongly recommend to increase the number of subjects in the future.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No additional comments.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This novel network works properly with several kind of images. Although small lesions and ‘difficult cases’ are not detected yet, the potential of this solutions could highly increase the clinical outcomes. In this sense, the comparison with junior and senior radiologist is highly promising.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
The author proposed a transformer based liver tumor segmentation network consists of 3 head for patient & tumor & and pixel based tumor segmentation && classification.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
clear logic on network design.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- author claimed that they added healthy volunteer for liver tumor patient screening. However, they patient/volunteer ratio is almost 1:1, which is way too artificial in real world. Since in practice, the screening is facing a super imbalanced scenario where we have less patient compared to volunteer.
- limited innovation.
- The author did not even give an visualization of their model results.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
not sure
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- used unbalanced dataset
- add figure to show your result.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
2
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
not a sound/well structured paper at all.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Key Strength: 1) Well-written and organized, and easy to follow. 2) The method is comprehensive with three different levels of decoder header to achieve pixel-level, lesion-level, and image-level classification/segmentation. 3) Experimental results are convincing and the methods are potentially used for both contrast and non-contrast CT images.
Weaknesses: 1) The issue of data unbalance in the experimental dataset. 2) The discussion of the method’s strength and weakness.
Author Feedback
We thank the Meta Reviewer (MR) and all reviewers for their constructive comments.
We propose Pixel-Lesion-Patient Network (PLAN), an end-to-end framework for joint lesion segmentation and classification. It is applied to liver tumor screening and diagnosis with non-contrast (NC) CT and dynamic contrast-enhanced (DCE) CT. Comprehensive experiments on a large-scale dataset validate PLAN’s superiority compared to strong baselines. We also conduct a reader study and show PLAN is on par with a senior radiologist with 16 years of experience. Our code will be released upon acceptance to promote reproducibility.
Strength and weakness of the method (MR, R1): 1.We pioneer to adapt the mask transformer paradigm for lesion segmentation and classification in 3D medical images. PLAN can leverage query-image cross-attention to explore global features for each lesion, meanwhile it uses query-query self-attention to exploit inter-lesion relations, achieving better lesion-wise precision, recall, and accuracy than the traditional nnU-Net. 2.To improve the mask transformer in the lesion branch, we propose a pixel branch and use anchor queries to bring prior information to the lesion branch, and a foreground-enhanced sampling loss to improve its recall. 3.We propose a patient branch with a lesion-patient consistency loss to aggregate image-wise information for improved patient-level diagnosis. All the novel strategies above have been verified by experimental results in patient, lesion, and pixel levels. (R4 about novelty) Limitations: PLAN has lower recall for small lesions (many other methods so). This is because small liver lesions are subtle in the image and sometimes hard to be distinguished from normal hypodense liver appearances. We will collect more data with small lesions and study algorithms to mitigate this issue, e.g., by zooming in local features. (R3 about small lesions) We will elaborate these aspects in the final paper.
Data imbalance (MR, R4): Thank you for pointing out this issue. In training, a relatively balanced tumor/normal subject ratio is helpful for the network to learn tumor features effectively. We agree our current ratio (tumor/normal=2/3) may not reflect real world screening setting, but it can reflect real world sensitivity and specificity largely in the scenerio of differential diagnosis. We are collecting more normal data for internal and external validation. Our latest experiments on 4K normal CT from another hospital have yielded similar specificity. Collecting medical data requires large efforts. In this paper, we have gathered a dataset with 910 normal subjects and 1089 patients (including reader study), which is larger than most existing works [1, 5, 20, 22, 24]. We will continue to add more data in the future. (R3 about more data)
Compare results with literature (R1): Since different data and metrics are used by different papers, we can only roughly compare our results with others.
- Using NC and DCE CTs, we obtain pixel-wise Dice of 77.2% (NC) and 84.2% (DCE) in tumor segmentation, respectively.
- The current state of the art (SOTA) of LiTS [1] achieved 82.2% in Dice using CTs in venous phase.
[22] achieved 81.3% in Dice using DCE CT of two phases.
- Our lesion-wise precision and recall are 80.1% and 81.9% for NC CT, 92.2% and 89.0% for DCE CT.
- [24] achieved 83% and 93% in lesion-wise precision and recall for DCE CT.
SOTA of LiTS [1] achieved 49.7% and 46.3% in lesion-wise precision and recall at 50% overlap.
- [20] classified lesions into 5 classes, achieving 84% accuracy for DCE and 49% for NC CT. We classify lesions into 8 classes with 85.9% accuracy for DCE and 78.5% for NC CT.
- [5] achieved AUC=0.75 in NC CT tumor screening, while our AUC is 0.985. In summary, our results are superior or comparable to existing works in all metrics (R3 about lesion-level results).
Visualization (R4): We have provided visualization in the original supplementary material, and mentioned it in the main paper.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors have successfully addressed the concerns raised by all reviewers. Regarding to data imbalance, it is normal to collect more diseased cases for model training although the ratio is quite different from the reality. Discussion is also enough to cover the strength and weakness of the paper.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
Presents a pixel-level, lesion-level, and image-level classification/segmentation approach. Strengths include well written paper, comprehensive logical approach, and convincing results on multiple types of CT as well as reader study. Major critique on better delineation of strengths and weaknesses is well addressed in rebuttal. Critique about use of artificially balanced dataset (comprising ~2000 cases) is a tad unfair; the cohort size + comprehensive evaluation is sufficient to demonstrate the promise of the method. Would suggest including the comparison to literature in the final manuscript.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The authors rebuttal is complete to the raised points and very specific. Their rebuttal regarding the comparison with related method is convincing too.