List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Junde Wu, Huihui Fang, Fangxin Shang, Dalu Yang, Zhaowei Wang, Jing Gao, Yehui Yang, Yanwu Xu
Abstract
Clinically, the accurate annotation of lesions/tissues can significantly facilitate the disease diagnosis. For example, the segmentation of optic disc/cup (OD/OC) on fundus image would facilitate the glaucoma diagnosis, the segmentation of skin lesions on dermoscopic images is helpful to the melanoma diagnosis, etc. With the advancement of deep learning techniques, a wide range of methods proved the lesions/tissues segmentation can also facilitate the automated disease diagnosis models. However, existing methods are limited in the sense that they can only capture static regional correlations in the images. Inspired by the global and dynamic nature of Vision Transformer, in this paper, we propose Segmentation-Assisted diagnosis Transformer (SeATrans) to transfer the segmentation knowledge to the disease diagnosis network. Specifically, we first propose an asymmetric multi-scale interaction strategy to correlate each single low-level diagnosis feature with multi-scale segmentation features. Then, an effective strategy called SeA-block is adopted to vitalize diagnosis feature via correlated segmentation features. To model the segmentation-diagnosis interaction, SeA-block first embeds the diagnosis feature based on the segmentation information via the encoder, and then transfers the embedding back to the diagnosis feature space by a decoder. Experimental results demonstrate that SeATrans surpasses a wide range of state-of-the-art (SOTA) segmentation-assisted diagnosis methods on several disease diagnosis tasks.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16434-7_65
SharedIt: https://rdcu.be/cVRsx
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
This paper presents a transformer-based architecture called SeATrans by utilizing the segmentation information to boost the diagnosis task. The key techniques include: asymmetric multi-scale interaction and SeA-block for the segmentation-diagnosis interaction. The experimental results improve a large margin compared with other SOTA methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The motivation is good by segmentation task to boost the classification task. Especillay with the non-regional information.
- The experiment results are very good.
- The method includes the Transformer to deal with segmentation-diagnosis feature interaction.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Could you explain more (or give an example) on the non-regional relationship between the segmentation and diagnosis task?
- Visualization of the relationship between segmentation and diagnsosis tasks may help to understand the reason behind the proposed method for performance improvement.
- Missing comparison with some papers which designed for the down-stream task learning, like MoCo, SimCLR, etc.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No code is provided now. Some training strategies have been described in the Experiemnetal part but not support for the reproducibility of the paper. Since this work is mainly based on the network design. The code release would benefit other researchers.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
See above.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The results.
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
The author proposes a new transformer model for medical image diagnosis, which is named SeATrans. SeATrans is a transformer model equipped with multi-scale feature integration architecture that achieves promising results on three different tasks.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The author proposes asymmetric multi-scale interaction to correlate each low-level diagnosis feature with multi-scale segmentation features. The author supplies a variety of experiments to validate the effectiveness of different components in SeATrans. The experiments show that SeATrans can achieve state-of-the-art performance on three publicly available datasets.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The state-of-the-art methods compared in Table 2 lack the reference citation. The author failed to analyze the efficiency of proposed methods with recent works.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The three datasets are available. The code is not available.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The author is suggested adding reference citation for those state-of-the-art methods compared in Table 2 and supplying some analysis of the efficiency difference between proposed methods and recent transformer models. Meanwhile, If the authors can provide some visual comparisons of the results from proposed method and recent models, the quality of this paper can be further improved
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The idea is interesting.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
The authors proposed a general framework for segmentation-assisted disease diagnosis. Their method consists of two jointly trained networks, one UNet to extract multi-scale segmentation-related features and one ResNet50 that performs the classification without the use of segmentation masks. They have proposed to combine the coarse and fine segmentation features of UNet to the first layers of the Resnet50 and use a transformer-based encoder-decoder block to learn the combined space. They validate their method on three public datasets from three different domains, outperforming the baselines.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Generalizability – Their method doesn’t make any prior assumptions about the medical images it is being used on. So it can be used on any kind of images. Novelty – Although the authors have used the well-known networks of UNet and ResNet, they have joined the multi-scale segmentation features and the high-level features extracted by the first layers of ResNet, in a novel way to account for the non-regional feature dependencies missed by the convolutional layers. Extensive validation – The authors have validated their performance by comparing to many existing methods that make the disease diagnosis based on the segmentation annotations.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Not well-written – The authors repeat their exact words many times. An example is the last two paragraphs of the introduction that are written very much similarly. Or the methodology starts with what the authors propose. This is stated in the introduction and now they should start explaining their methods.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors have used public data and they will make their code public, according to their answers in the checklist. So I believe their work is reproducible.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
-
I believe the authors should provide references for those “commonly used segmentation-assisted diagnosis techniques” in the second paragraph of “Experimental Settings”.
-
I believe the authors should add some info about the distribution of data. The sensitivities reported are the lowest among all the metric and I wonder if it is related to the imbalancy in the data.
-
I believe the authors should perform some statistical analysis to establish whether the improvement reported by their method is in fact statistically significant.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The authors have used an innovative approach to fuse the segmentation masks into the classification network. They have validated their work by comparing to a large number of existing methods. And their method outperforms these baselines in terms of AUC. They have also investigated the effectiveness of adding each part of their proposed method by an ablation study.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
All reviewers agreed on the technical contribution of the approach and the experimental validation on public datasets adequate, recommending acceptance unanimously. The final version of the paper should include reviewers’ comments, in particular: add reference in Table 2, additional experimental details, and a visualization results compared with other methods.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
2
Author Feedback
We sincerely thank the reviewers for their high-quality reviews and constructive feedback on our manuscript. We are happy to learn that all reviewers appreciate our motivation and novelty. The final version will be revised as suggested.