List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Di Liu, Yunhe Gao, Qilong Zhangli, Ligong Han, Xiaoxiao He, Zhaoyang Xia, Song Wen, Qi Chang, Zhennan Yan, Mu Zhou, Dimitris Metaxas
Abstract
Combining information from multi-view images is crucial to improve the performance and robustness of automated methods for disease diagnosis. However, due to the non-alignment characteristics of multi-view images, building correlation and data fusion across views largely remain an open problem. In this study, we present TransFusion, a Transformer-based architecture to merge divergent multi-view imaging information using convolutional layers and powerful attention mechanisms. In particular, the Divergent Fusion Attention (DiFA) module is proposed for rich cross-view context modeling and semantic dependency mining, addressing the critical issue of capturing long-range correlations between unaligned data from different image views. We further propose the Multi-Scale Attention (MSA) to collect global correspondence of multi-scale feature representations. We evaluate TransFusion on the Multi-Disease, Multi-View & Multi-Center Right Ventricular Segmentation in Cardiac MRI (M&Ms-2) challenge cohort. TransFusion demonstrates leading performance against the state-of-the-art methods and opens up new perspectives for multi-view imaging integration towards robust medical image segmentation.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16443-9_47
SharedIt: https://rdcu.be/cVRy1
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The paper has presented a transformer architecture to combine cross-view information in medical images that can help in segmentation. The TransFusion approach proposed by the authors is able to capture long range dependencies between different scales and views.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The problem is well-formulated. The authors claim that the size, modality and even dimension can be different among views. This is an interesting take.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The average runtime for each result, or estimated energy cost is missing. The motivation for choosing transformers based architecture for the task is not well elaborated. Since the model is evaluated on cardiac MRI only, it is not appropriate to claim that it is a medical image segmentation model. Accordingly, the title and text should be updated and made specific.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors have chosen YES in response to all the questions but I am not sure if they address the questions. For example, the scenarios where the approach may fail are not clearly reported. The average runtime for each result, or estimated energy cost is missing.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
In last para of Introduction, I would suggest to link the terms with the figure. For example, the terms such DIFA and MSA are mentioned with no background on where these terms come from (though they are later described in the overall method). The authors should also highlight the limitations of the approach with examples where the proposed method fails to work. Similarly, information on required training resources and how these compare with other methods should be included.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
I think the problem is nicely described. The method seems novel. I am of the opinion that it will generate good discussion. There are some limitations in the way the paper is presented that I have highlighted in my comments for the authors (discuss limitations and provide comparison for computational resources).
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Somewhat Confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
In this work, the authors have proposed TranFusion algorithm for the segmentation of Right Ventricles from cardiac-MRI. The proposed algorithm use to merge the divergent information from multiple views and scale. using attention mechanism.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Following are the strength of the paper:
- The paper is well written and the work flow is explained properly.
- The novelty of the work is satisfactory.
- The algorithm was tested on multi-centric, multi-View and Multi-disease data.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Following are the weakness:
- It would be great, if the authors would provide the computation details like the training time of all the methods use to compare with TranFusion.
- Different types of U-Net (TranU-Net or Res-U-Net or vanilla U-Net) were compared. It would be nice if it is compared with U-Net having attention block. Though, Trans U-Net has the transformer component but still it would be interesting to see the comparision with Attention U-Net.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The paper looks reproducable from the response. The proposed algorithm is validated on the multi-disease and -centric data, hence the paper is reproducable.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The proposed method looks interesting and the validation is also proper. Kindly, include the computation details of the proposed methods as well as the methods used for comparision including the hardware details.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The novelty and validation of the proposed method looks good to me. The paper is well written and structured.
- Number of papers in your stack
5
- What is the ranking of this paper in your review stack?
1
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #4
- Please describe the contribution of the paper
This manuscript proposes TransFu-sion, a Transformer-based architecture to merge divergent multi-view imaging information using convolutional layers and powerful attention mechanisms for automated medical image segmentation.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- The proposed method is crucial to improve the performance and robustness of automated methods for disease diagnosis, which combines information from multi-view images.
- The authors propose a divergent fusion attention (DiFA) module to handle the multi-view inputs.
- A Multi-Scale Attention (MSA) is used to collect global correspondence of multi-scale feature representations.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- The Fig.2 misses some detailed explanations, and it is hard to understand.
- The proposed DiFA and MSA modules seems to be the increment from the single input to the multi-view inputs.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Reproducibility of the paper is good.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The proposed attention modules seem to be much heavier than the baselines. The model computations and time consuming may be considered for comparison experiments.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The proposed modules are effective for multi-view inputs and the conducted experiments are solid.
- Number of papers in your stack
8
- What is the ranking of this paper in your review stack?
3
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors have proposed TranFusion algorithm for the segmentation of heart cardiac-MRI. All reviewers agreed on the novelty and clear presentation of this work. Congratulations! In the final version, please update the details of implementation and computation details.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
3
Author Feedback
Thank you for all the comments from reviewers and meta-reviewer. I have carefully revised the paper and updated it.