List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Amine Amyar, Shiro Nakamori, Manuel Morales, Siyeop Yoon, Jennifer Rodriguez, Jiwon Kim, Robert M. Judd, Jonathan W. Weinsaft, Reza Nezafat
Abstract
Gadolinium-based contrast agents are commonly used in cardiac magnetic resonance (CMR) imaging to characterize myocardial scar tissue. Recent works using deep learning have shown the promise of contrast-free short-axis cine images to detect scars based on wall motion abnormalities (WMA) in ischemic patients. However, WMA can occur in patients without a scar. Moreover, the presence of a scar may not always be accompanied by WMA, particularly in non-ischemic heart disease, posing a significant challenge in detecting scars in such cases. To overcome this limitation, we propose a novel deep spatiotemporal residual attention network (ST-RAN) that leverages temporal and spatialinformation at different scales to detect scars in both ischemic and non-ischemic heart diseases. Our model comprises three primary components. First, we develop a novel factorized 4D (3D+time) convolutional layer that extracts 3D spatial features of the heart and a deep 1D kernel in the temporal direction to extract heart motion. Secondly, we enhance the power of the 4D (3D+time) layer with spatiotemporal attention to extract rich whole-heart features while tracking the long-range temporal relationship between the frames. Lastly, we introduce a residual attention block that extracts spatial and temporal features at different scales to obtain global and local motion features and to detect subtle changes in contrast related to scar. We train and validate our model on a large dataset of 3000 patients who underwent clinical CMR with various indications and different field strengths (1.5T, 3T) from multiple vendors (GE, Siemens) to demonstrate the generalizability and robustness of our model. We show that our model works on both ischemic and non-ischemic heart diseases outperforming state-of-the-art methods. Our code is available at https://github.com/HMS-CardiacMR/Myocardial_Scar_Detection.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43895-0_60
SharedIt: https://rdcu.be/dnwzs
Link to the code repository
https://github.com/HMS-CardiacMR/Myocardial_Scar_Detection
Link to the dataset(s)
N/A
Reviews
Review #4
- Please describe the contribution of the paper
The paper proposed a spatiotemporal residual attention network to detect myocardial scar using gadolinium-free cardiac cine MR images. The network leverages both spatial and temporal information from cine images where the spatial information could capture changes in contrast and temporal information could capture wall motion abnormality.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The authors identified the gap between wall motion abnormality and scar tissue detection and proposed to leverage spatial and temporal information to better capture the scar tissue. They used a large and diverse dataset of 3000 patients, two vendors (GE and Siemens) and two field strengths (1.5T and 3T).
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The authors state that their contribution is using contrast-free cine images to detect scar. However, in their dataset, there are a large portion of LGE images (1130/3000 patients) which are gadolinium enhanced. LGE image is the gold standard for scar detection where the scar tissue has a different signal intensity compared to healthy tissue. The mixed data with gadolinium-enhanced and contrast-free images is not an effective way of evaluating the network performance. Please consider exclude the LGE images and only use cine images for network training and evaluation. The manuscript also didn’t explain the important statistics of the patient cohort such as the number of patients with scar, number of ischemic and non-ischemic patients which would impact the network performance.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The authors used a private dataset and claimed they would publish their code and model upon acceptance.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
- As commented in the main weakness section, the dataset contains gadolinium-enhanced and contrast-free images, which needs to be modified or justified.
- A minor typo in section 2.1 paragraph 1: “Y is high” should be “Y is height”?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The idea of using spatiotemporal information in contrast-free cine images is novel. But the paper also has moderate weaknesses in terms of the use of the mixed data with and without contrast and the evaluation of the network.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
Proposing a new deep learning approach, this study aims to detect myocardial scars in cardiac magnetic resonance (CMR) imaging, overcoming the limitations of relying solely on wall motion abnormalities (WMA). Using contrast-free short-axis cine images, the proposed approach leverages spatiotemporal information at different scales to detect scars in both ischemic and non-ischemic heart diseases, outperforming state-of-the-art methods. The model comprises a novel factorized 4D convolutional layer, spatiotemporal attention, and a residual attention block. The model is trained and validated on a large dataset of 3000 patients.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The manuscript is well-structured, with clear technical descriptions and novelty in the techniques proposed by the authors. The appropriate ablation and state-of-the-art comparisons are provided, contributing to the strength of the study. The paper is well-written and presented in a clear and understandable manner, enhancing its accessibility to a wider audience.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Although the paper presents a novel framework, there are some notable weaknesses. While the authors provide a training-testing-validation scheme, there is a lack of reference to data augmentation techniques or strategies to enhance the model’s generalization capabilities. Additionally, the study lacks an external cohort evaluation beyond the distribution, and there is no mention of any explainability techniques employed to ensure the transparency of the pipeline.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
reproducible
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
While the paper presents a novel framework, there are some notable weaknesses that need to be addressed. Although the authors provide a training-testing-validation scheme, there is a lack of reference to data augmentation techniques or strategies to enhance the model’s generalization capabilities. Furthermore, the study lacks an external cohort evaluation beyond the distribution, and there is no mention of any explainability techniques employed to ensure the transparency of the pipeline.
In addition, I suggest combining Figure 1 and 2 into a superfigure for a more comprehensive understanding of the pipeline. It would also be helpful to highlight the different attentions of spatial and time domain pipelines more clearly.
Additionally, the authors should include more evaluation metrics such as F1 score, Recall, Precision, and ROC curves to provide a more thorough analysis of the model’s performance.
Overall, the manuscript is well-organized, covers the technical description adequately, and provides appropriate ablation and state-of-the-art comparisons. The paper is easy to read and understand, but the aforementioned weaknesses need to be addressed to strengthen the study’s contributions.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the paper presents a novel framework, there are some notable weaknesses that need to be addressed. Although the authors provide a training-testing-validation scheme, there is a lack of reference to data augmentation techniques or strategies to enhance the model’s generalization capabilities. Furthermore, the study lacks an external cohort evaluation beyond the distribution, and there is no mention of any explainability techniques employed to ensure the transparency of the pipeline. In addition, I suggest combining Figure 1 and 2 into a superfigure for a more comprehensive understanding of the pipeline. It would also be helpful to highlight the different attentions of spatial and time domain pipelines more clearly. Additionally, the authors should include more evaluation metrics such as F1 score, Recall, Precision, and ROC curves to provide a more thorough analysis of the model’s performance.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper introduces a novel deep spatiotemporal residual attention network (ST-RAN) for myocardial scar detection in both ischemic and non-ischemic heart diseases without using gadolinium-based contrast agents in cardiac magnetic resonance (CMR) imaging. The proposed model effectively leverages temporal and spatial information at different scales and demonstrates superior performance over existing methods in detecting myocardial scars. The authors train and validate their model on a large, diverse dataset of 3000 patients, showcasing its robustness and generalisability.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
-
The paper presents a novel deep spatiotemporal residual attention network (ST-RAN) that leverages both spatial and temporal information to detect myocardial scars in ischemic and non-ischemic heart diseases. This approach addresses the limitations of previous methods that mainly focused on wall motion abnormalities in ischemic patients.
-
The proposed model comprises three primary components: a factorized 4D (3D+time) convolutional layer, spatiotemporal attention, and a residual attention block. These components work together to extract spatial and temporal features at different scales, obtaining global and local motion features and detecting subtle changes in contrast related to scar tissue. This comprehensive approach contributes to the model’s improved performance compared to CNN methods.
-
The study demonstrates the clinical feasibility of the proposed method by training and validating the model on a large dataset of 3000 patients who underwent clinical CMR with various indications and different field strengths (1.5T, 3T) from multiple vendors (GE, Siemens). This helps to showcase the generalizability and robustness of the model.
-
The paper provides a thorough evaluation, including an ablation study that assesses the contribution of each component to the overall performance of the proposed model.
-
The potential clinical impact of the proposed model is noteworthy, as it can help in screening patients with and without myocardial scars, reducing the need for gadolinium-based contrast agents, and decreasing costs and environmental pollution associated with their use.
-
The paper is very well written, with all components, methodology, experiments, and results clearly explained and presented. This makes it easy for readers to understand the novel aspects of the proposed method and its significance in the context of myocardial scar detection.
-
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
-
The pipeline only provides a binary output, determining the presence or absence of a scar, rather than quantifying the scar burden or localizing it in the AHA myocardial segments. This limitation reduces the potential benefits of the proposed method.
-
The chosen methods for comparison in the ablation study are not state-of-the-art. The inclusion of transformer-based architectures would have better supported the novelty of the proposed method. Additionally, existing methods that derive virtual LGE images with more advanced metrics should have been considered for comparison.
-
The paper lacks a detailed discussion on the translation of the proposed model into clinical practice, given its current accuracy levels. This limits the understanding of the real-world applicability and usefulness of the model in a clinical context.
-
The paper does not explore the model’s performance in detecting different types of myocardial scars, such as transmural versus non-transmural scars, or scars with different etiologies. This would be important information for understanding the model’s versatility and utility in a clinical setting.
-
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The manuscript mentions the code will be available in a repository and the checklist supports this statement.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
-
The population should be briefly described.
-
While the proposed pipeline has many merits by incorporating spatiotemporal information in the whole short-axis stack, the potential benefits are highly reduced with the fact that the pipeline only has a binary output, whether or not there is scar. It would have been more beneficial if the pipeline quantifies the scar burden and even localises it in the AHA myocardial segments, which could probably also do it. This limitation should be clearly stated from the beginning.
-
While the ablation study is correctly done, the chosen methods for comparison are not the state of the art. In the technical aspects, transformer-based architectures should have been used to support the novelty of the proposed method. In the clinical aspect, there are already methods to derive virtual LGE images, with their metrics that go beyond a binary result, which should be compared with. Otherwise, the employed methods should be considered as baseline methods, while stating the limitations of the analysis.
-
Given the current accuracy, please discuss its limited capacity when translating it into clinical practice.
-
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper presents a novel approach for myocardial scar detection using a spatiotemporal residual attention network, demonstrating improved performance over existing CNN methods. However, limitations include binary output, comparison with non-state-of-the-art methods, and limited discussion of clinical applicability. The potential benefits and innovative aspects of the proposed model warrant weak acceptance.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The reviewers are positive about the motivation, presentation, and results of the work. Congratulations!
Author Feedback
We thank all the reviewers and the meta-reviewer for their insightful and valuable comments, as well as their supportive feedback.
In response to R2 regarding the clinical benefit, we acknowledge the potential value of quantifying scars and localizing them on AHA segments. However, it is essential to consider that the state-of-the-art methods mentioned were developed using relatively small datasets of ischemic patients. Given the inclusion of a heterogeneous large dataset involving challenging non-ischemic patients, a thorough investigation of these methods is necessary. It is worth highlighting that nearly 50% of non-ischemic cardiomyopathy patients who undergo gadolinium (Gd) based imaging cardiac MRI show no myocardial scars. Thus, identifying these patients before Gd injection can lead to enhanced cost-effectiveness, scan efficiency, and safety by minimizing unnecessary Gd administration. We will address the discussion of clinical applicability accordingly.
In response to R3 regarding explainability, we acknowledge the importance of model explainability, and we recognize that a comprehensive future study should delve into this aspect in greater depth.
In response to R4 comment about mixing LGE and cine data, we want to clarify that our study exclusively utilized cine data and did not involve LGE data. The terms “LGE+” and “LGE-“ were used as medical terminology to indicate the presence or absence of a scar. However, to maintain clarity and avoid any confusion, we will replace them with “scar+” and “scar-“.