List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Florian Thamm, Oliver Taubmann, Markus Jürgens, Aleksandra Thamm, Felix Denzinger, Leonhard Rist, Hendrik Ditt, Andreas Maier
Abstract
Ischemic strokes are often caused by large vessel occlusions (LVOs), which can be visualized and diagnosed with Computed Tomography Angiography scans. As time is brain, a fast, accurate and automated diagnosis of these scans is desirable. Human readers compare the left and right hemispheres in their assessment of strokes. A large training data set is required for a standard deep learning-based model to learn this strategy from data. As labeled medical data in this field is rare, other approaches need to be developed. To both include the prior knowledge of side comparison and increase the amount of training data, we propose an augmentation method that generates artificial training samples by recombining vessel tree segmentations of the hemispheres or hemisphere subregions from different patients. The subregions cover vessels commonly affected by LVOs, namely the internal carotid artery (ICA) and middle cerebral artery (MCA). In line with the augmentation scheme, we use a 3D-DenseNet fed with task-specific input, fostering a side-by-side comparison between the hemispheres. Furthermore, we propose an extension of that architecture to process the individual hemisphere subregions. All configurations predict the presence of an LVO, its side, and the affected subregion. We show the effect of recombination as an augmentation strategy in a 5-fold cross validated ablation study. We enhanced the AUC for patient-wise classification regarding the presence of an LVO of all investigated architectures. For one variant, the proposed method improved the AUC from 0.73 without augmentation to 0.89. The best configuration detects LVOs with an AUC of 0.91, LVOs in the ICA with an AUC of 0.96, and in the MCA with 0.91 while accurately predicting the affected side.
Link to paper
DOI: https://link.springer.com/chapter/10.1007/978-3-031-16437-8_61
SharedIt: https://rdcu.be/cVRuM
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
The authors propose a new augmentation method for training deep learning models to automatically classify large vessel occlusions by combining parts of relevant images from different patients.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
- interesting idea and novel method for this application.
- good evaluation
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
- Paper is hard to follow and clarity could be improved.
- Motivation for this work can be improved
- No actual images showing the recombination of vessels
- Please rate the clarity and organization of this paper
Satisfactory
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Limited reproducibility because of the lack of clarity in the methods section
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
• Overall, the motivation for this work is not very clear. It is really not that challenging to identify if a patient has an LVO. It would be clinically more useful to locate the LVO. • The authors argue that data availability may be limited. However, the previous research papers in this domain were able to collect a vast amount of data. Thus, the claim that data availability is a problem is not well justified. • Only a subset of the patients included actually suffered of an LVO. What was the reason for imaging in the other patients? • The pre-processing section could be improved by adding more details rather than just citing other papers. • Figure 1 shows multiple discontinuities after recombination, which are not biologically plausible and may lead to false detections of clots. It is unclear what benefit these simulated datasets have. • More generally, the artery tree is highly variable between individuals and the branching pattern can be very different. It may be questioned what value such an augmentation has. • For the Recombination of ICA and MCA Subvolumes, I don’t see the information that if the ICA is affected there also should be only reduced signal (at most) for the connected MCA branches.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
While the methods is interesting, the methods description is hard to follow and could be improved
- Number of papers in your stack
4
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
Propose “recombination” (generates artificial training samples by recombining vessel tree segmentations of the hemispheres or hemisphere subregions from different patients) as a simple but effective data augmentation strategy for large vessel occlusions classification. On a private dataset, the method is showing better performance than baseline augmentation methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
• An interesting way of augmenting the brain data • Comprehensive experiments: five fold cross validation on three models, each with ablation study
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
• The augmentation method might only be useful to limited applications, thus the impact is limited to a small field. • No public datasets. No access to codes.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No code or data available from the paper.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
The paper is a natural extension of [14], which introduced deformation and mirroring as augmentation methods to help large vessel occlusions classification from the vessel segmentation masks. With the new recombination augmentation method, the healthy parts of the brain vessels from all the training set are shared and thus improve the classification performance. This idea is especially useful when the dataset is limited and the task is relatively simple. The DenseNet used in three models was not the same (number of parameters), which introduces another variable in comparison between three models. Considering the dataset was from the same source as ref [14] (151 patients in this paper, 168 in ref [14]), it is important to make clear whether this is using the same or partial data (why cherry picking if that is the case) as ref [14]. It would be ideal if this paper follows the same dataset as previous ones, if available. In ref [14] segmentation masks were deformed 20 times as deformation augmentation (10 times with 4 random anchors + 10 times with 5 random anchors), but the authors implemented with “Each data set is deformed 10 times with a random elastic field”. Is that an implementation difference? All the cases were successfully segmented and registered? How the performance is impacted by segmentation and registration quality? The basilar arteries might be tortuous and in both sides of the brain mid-plane. Mirroring all left-sided hemisphere’s vessel trees in sagittal direction might cause broken/irregular basilar arteries in the artificial images. Grammar errors: it can supplemented system enables to split
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well written (despite some grammar errors) but the innovation might not be high. Also the specific application on large vessel occlusion classification limits its reader group/impact.
- Number of papers in your stack
1
- What is the ranking of this paper in your review stack?
4
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
The authors present an augmentation method for large vessel occlusion (LVO) classification that recombines subvolumes of vessel segmentations from different patients.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The presented method is a simple yet effective and clinically relevant solution to solve the problem. The ROC AUC is significantly improved with respect to the compared method. The paper is very well written and structured.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
The motivation for developing an automatic classification method is not clear. Section 1 reads “Despite common anatomical patterns, the individual configuration and appearance of the vessel tree can differ substantially between patients, hence automated and accurate methods for LVO detection are desirable”. I don’t see why differences between vessels trees across patients justifies a desire for automated methods. Can a radiologist perform this task quickly and accurately? If yes, why do we need an automated method?
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
No code, data or models are shared. The reasons for this are not mentioned. Most hyperparameters are shared, but the strategy to choose them is not described.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2022/en/REVIEWER-GUIDELINES.html
Please check for typos and incongruences: “as time is brain”, “it can supplemented”, “models specifically designed exploiting”… Please check for missing or undefined acronyms (I recommend the \acronym LaTeX package).
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is well presented and the proposed method is clear and effective. Although the motivation is not properly explained, it might be guessed. For example, the method could be used for triage. I would have liked to see more significant efforts in terms of reproducibility.
- Number of papers in your stack
6
- What is the ranking of this paper in your review stack?
2
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
The authors present a novel augmentation method for training deep learning models to automatically classify large vessel occlusions by combining parts of relevant images from different patients. The method is very interesting and experiments are comprehensive.
- What is the ranking of this paper in your stack? Use a number between 1 (best paper in your stack) and n (worst paper in your stack of n papers). If this paper is among the bottom 30% of your stack, feel free to use NR (not ranked).
NR
Author Feedback
We want to thank the reviewers and the meta reviewer for the feedback and appreciate the valuable comments about our manuscript. We will revise the manuscript according to the improvements suggested by the reviewers.
Concerns about the motivation were raised by R1 and R2, whether demands exist for automated solutions in the detection of LVOs. Like in many other medical fields, there is a need from clinician’s perspective to automate diagnoses which can be automated, so valuable time can be invested into more difficult cases. In our related works section we have cited two different commercially available products, which indicates a need in this market.
Another comment was made by R1 and R2 about possible discontinuations close to the sagittal mid-plane. However, the affected vessels are not of interest in the work at hand, as we designed the LVO detection for the ICA and MCA branches. We fully agree with R2, that the proposed method, in its current design, is not suitable for LVOs located in the basilar artery.