List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Deeksha M. Shama, Jiasen Jing, Archana Venkataraman
Abstract
We propose a robust deep learning framework to simultane-
ously detect and localize seizure activity from multichannel scalp EEG.
Our model, called DeepSOZ, consists of a transformer encoder to gen-
erate global and channel-wise encodings. The global branch is combined
with an LSTM for temporal seizure detection. In parallel, we employ
attention-weighted multi-instance pooling of channel-wise encodings to
predict the seizure onset zone. DeepSOZ is trained in a supervised fash-
ion and generates high-resolution predictions on the order of each sec-
ond (temporal) and EEG channel (spatial). We validate DeepSOZ via
bootstrapped nested cross-validation on a large dataset of 120 patients
curated from the Temple University Hospital corpus. As compared to
baseline approaches, DeepSOZ provides robust overall performance in
our multi-task learning setup. We also evaluate the intra-seizure and
intra-patient consistency of DeepSOZ as a first step to establishing its
trustworthiness for integration into the clinical workflow for epilepsy.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43993-3_18
SharedIt: https://rdcu.be/dnwNj
Link to the code repository
https://github.com/deeksha-ms/DeepSOZ.git
Link to the dataset(s)
https://isip.piconepress.com/projects/tuh_eeg/html/downloads.shtml
Reviews
Review #1
- Please describe the contribution of the paper
Paper presents a novel seizure onset detection and localisation method based on transformers and LSTM. Methods are compared against published benchmarks and shown to be better.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Novel method. Implemented on a public dataset so it can be benchmarked against other methods.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Not clear how much data per patient there is. This is important for having any notion of clinical utility with respect to false positives.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Most detail is provided. No links to data or code, but data is publicly available and can be found via google search
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Whats the unifocal vs multifoci onset accuracy?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Paper is noverl and method can be benchmarked. The dataset is public, which enables benchmarking. It would be helpful though to have larger datasets as well, along with long-term continuous recordings.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #3
- Please describe the contribution of the paper
This paper proposed a robust deep learning framework to simultaneously detect and localize seizure activity from multichannel scalp EEG, which outperformed other SOTA methods.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
This paper proposed a robust deep learning framework to simultaneously detect and localize seizure activity from multichannel scalp EEG, which outperformed other SOTA methods. The paper also conducted the experiments in multi-level, which is convincing. The figures are clear and the paper is nicely written. The methods are clear and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
No major weakness.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It would be easy to reproduce if they make their code public.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
This work used a novel attention-weighted multi-instance pooling to supervise seizure-level SOZ localization at the single channel resolution. However, in ablation study, when replacing this with maxpool, it seems to have better performance in temporal seizure detection. Can you elaborate more about this?
At the end of experiment results, the author said, “As seen, DeepSOZ accurately detects the seizure interval in all cases but has two false positive detections for Patient 1.”, but if I understand the figure correctly, the subplot3 in Patient 2 is false negative?
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
This paper proposed a robust deep learning framework to simultaneously detect and localize seizure activity from multichannel scalp EEG, which outperformed other SOTA methods. Overall the paper is clear, easy to follow and convincing.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #2
- Please describe the contribution of the paper
This paper is showing a new way of seizure onset detection as well as spatial localization (sensor level), using Transformers. The manuscript is well organized but could have been written better.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The application of transformers has been growing rapidly during the past few years in different applications next to the convolutional neural networks. In this paper, the transformer model was used for the task or temporal and spatial detection. The authors provided fair comparison with respect to some other SoA models for the same tasks.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
There are bits and pieces are missing. When the authors mention an example of a technique being used, it would be better to mention a pioneering literature. For example in page 2, the author mentioned that CNNs being used in EEG and then provided reference 4 as the only example which came several years after other references used CNNs for EEG.
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
There is no implementation of the code provided or at least mentioned that the authors would do it in the future. Although it is not necessary but it could help the reviewers to verify the results.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Providing the spatial localization next to temporal detection of the seizure using transformers is an interesting topic. The authors only covered the sensor level spatial localization. It would be helpful to have the source level (3D space) localization results as well.
In the Table 1, and 2 how did the authors compare their work with other methods? did they implement the other methods for comparison or just reported their results from other papers?
The equations are not consistent in terms of use of italic characters.
In figure 1, where is the x^~ or Z^t?
In equation 5, where is L1 which is mentioned in the following line?
It would be better to add some more references for such a well studies topic and given the fact that the authors still have space for it.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
5
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
It is hard to judge the reported results. Also small mistakes are here and there.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
This manuscript present a novel deep learning method to aid in the detection and localization of seizure onset using a combination of transformer and LSTM architectures. They also present a novel attention pooling mechanism that vastly improves the ability of the model to localize the seizure. R1 and R2 have some minor suggestions to improve the clarity of the paper including defining some clinical terms (unifocal and multifocal), describing how comparisons with SOTA methods was performed, and some minor corrections to ensure equations and notation are consistent.
Author Feedback
We thank reviewers for their appreciation of our work and for noting the difficulty of jointly detecting and localizing the seizure onset. We will fix the variable typos, clarify our equations, and add more citations on deep learning for seizure detection to our final paper. Below, we address the remaining concerns in a point-by-point fashion:
PATIENT CHARACTERISTICS (Rev 1): We include all patients with a unifocal seizure onset that has been localized by clinicians to a lobe or quadrant of the brain. The number of seizures per patient is 12.5 on average with at least one seizure recorded in the EEG for each patient. The average seizure duration is approximately 90 seconds. Our EEG dataset consists of 10 minute recordings that contain a single seizure randomly positioned within this interval. We shall include comprehensive details of the patient and seizure characteristics in our final version.
SOURCE LOCALIZATION (Rev 2): While we agree that source localization at the voxel level has been explored for epilepsy, it is beyond the scope of this study. The reason is that our EEG dataset is acquired using the standard 10-20 montage, which contains only 19 channels. This sensor resolution is insufficient being highly prone to error for fine-grained source localization [1]
[1] Lu, Yunfeng, et al. “Seizure source imaging by means of FINE spatio-temporal dipole localization and directed transfer function in partial epilepsy patients.” Clinical Neurophysiology 123.7 (2012): 1275-1283.
BASELINE MODEL IMPLEMENTATION (Rev 2): We trained DeepSOZ and the baseline models from scratch in each cross validation fold to compute the metrics in Table 1 and Table 2. Where possible, we have fixed the baseline model hyperparameters according to the original reference. We believe that this strategy provides a fair comparison across models.
PERFORMANCE OF DEEPSOZ-MAX (Rev 3): DeepSOZ and DeepSOZ-max have the same architecture for temporal seizure detection. Accordingly, while DeepSOZ-max achieves a slightly higher performance in Table 1, it is not statistically significant. On the other hand, DeepSOZ outperforms DeepSOZ-max by a large margin in the seizure localization task. Taken together, we believe DeepSOZ to be the superior model for joint detection and localization.
FIGURE 3 CLARIFICATION (Rev 3): There are two patients reported in Figure 3. For Patient 1, DeepSOZ correctly detects the onset in the left frontal area. it also strongly detects left fronto-temporal and right temporal regions to be positive, which are the “two false positives” we are noting in the text. For Patient 2, we indeed have some false positive and false negative detections, which we note in the text as a tricky seizure onset presentation.
CODE AVAILABILITY (All Revs): To facilitate reproducibility, we will create a GitHub repository that contains our preprocessing scripts, model architectures, trained models, and codes.