List of Papers By topics Author List
| Paper Info | Reviews | Meta-review | Author Feedback | Post-Rebuttal Meta-reviews |
Authors
Tianyang Zhong, Xiaozheng Wei, Enze Shi, Jiaxing Gao, Chong Ma, Yaonai Wei, Songyao Zhang, Lei Guo, Junwei Han, Tianming Liu, Tuo Zhang
Abstract
Motor imagery (MI) electroencephalogram (EEG) decoding, as a core component widely used in noninvasive brain-computer interface (BCI) system, is critical to realize the interaction purpose of physical world and brain activity. However, the conventional methods are challenging to obtain desirable results for two main reasons: there is a small amount of labeled data making it difficult to fully exploit the features of EEG signals, and lack of unified expert knowledge among different individuals. To handle these dilemmas, a novel small-sample EEG decoding method based on abductive learning (SSE-ABL) is proposed in this paper, which integrates perceiving module that can extract multiscale features of multi-channel EEG in semantic level and knowledge base module of brain science. The former module is trained via pseudo-labels of unlabeled EEG signals generated by abductive learning, and the latter is refined via the label distribution predicted by semi-supervised learning. Experimental results demonstrate that SSE-ABL has a superior performance compared with state-of-the-art methods and is also convenient for visualizing the underlying information flow of EEG decoding.
Link to paper
DOI: https://doi.org/10.1007/978-3-031-43907-0_40
SharedIt: https://rdcu.be/dnwcR
Link to the code repository
N/A
Link to the dataset(s)
N/A
Reviews
Review #1
- Please describe the contribution of the paper
A novel small-sample EE-G decoding method based on abductive learning (SSE-ABL) is proposed, which integrates perceiving module that can extract multiscale features of multi-channel EEG in semantic level and knowledge base module of brain science.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The method seems novel and interesting. The paper is well organised and easy to follow.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Some key details of the experiment may be missing:
- For the baseline models, are they also trained with unlabeled data? If the answer is ‘yes’, what method is used for the semi-supervised training? Please make clear. If the answer is ‘no’, it is not fair for the baselines.
- Ablation studies may be needed to further explore the effect of every module.
- Please rate the clarity and organization of this paper
Very Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
Good. The authors will release the code.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
The meaning of the abbreviation MIR is not explained.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
4
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
Some key details of the experiment may be missing:
- For the baseline models, are they also trained with unlabeled data? If the answer is ‘yes’, what method is used for the semi-supervised training? Please make clear. If the answer is ‘no’, it is not fair for the baselines.
- Ablation studies may be needed to further explore the effect of every module.
- Reviewer confidence
Confident but not absolutely certain
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
6
- [Post rebuttal] Please justify your decision
The authors addressed all my concern in the response. Therefore I have raised my score to ‘accept’.
Review #3
- Please describe the contribution of the paper
The paper proposed a novel small-sample EEG decoding method that can extract multiscale features for MI recognition. The model outperforms other SOTA method with a small amount of labeled EEG data.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
Their approach solves the problem of low precision and poor robustness ability of EEG decoding under a small amount of labeled data. The method is very clearly written. The experiments are convincing.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
No major weakness. Several minor issues are mentioned in the detailed comments below.
- Please rate the clarity and organization of this paper
Excellent
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
It would be easy to reproduce if they make their code public.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
It’s a little bit hard to follow the step 6 in Algorithm 2, what does the X here stand for? EEG data? There is a problem in reference 16. There are some minor grammar problems in the paper.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
7
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The paper is nicely written. The method is clear, and the experiments are comprehensive with convincing results.
- Reviewer confidence
Very confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Review #5
- Please describe the contribution of the paper
The main contributions of the paper are as follows: Novel EEG decoding method for the Motor Imagery Recognition (MIR) problem with small-sample EEG signals. This method ensures accuracy and robustness without relying on strict mathematical assumptions or being affected by strong interference. Design of a multi-scale feature fusion network that enhances abstract features in EEG signals, capturing temporal and frequency information across multiple channels and spatial relationships among electrodes. Construction of an effective knowledge base module for motor imagery, symbolising and using large-scale un-labeled EEG signals to upgrade the model space and extract valuable information.
- Please list the main strengths of the paper; you should write about a novel formulation, an original way to use data, demonstration of clinical feasibility, a novel application, a particularly strong evaluation, or anything else that is a strong aspect of this work. Please provide details, for instance, if a method is novel, explain what aspect is novel and why this is interesting.
The use of Abductive Reasoning Optimization is interesting. This method solves the problem of low precision and poor robustness ability of EEG decoding under a small amount of labeled data. In situations where the classifier is inadequately trained, the pseudo-labels generated may be incorrect. In order to ensure consistent abductions, the SSE-ABL method proposed in this paper employs a gradient-free optimisation method to correct these erroneous pseudo-labels, adhering to the principle of minimal inconsistency. The pseudo-codes are properly described.
- Please list the main weaknesses of the paper. Please provide details, for instance, if you think a method is not novel, explain why and provide a reference to prior work.
Clarify whether the baseline models were trained with un-labeled data and specify the method used for the semi-supervised training. Ablation studies should be conducted. To what extent is this method resilient and enduring when it comes to the alteration or update of the knowledge base?
- Please rate the clarity and organization of this paper
Good
- Please comment on the reproducibility of the paper. Note, that authors have filled out a reproducibility checklist upon submission. Please be aware that authors are not required to meet all criteria on the checklist - for instance, providing code and data is a plus, but not a requirement for acceptance
The code could have been already released at the submission stage.
- Please provide detailed and constructive comments for the authors. Please also refer to our Reviewer’s guide on what makes a good review: https://conferences.miccai.org/2023/en/REVIEWER-GUIDELINES.html
Even if they could be obvious to expert readers, all acronyms in the document should be explained at the first use, in the paper. Clarify how the baseline models were trained. Ablation studies could conducted. Examine the method’s robustness and sustainability in relation to the modification or update of the knowledge base.
- Rate the paper on a scale of 1-8, 8 being the strongest (8-5: accept; 4-1: reject). Spreading the score helps create a distribution for decision-making
6
- Please justify your recommendation. What were the major factors that led you to your overall score for this paper?
The originality of the approach and its potential application in clinical settings.
- Reviewer confidence
Somewhat confident
- [Post rebuttal] After reading the author’s rebuttal, state your overall opinion of the paper if it has been changed
N/A
- [Post rebuttal] Please justify your decision
N/A
Primary Meta-Review
- Please provide your assessment of this work, taking into account all reviews. Summarize the key strengths and weaknesses of the paper and justify your recommendation. In case you deviate from the reviewers’ recommendations, explain in detail the reasons why. In case of an invitation for rebuttal, clarify which points are important to address in the rebuttal.
Summary of the Key Strengths and Weaknesses of the Paper:
Key Strengths: • The paper presents a novel and interesting method that addresses the issues of low precision and poor robustness in EEG decoding with limited labeled data. • The organization and clarity of the paper make it easy to follow. • The method is well explained and the experiments provide convincing results.
Areas of weakness in the paper that require further attention: • There is a need for clarification on whether the baseline models were trained with un-labeled data and the specific method used for semi-supervised training. • Ablation studies should be conducted to further investigate the effect of each module. • The paper should discuss the robustness and sustainability of the method in relation to changes or updates in the knowledge dataset.
To assist the authors in enhancing this study, we offer the following recommendations: • Algorithm 2, specifically step 6, needs further clarification on the meaning of “X.” Is it referring to EEG data? • There is an issue with reference 16 that needs to be addressed. • Minor grammar problems should be corrected throughout the paper. • Ensure that all acronyms are explained at their first use, even if they may be obvious to expert readers.
Key points the authors should focus on in their rebuttal responses: • Provide clarification on how the baseline models were trained. • Consider conducting ablation studies to explore the individual module’s impact. • Examine and discuss the method’s robustness and sustainability in relation to modifications or updates in the knowledge base. • We strongly encourage the release of the code associated with this work.
Author Feedback
We appreciate the meta reviewer and reviewers for insightful comments. We first response to common concerns and then to individual reviewer.
Q1(AC&R1&R5): Provide clarification on how the baseline models were trained. The baseline models are trained with unlabeled data by adopting self-teaching algorithm, which is widely used by the mainstream methods in semi-supervised learning. Specifically, the initial baseline models are first trained with a small amount of labeled EEG data, and then the trained baseline models are used to predict the unlabeled data. Next, the unlabeled samples with high confidence are added to the original training data together with the predicted class labels. The baseline models are retrained with the new training data, and the above steps are executed iteratively.
Q2(AC&R1&R5): Further exploration of the effect of every module via ablation studies, and of robustness and sustainability in relation to the modification of the knowledge base. In fact, some preliminary ablation experiments have been performed but are not included in the current submission due to the limitation of space. These experiments include the effects of different perception modules (C1) and the modifications of knowledge base (C2), as shown below using 10% training dataset. (-:placeholder) C1: Method &Sub1&Sub2&Sub3&Sub4&Sub5&Sub6&Sub7&Sub8&Sub9&AVE TF—- &79.64&73.29&77.13&75.68&73.68&77.43&88.05&86.10&79.73&78.97 BERT &78.01&69.52&75.23&76.86&70.86&77.13&88.10&85.19&77.76&77.63 MEET &79.98&70.86&76.03&77.19&71.94&77.06&87.53&85.86&78.10&78.28 ViT—- &79.22&76.01&78.90&78.43&72.44&78.66&89.19&88.05&79.40&80.03
C2: Method &Sub1&Sub2&Sub3&Sub4&Sub5&Sub6&Sub7&Sub8&Sub9&AVE 4 rules &79.22&76.01&78.90&78.43&72.44&78.66&89.19&88.05&79.40&80.03 3 rules &76.37&75.97&77.65&77.29&72.16&76.50&88.27&87.64&78.65&78.94 2 rules &70.59&70.16&71.11&72.52&68.92&72.96&84.73&83.98&73.96&74.33 1 rule–&58.99&63.31&65.73&63.86&59.36&69.11&73.52&70.05&61.32&65.03 0 rules Not be optimized
On these results, it is found that:
- Different perception modules have certain fluctuations on the performance of the proposed method, but the overall results remain satisfactory with relatively stable direction. The main reason is that the reasoning module can correct the incorrect judgment results from different perception modules, so that they can be greatly improved at the data level.
- When the number of rules contained in the knowledge base gradually decreases from four rules to one, the search space of the proposed model would grow from O(2^n) to O(4^n) exponentially and the performance would deteriorate. Especially, if there are no rules, the model can’t be executed. The main reason is that the solution space can’t be found in huge search space, and each instance is classified as invalid. Fortunately, we can be surprised to find that the model can solve the most conducive solution under limited rules (only two rules).
Q3(AC&R1&R5): Acronyms explanation. MIR: motion intention recognition; VarEpoch: epochs of training model; Q, K, V: query, key, value; SA: self-attention block; X: EEG data.
Q4(AC&R3): Following the step 6 in Algorithm 2, what does the X here stand for? The X stands for EEG data. The meaning is that the information concerned by Q, K, V in Transformer model and the original EEG information are used to extract important information to reduce the feature redundancy by linear space transformation.
Q5(AC): Releasing the code. The code will be released once the paper is accepted.
Q6(AC&R3): Format of reference 16. The correct format is ‘16. Song Y, Jia X, Yang L, et al …’.
Q7(AC&R3): Minor grammar problems.
- ‘as the lack of large-scale labeled data, making the identification results more vulnerable to be affected–> the lack of … data makes the identification …’.
- ‘there is still an unsolved urgent challenge need to be considered further–> … challenge that needs …’.
Thank you again for your valuable comments.
Post-rebuttal Meta-Reviews
Meta-review # 1 (Primary)
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
The responses were convincing. Pls. consider all the remarks, exchanges as your own commitments to make it a successful communication.
Meta-review #2
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
This paper studied the EEG decoding problem and proposed an abductive learning-based method that extracts multiscale EEG features with a pseudo-label trained semantic module and a semi-supervised learning-based knowledge module. The paper reported better results than baseline methods. The rebuttal generally addresses the main concerns.
Meta-review #3
- Please provide your assessment of the paper taking all information into account, including rebuttal. Highlight the key strengths and weaknesses of the paper, clarify how you reconciled contrasting review comments and scores, indicate if concerns were successfully addressed in the rebuttal, and provide a clear justification of your decision. If you disagree with some of the (meta)reviewer statements, you can indicate so in your meta-review. Please make sure that the authors, program chairs, and the public can understand the reason for your decision.
After carefully evaluating the authors’ feedback and the final decisions of the reviewers, it is evident that an unanimous majority has emerged in favor of accepting the paper. Notably, even the reviewer who initially recommended rejection has changed their score to acceptance, acknowledging the value and significance of the paper.
All the reviewers have recognized the impact of the findings presented in the paper, underscoring its importance in the field. The Meta Reviewer concurs with the majority consensus and agrees that the paper should be accepted for publication.