MAE-EEG-Transformer: A transformer-based approach combining masked autoencoder and cross-individual data augmentation pre-training for EEG classification

• Published in: Biomedical signal processing and control Vol. 94; p. 106131
• Main Authors: Cai, Miao, Zeng, Yu
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.08.2024
• Subjects: EEG; Masked autoencoder; Motor imagery; Pre-training; Transformer; Pre-training; Motor imagery; Transformer; Masked autoencoder; EEG
• ISSN: 1746-8094
• DOI: 10.1016/j.bspc.2024.106131

Convolutional neural networks (CNN) may not be ideal for extracting global temporal features from non-stationary Electroencephalogram (EEG) signals. The application of the masking-based method in EEG classification is not well studied, and there is a shortage of commonly accepted models for verifying inter-individual results in motor imagery classification tasks. The MAE-EEG-Transformer, a transformer with masking mechanism, is proposed in this article. It pre-trains by randomly masking signals and forces the model to learn semantic features. The pre-trained encoder module is fine-tuned and moved to the classification task to obtain the category of EEG signals. The effectiveness of features with and without pre-training is compared using t-SNE visualization to demonstrate pre-training’s inter-subject efficacy. The MAE EEG Transformer was extensively evaluated across three prevalent datasets in EEG-based motor imagery, demonstrating performance comparable to state-of-the-art models while requiring only approximately 20% of the computational cost (results in Table 1, 2, 3 and 4). •Extracting global temporal geatures from non-stationary EEG signals.•Random masking and reconstruction forces the model to learn semantic features.•Cross-Subject data augmentation pre-training.•Achieving similar performance to SOTA models with 20% of the computational cost.