Multi-Branch Mutual-Distillation Transformer for EEG-Based Seizure Subtype Classification

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 32; pp. 831 - 839
• Main Authors: Peng, Ruimin, Du, Zhenbang, Zhao, Changming, Luo, Jingwei, Liu, Wenzhong, Chen, Xinxing, Wu, Dongrui
• Format: Journal Article
• Language: English
• Published: United States IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Brain modeling; Classification; Coders; Deep learning; Distillation; EEG; Electric Power Supplies; Electroencephalography; Epilepsy; Feature extraction; Frequencies; Humans; knowledge distillation; Knowledge management; Machine Learning; seizure subtype classification; Seizures; Seizures - diagnosis; Training; Transformer; Transformers; Wavelet transforms
• ISSN: 1534-4320; 1558-0210; 1558-0210
• DOI: 10.1109/TNSRE.2024.3365713

Cross-subject electroencephalogram (EEG) based seizure subtype classification is very important in precise epilepsy diagnostics. Deep learning is a promising solution, due to its ability to automatically extract latent patterns. However, it usually requires a large amount of training data, which may not always be available in clinical practice. This paper proposes Multi-Branch Mutual-Distillation (MBMD) Transformer for cross-subject EEG-based seizure subtype classification, which can be effectively trained from small labeled data. MBMD Transformer replaces all even-numbered encoder blocks of the vanilla Vision Transformer by our designed multi-branch encoder blocks. A mutual-distillation strategy is proposed to transfer knowledge between the raw EEG data and its wavelets of different frequency bands. Experiments on two public EEG datasets demonstrated that our proposed MBMD Transformer outperformed several traditional machine learning and state-of-the-art deep learning approaches. To our knowledge, this is the first work on knowledge distillation for EEG-based seizure subtype classification.