Hierarchical Transformer for Motor Imagery-Based Brain Computer Interface

• Published in: IEEE journal of biomedical and health informatics Vol. 27; no. 11; pp. 1 - 12
• Main Authors: Deny, Permana, Cheon, Saewon, Son, Hayoung, Choi, Kae Won
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.11.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Brain; Brain modeling; brain-computer interface (BCI); Classification; Classification algorithms; Computer applications; Computer vision; Convolutional neural networks; Deep learning; EEG; electroencephalogram (EEG); Electroencephalography; Feature extraction; hierarchical transformer; Human-computer interface; Imagery; Implants; Intervals; Machine learning; Mental task performance; Motor imagery (MI); Natural language processing; Task analysis; Transformers
• ISSN: 2168-2194; 2168-2208; 2168-2208
• DOI: 10.1109/JBHI.2023.3304646

In this paper, we propose a novel transformer-based classification algorithm for the brain computer interface (BCI) using a motor imagery (MI) electroencephalogram (EEG) signal. To design the MI classification algorithm, we apply an up-to-date deep learning model, the transformer, that has revolutionized the natural language processing (NLP) and successfully widened its application to many other domains such as the computer vision. Within a long MI trial spanning a few seconds, the classification algorithm should give more attention to the time periods during which the intended motor task is imagined by the subject without any artifact. To achieve this goal, we propose a hierarchical transformer architecture that consists of a high-level transformer (HLT) and a low-level transformer (LLT). We break down a long MI trial into a number of short-term intervals. The LLT extracts a feature from each short-term interval, and the HLT pays more attention to the features from more relevant short-term intervals by using the self-attention mechanism of the transformer. We have done extensive tests of the proposed scheme on four open MI datasets, and shown that the proposed hierarchical transformer excels in both the subject-dependent and subject-independent tests.