Mixed reality-based brain computer interface system using an adaptive bandpass filter: Application to remote control of mobile manipulator

• Published in: Biomedical signal processing and control Vol. 83; p. 104646
• Main Authors: Li, Qi, Sun, Meiqi, Song, Yu, Zhao, Di, Zhang, Tingjia, Zhang, Zhilin, Wu, Jinglong
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.05.2023
• Subjects: Adaptive filtering; Brain-computer interface (BCI); Electroencephalogram (EEG); Event-related potential (ERP); Manipulator grasping; Mixed Reality (MR); Adaptive filtering; Manipulator grasping; Brain-computer interface (BCI); Electroencephalogram (EEG); Mixed Reality (MR); Event-related potential (ERP)
• ISSN: 1746-8094
• DOI: 10.1016/j.bspc.2023.104646

•An adaptive filtering method more suitable for practical systems, which can improve the generalization ability of the system and achieve good control results;•Improvements to the MR-based BCI system by embedding a live view in the center of visual stimulation paradigm to avoid the output of erroneous commands when the object is out of the line of sight in remote control;•Development of a portable brain-computer interface system using a small size EEG acquisition device and wearable MR glasses. Brain-computer interface (BCI) systems based on mixed reality (MR) have promising applications in assisting people with disabilities to control manipulators. Using MR glasses instead of a computer screen to display visual stimulator can effectively avoid frequent switching of attention between the visual stimulator and the manipulator. When the manipulator moves out of the sight of the subject, the subject may not be able to control it accurately. Our system uses Microsoft Hololens2 as the display device to synchronize the command matrix with a live view of the mobile manipulator's position, thus tracking the position in real-time. Another problem in previous studies is that they have good accuracy in trained subjects, however, the accuracy drops dramatically when faced with untrained subjects, suggesting poor generalization capabilities. In our study, an adaptive filtering method combined with convolutional neural networks (CNN) is proposed, which has few learning parameters and fast convergence, and can improve the generalization ability of the system in the face of untrained subjects. When faced with untrained subjects, the average accuracy of our method was 93.04%, and the average ITR was 20.96 bits/min. All subjects can successfully complete the grasping task without colliding with obstacles. The results show that the BCI system developed in this study has strong practicability and high research significance.