A Novel EOG/EEG Hybrid Human-Machine Interface Adopting Eye Movements and ERPs: Application to Robot Control

• Published in: IEEE transactions on biomedical engineering Vol. 62; no. 3; pp. 876 - 889
• Main Authors: Ma, Jiaxin, Zhang, Yu, Cichocki, Andrzej, Matsuno, Fumitoshi
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2015; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Adult; Algorithms; Brain modeling; Electrodes; Electroencephalogram (EEG); Electroencephalography; Electroencephalography - instrumentation; Electroencephalography - methods; Electrooculogram (EOG); Electrooculography; Electrooculography - instrumentation; Electrooculography - methods; event-related potential (ERP); Evoked Potentials - physiology; Eye movements; Eye Movements - physiology; Face; Female; human-machine interface (HMI); Humans; Male; Man machine interfaces; Man machine systems; Recognition; Robot control; Robotics - instrumentation; Robots; Software; Task Performance and Analysis; Tasks; Young Adult; electrooculogram (EOG); event-related potential (ERP); Electroencephalogram (EEG); robot control; human–machine interface (HMI)
• ISSN: 0018-9294; 1558-2531; 1558-2531
• DOI: 10.1109/TBME.2014.2369483

This study presents a novel human-machine interface (HMI) based on both electrooculography (EOG) and electroencephalography (EEG). This hybrid interface works in two modes: an EOG mode recognizes eye movements such as blinks, and an EEG mode detects event related potentials (ERPs) like P300. While both eye movements and ERPs have been separately used for implementing assistive interfaces, which help patients with motor disabilities in performing daily tasks, the proposed hybrid interface integrates them together. In this way, both the eye movements and ERPs complement each other. Therefore, it can provide a better efficiency and a wider scope of application. In this study, we design a threshold algorithm that can recognize four kinds of eye movements including blink, wink, gaze, and frown. In addition, an oddball paradigm with stimuli of inverted faces is used to evoke multiple ERP components including P300, N170, and VPP. To verify the effectiveness of the proposed system, two different online experiments are carried out. One is to control a multifunctional humanoid robot, and the other is to control four mobile robots. In both experiments, the subjects can complete tasks effectively by using the proposed interface, whereas the best completion time is relatively short and very close to the one operated by hand.