Decoding spatial attention by using cortical currents estimated from electroencephalography with near-infrared spectroscopy prior information

• Published in: NeuroImage (Orlando, Fla.) Vol. 90; pp. 128 - 139
• Main Authors: Morioka, Hiroshi, Kanemura, Atsunori, Morimoto, Satoshi, Yoshioka, Taku, Oba, Shigeyuki, Kawanabe, Motoaki, Ishii, Shin
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier Inc 15.04.2014; Elsevier; Elsevier Limited
• Subjects: Adult; Attention - physiology; Bayes Theorem; Biological and medical sciences; Brain; Brain - physiology; Brain Mapping - methods; Brain-Computer Interfaces; Brain–machine interface (BMI); Electroencephalography; Electroencephalography (EEG); Electroencephalography Phase Synchronization; Estimates; Fundamental and applied biological sciences. Psychology; Humans; Male; Near-infrared spectroscopy (NIRS); Neurosciences; NIRS–EEG simultaneous measurement; Probability distribution; Signal Processing, Computer-Assisted; Space Perception - physiology; Spatial attention; Spectroscopy, Near-Infrared; Variational Bayesian Multimodal EncephaloGraphy (VBMEG); Vertebrates: nervous system and sense organs; Young Adult; Near-infrared spectroscopy (NIRS); Brain–machine interface (BMI); Electroencephalography (EEG); NIRS–EEG simultaneous measurement; Spatial attention; Variational Bayesian Multimodal EncephaloGraphy (VBMEG); EncephaloGraphy (VBMEG); NIRS-EEG simultaneous measurement; Central nervous system; Electrophysiology; Electroencephalography; Variational Bayesian Multimodal; Brain-machine interface (BMI); Encephalon
• ISSN: 1053-8119; 1095-9572
• DOI: 10.1016/j.neuroimage.2013.12.035

For practical brain–machine interfaces (BMIs), electroencephalography (EEG) and near-infrared spectroscopy (NIRS) are the only current methods that are non-invasive and available in non-laboratory environments. However, the use of EEG and NIRS involves certain inherent problems. EEG signals are generally a mixture of neural activity from broad areas, some of which may not be related to the task targeted by BMI, hence impairing BMI performance. NIRS has an inherent time delay as it measures blood flow, which therefore detracts from practical real-time BMI utility. To try to improve real environment EEG–NIRS-based BMIs, we propose here a novel methodology in which the subjects' mental states are decoded from cortical currents estimated from EEG, with the help of information from NIRS. Using a Variational Bayesian Multimodal EncephaloGraphy (VBMEG) methodology, we incorporated a novel form of NIRS-based prior to capture event related desynchronization from isolated current sources on the cortical surface. Then, we applied a Bayesian logistic regression technique to decode subjects' mental states from further sparsified current sources. Applying our methodology to a spatial attention task, we found our EEG–NIRS-based decoder exhibited significant performance improvement over decoding methods based on EEG sensor signals alone. The advancement of our methodology, decoding from current sources sparsely isolated on the cortex, was also supported by neuroscientific considerations; intraparietal sulcus, a region known to be involved in spatial attention, was a key responsible region in our task. These results suggest that our methodology is not only a practical option for EEG–NIRS-based BMI applications, but also a potential tool to investigate brain activity in non-laboratory and naturalistic environments. •Portable multi-modal imaging (NIRS and EEG) was used for decoding spatial attention.•Cortical currents estimated from EEG–NIRS explained the validity of classifiers.•The accuracy was best available among non-invasive portable devices.•This research has potential to study brain activities in naturalistic conditions.