Independent Decision Path Fusion for Bimodal Asynchronous Brain-Computer Interface to Discriminate Multiclass Mental States

• Published in: IEEE access Vol. 7; pp. 165303 - 165317
• Main Authors: Jiang, Xinyu, Gu, Xiao, Xu, Ke, Ren, Haoran, Chen, Wei
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Bimodal BCIs; Decision analysis; EEG; Electroencephalography; Electrooculography; Feature extraction; fNIRS; Human-computer interface; Image classification; Infrared spectra; Machine learning; Medical imaging; Mental health; mental states; Monitoring; Near infrared radiation; Neurological diseases; Sensors; Signal processing algorithms; Task analysis
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2019.2953535

With the increasing development of brain imaging and sensing technologies, a wide variety of medical signals in multiple modalities have facilitated a better understanding of mental health. The brain-computer interface (BCI) is a technology capable of detecting mental states automatically by employing various brain sensing and machine learning methods. This contributes to efforts involving neurological disease management and restoration of cognitive function. In this study, we present an independent decision path fusion (IDPF) method by developing a bimodal asynchronous BCI based on electroencephalographs (EEGs) and functional near-infrared spectroscopy (fNIRS) to discriminate multiple mental states. The proposed IDPF method generates several independent decision paths, each of which is capable of analyzing cerebral information with respect to a specific aspect, thus interpreting the brain state from multiple points of view. Moreover, in one particular decision path for the fNIRS analysis of the IDPF method, we modified the EEG-based common spatial pattern (CSP) algorithm according to the characteristics of fNIRS. The results show that the modified common spatial pattern (MCSP) significantly outperforms CSP in fNIRS-based BCIs. Through validation on an open-access EEG-fNIRS dataset and comparison with recent studies, we found that our IDPF method achieves a high accuracy of 70.32% for a four-class classification problem (left hand motor imagery, right hand motor imagery, mental arithmetic, and resting state).