Cerebral and gaze data fusion for wheelchair navigation enhancement: case of distracted users

• Published in: Robotica Vol. 37; no. 2; pp. 246 - 263
• Main Authors: Lamti, Hachem A., Ben Khelifa, Mohamed Moncef, Hugel, Vincent
• Format: Journal Article
• Language: English
• Published: Cambridge, UK Cambridge University Press 01.02.2019
• Subjects: Automatic; Blinking; Brain; Data integration; Engineering Sciences; Hybrid systems; Multilayers; Multisensor fusion; Navigation; Neural networks; Obstacle avoidance; Shortages; Wheelchairs; Navigation; Gaze; EEG; Behavioral entropy; Wheelchair; Distraction; BCI; SSVEP
• ISSN: 0263-5747; 1469-8668
• DOI: 10.1017/S0263574718000991

The goal of this paper is to present a new hybrid system based on the fusion of gaze data and Steady State Visual Evoked Potentials (SSVEP) not only to command a powered wheelchair, but also to account for users distraction levels (concentrated or distracted). For this purpose, a multi-layer perception neural network was set up in order to combine relevant gazing and blinking features from gaze sequence and brainwave features from occipital and parietal brain regions. The motivation behind this work is the shortages raised from the individual use of gaze-based and SSVEP-based wheelchair command techniques. The proposed framework is based on three main modules: a gaze module to select command and activate the flashing stimuli. An SSVEP module to validate the selected command. In parallel, a distraction level module estimates the intention of the user by mean of behavioral entropy and validates/inhibits the command accordingly. An experimental protocol was set up and the prototype was tested on five paraplegic subjects and compared with standard SSVEP and gaze-based systems. The results showed that the new framework performed better than conventional gaze-based and SSVEP-based systems. Navigation performance was assessed based on navigation time and obstacles collisions.