Plug-and-play supervisory control using muscle and brain signals for real-time gesture and error detection

Effective human supervision of robots can be key for ensuring correct robot operation in a variety of potentially safety-critical scenarios. This paper takes a step towards fast and reliable human intervention in supervisory control tasks by combining two streams of human biosignals: muscle and brai...

Full description

Saved in:
Bibliographic Details
Published inAutonomous robots Vol. 44; no. 7; pp. 1303 - 1322
Main Authors DelPreto, Joseph, Salazar-Gomez, Andres F., Gil, Stephanie, Hasani, Ramin, Guenther, Frank H., Rus, Daniela
Format Journal Article
LanguageEnglish
Published New York Springer US 01.09.2020
Springer Nature B.V
Subjects
Artificial Intelligence
Brain
Computer Imaging
Control
Control tasks
Engineering
Error correction
Error detection
Hybrid systems
Mechatronics
Multiple choice
Muscles
Pattern Recognition and Graphics
Plug & play
Robotics
Robotics and Automation
Robots
Safety critical
Signal classification
Special Issue: Robotics: Science and Systems 2018
Supervisory control
Vision
Error-related potentials
EEG control
Hybrid control
Gesture detection
Human–robot interaction
EMG control
Plug-and-play supervisory control
Online AccessGet full text

Cover

More Information
Summary:Effective human supervision of robots can be key for ensuring correct robot operation in a variety of potentially safety-critical scenarios. This paper takes a step towards fast and reliable human intervention in supervisory control tasks by combining two streams of human biosignals: muscle and brain activity acquired via EMG and EEG, respectively. It presents continuous classification of left and right hand-gestures using muscle signals, time-locked classification of error-related potentials using brain signals (unconsciously produced when observing an error), and a framework that combines these pipelines to detect and correct robot mistakes during multiple-choice tasks. The resulting hybrid system is evaluated in a “plug-and-play” fashion with 7 untrained subjects supervising an autonomous robot performing a target selection task. Offline analysis further explores the EMG classification performance, and investigates methods to select subsets of training data that may facilitate generalizable plug-and-play classifiers.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:0929-5593
1573-7527
DOI:10.1007/s10514-020-09916-x