Coadaptive Brain-Machine Interface via Reinforcement Learning

• Published in: IEEE transactions on biomedical engineering Vol. 56; no. 1; pp. 54 - 64
• Main Authors: DiGiovanna $^$, Jack, Mahmoudi, Babak, Fortes, Jose, Principe, Jose C., Sanchez, Justin C.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2009; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Animals; Architecture; Biomedical engineering; Bismaleimides; Brain - physiology; Brain-machine interface (BMI); coadaptation; Control algorithms; Control theory; Electrodes, Implanted; In vivo; Kinematics; Learning; Learning - physiology; Lifting equipment; Male; Man-Machine Systems; Microelectrodes; Models, Neurological; Neural prosthesis; neuroprosthetic; Organisms; Prosthetics; Rats; Rats, Sprague-Dawley; Reinforcement; Reinforcement (Psychology); reinforcement learning (RL); Reward; Robotics - instrumentation; Semisupervised learning; Shape control; Surgical implants; Tasks; Reinforcement Learning; Neuroprosthetic; Brain-Machine Interfaces; Co-adaptation
• ISSN: 0018-9294; 1558-2531; 1558-2531
• DOI: 10.1109/TBME.2008.926699

This paper introduces and demonstrates a novel brain-machine interface (BMI) architecture based on the concepts of reinforcement learning (RL), coadaptation, and shaping. RL allows the BMI control algorithm to learn to complete tasks from interactions with the environment, rather than an explicit training signal. Coadaption enables continuous, synergistic adaptation between the BMI control algorithm and BMI user working in changing environments. Shaping is designed to reduce the learning curve for BMI users attempting to control a prosthetic. Here, we present the theory and in vivo experimental paradigm to illustrate how this BMI learns to complete a reaching task using a prosthetic arm in a 3-D workspace based on the user's neuronal activity. This semisupervised learning framework does not require user movements. We quantify BMI performance in closed-loop brain control over six to ten days for three rats as a function of increasing task difficulty. All three subjects coadapted with their BMI control algorithms to control the prosthetic significantly above chance at each level of difficulty.