ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke

• Published in: IEEE robotics and automation letters Vol. 10; no. 2; pp. 907 - 914
• Main Authors: Xu, Jingxi, Wang, Runsheng, Shang, Siqi, Chen, Ava, Winterbottom, Lauren, Hsu, To-Liang, Chen, Wenxi, Ahmed, Khondoker, Rotta, Pedro Leandro La, Zhu, Xinyue, Nilsen, Dawn M., Stein, Joel, Ciocarlie, Matei
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.02.2025; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Context; Data collection; Data models; Datasets; Electromyography; End effectors; Generative AI; Motors; Orthoses; prosthetics and exoskeletons; rehabilitation robotics; Robot control; Robots; Signal generation; Stroke; Stroke (medical condition); Synthetic data; Training; Training data; Urban areas; wearable robotics
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2024.3511372

Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train intent classifiers; however, this data collection process is burdensome and time-consuming. In this letter, we propose ChatEMG, an autoregressive generative model that can generate synthetic EMG signals conditioned on prompts (i.e., a given sequence of EMG signals). ChatEMG enables us to collect only a small dataset from the new condition, session, or subject and expand it with synthetic samples conditioned on prompts from this new context. ChatEMG leverages a vast repository of previous data via generative training while still remaining context-specific via prompting. Our experiments show that these synthetic samples are classifier-agnostic and can improve intent inferral accuracy for different types of classifiers. We demonstrate that our complete approach can be integrated into a single patient session, including the use of the classifier for functional orthosis-assisted tasks. To the best of our knowledge, this is the first time an intent classifier trained partially on synthetic data has been deployed for functional control of an orthosis by a stroke survivor.