EchoGest: Soft Ultrasonic Waveguides Based Sensing Skin for Subject-Independent Hand Gesture Recognition

• Published in: IEEE transactions on neural systems and rehabilitation engineering Vol. 32; pp. 2366 - 2375
• Main Authors: Alemu, Medhanit Y., Lin, Yuan, Shull, Peter B.
• Format: Journal Article
• Language: English
• Published: United States IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Acoustic waveguides; Acoustics; Adult; Algorithms; Copper wire; Elastomers; Female; Gesture recognition; Gestures; Hand (anatomy); Hand - physiology; Hand gesture recognition; Humans; Language translation; Male; Mobility; Pattern Recognition, Automated - methods; Piezoelectric transducers; Real time; Rehabilitation; Reproducibility of Results; sensing skin; Sensors; Sign language; Silicone Elastomers; Silicones; Silver plating; Skin; subject-independent validation; Transducers; ultrasonic waveguides; Ultrasonics; Waveguide lasers; Waveguides; Wearable Electronic Devices; Young Adult
• ISSN: 1534-4320; 1558-0210; 1558-0210
• DOI: 10.1109/TNSRE.2024.3414136

Gesture recognition is crucial for enhancing human-computer interaction and is particularly pivotal in rehabilitation contexts, aiding individuals recovering from physical impairments and significantly improving their mobility and interactive capabilities. However, current wearable hand gesture recognition approaches are often limited in detection performance, wearability, and generalization. We thus introduce EchoGest, a novel hand gesture recognition system based on soft, stretchable, transparent artificial skin with integrated ultrasonic waveguides. Our presented system is the first to use soft ultrasonic waveguides for hand gesture recognition. EcoflexTM 00-31 and EcoflexTM 00-45 Near ClearTM silicone elastomers were employed to fabricate the artificial skin and ultrasonic waveguides, while 0.1 mm diameter silver-plated copper wires connected the transducers in the waveguides to the electrical system. The wires are enclosed within an additional elastomer layer, achieving a sensing skin with a total thickness of around <inline-formula> <tex-math notation="LaTeX">500~\mu </tex-math></inline-formula> m. Ten participants wore the EchoGest system and performed static hand gestures from two gesture sets: 8 daily life gestures and 10 American Sign Language (ASL) digits 0-9. Leave-One-Subject-Out Cross-Validation analysis demonstrated accuracies of 91.13% for daily life gestures and 88.5% for ASL gestures. The EchoGest system has significant potential in rehabilitation, particularly for tracking and evaluating hand mobility, which could substantially reduce the workload of therapists in both clinical and home-based settings. Integrating this technology could revolutionize hand gesture recognition applications, from real-time sign language translation to innovative rehabilitation techniques.