Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces

• Published in: Nature communications Vol. 13; no. 1; pp. 5815 - 12
• Main Authors: Kim, Taemin, Shin, Yejee, Kang, Kyowon, Kim, Kiho, Kim, Gwanho, Byeon, Yunsu, Kim, Hwayeon, Gao, Yuyan, Lee, Jeong Ryong, Son, Geonhui, Kim, Taeseong, Jun, Yohan, Kim, Jihyun, Lee, Jinyoung, Um, Seyun, Kwon, Yoohwan, Son, Byung Gwan, Cho, Myeongki, Sang, Mingyu, Shin, Jongwoon, Kim, Kyubeen, Suh, Jungmin, Choi, Heekyeong, Hong, Seokjun, Cheng, Huanyu, Kang, Hong-Goo, Hwang, Dosik, Yu, Ki Jun
• Format: Journal Article
• Language: English
• Published: London Nature Publishing Group UK 03.10.2022; Nature Publishing Group; Nature Portfolio
• Subjects: 142/126; 639/166/985; 639/166/987; 639/925/350/59; 639/925/927/59; Accuracy; Algorithms; Classification; Crystal structure; Crystallinity; Deep learning; Electromyography; Gauges; Humanities and Social Sciences; Interfaces; Machine learning; multidisciplinary; Reliability analysis; Science; Science (multidisciplinary); Sensors; Signal quality; Signal to noise ratio; Silicon; Speech; Strain gauges; Verbal communication; Vocalization behavior; Words (language)
• ISSN: 2041-1723; 2041-1723
• DOI: 10.1038/s41467-022-33457-9

A wearable silent speech interface (SSI) is a promising platform that enables verbal communication without vocalization. The most widely studied methodology for SSI focuses on surface electromyography (sEMG). However, sEMG suffers from low scalability because of signal quality-related issues, including signal-to-noise ratio and interelectrode interference. Hence, here, we present a novel SSI by utilizing crystalline-silicon-based strain sensors combined with a 3D convolutional deep learning algorithm. Two perpendicularly placed strain gauges with minimized cell dimension (<0.1 mm 2 ) could effectively capture the biaxial strain information with high reliability. We attached four strain sensors near the subject’s mouths and collected strain data of unprecedently large wordsets (100 words), which our SSI can classify at a high accuracy rate (87.53%). Several analysis methods were demonstrated to verify the system’s reliability, as well as the performance comparison with another SSI using sEMG electrodes with the same dimension, which exhibited a relatively low accuracy rate (42.60%). Designing an efficient platform that enables verbal communication without vocalization remains a challenge. Here, the authors propose a silent speech interface by utilizing a deep learning algorithm combined with strain sensors attached near the subject’s mouth, able to collect 100 words and classify at a high accuracy rate.