Ultra Write: A Lightweight Continuous Gesture Input System with Ultrasonic Signals on COTS Devices

• Published in: Proceedings of the IEEE International Conference on Pervasive Computing and Communications pp. 174 - 183
• Main Authors: Chen, Weiyu, Zheng, Canlin, He, Wenfeng, Zou, Yongpan, Wu, Kaishun
• Format: Conference Proceeding
• Language: English
• Published: IEEE 11.03.2024
• Subjects: Acoustic Sensing; Acoustics; Costs; Cross-Domain Learning; Gesture Input; Natural language processing; Performance evaluation; Pervasive computing; Privacy; Robustness
• ISSN: 2474-249X
• DOI: 10.1109/PerCom59722.2024.10494485

Due to the advantages of acoustic sensing such as device ubiquity, hands-free interaction and privacy security, acoustic-based gesture input techniques have gained extensive at-tention and many excellent works have been proposed. However, these works have the following shortcomings: high cost of system construction, non-continuous input, and degraded performance in cross-user scenarios. To overcome the above shortcomings, we propose UltraWrite, a continuous gesture input system that needs rather low system construction cost, supports continuous input, and achieves high cross-user recognition performance. The key idea of our solution is to synthesize the data of continuous gestures from isolated ones, and build an end-to-end continuous gesture recognition model by introducing the connectionist temporal classification (CTC) mechanism widely used in natural language processing. We have implemented the system on a commercial tablet and conducted comprehensive experiments to evaluate its performance. The results demonstrate that UltraWrite achieves an average word accuracy of 99.3% and word error rate of 0.34% when considering only the first output candidate word. In addition, we have also evaluated the system's robustness to background noise, sensing distance and angle. The results reveal that UltraWrite displays strong robustness to these factors.