CTrack: Acoustic Device-Free and Collaborative Hands Motion Tracking on Smartphones

• Published in: IEEE internet of things journal Vol. 8; no. 19; pp. 14658 - 14671
• Main Authors: Jiang, Hongbo, Wang, Minglin, Liu, Daibo, Zhou, Siwang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.10.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Acoustics; Adaptive tracking; Arrays; Chirp; Collaboration; Echoes; Electronic devices; Ellipses; Hands; human–computer interaction (HCI); inaudible sound sensing; Internet of Things; localization; Microphone arrays; Microphones; Sensors; Smart phones; Smartphones; Sonar; Tracking; Tracking devices
• ISSN: 2327-4662; 2327-4662
• DOI: 10.1109/JIOT.2021.3071287

Enabling contactless and device-free hands tracking on mobile device leads to new user interaction experiences. In this article, we propose CTrack, a device-free and collaborative hands motion tracking solution for above-device interaction by using acoustic signals. CTrack does not require instrumenting hands with sensors. We achieve this by transforming the device into an active sonar system that transmits inaudible sound signals and tracks the echoes of the hand at its microphones. To guarantee subcentimeter-level tracking accuracies, we present an adaptive approach that uses the chirp's time of flight to accurately measure the distance from the hand to an in-built speaker array. Then, the hand, speaker array, and microphone array yield a set of different ellipses. The hand position can be pinpointed exactly by solving and optimizing the intersection of these ellipses. Our evaluation shows that CTrack can achieve 2-D motion tracking with an average accuracy of 14 mm using the in-built microphones and speakers of a Nexus 6P.