ThumbUp: Identification and Authentication by Smartwatch using Simple Hand Gestures

• Published in: Proceedings of the IEEE International Conference on Pervasive Computing and Communications pp. 1 - 10
• Main Authors: Yu, Xiaojing, Zhou, Zhijun, Xu, Mingxue, You, Xuanke, Li, Xiang-Yang
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.03.2020
• Subjects: Authentication; Deep learning; Hand Gestures; Pervasive computing; Real-time systems; Robustness; Smartwatch; System performance; Thumb
• ISSN: 2474-249X
• DOI: 10.1109/PerCom45495.2020.9127367

The widespread creative application and smart devices call for convenient and secure interaction with human users. We propose, design, and implement a smartwatch-based two-factor real-time identification and authentication system named ThumbUp, where smartwatch users can identify and authenticate themselves by some simple hand and finger gestures, such as thumb-up. ThumbUp leverages the signal collected from the Inertial Measurement Unit (IMU) in Commercial Off-The-Shelf (COTS) smart devices and discovers the unique fingerprint pattern produced by each user's simple hand gestures using a carefully crafted deep learning model. We implement our system and conduct extensive experiments to evaluate its efficacy and efficiency with 65 different users over a period of more than 3 months. It reaches an accuracy of 97% for identification, and EER 0.014 for authentication using only one simple gesture. We also survey the users' acceptance of our system and discuss how the proficiency of gestures affects authentication accuracy.