Hand Gesture Recognition Applied to the Interaction with Video Games

• Published in: Advances in Computational Intelligence pp. 36 - 52
• Main Authors: Barona López, Lorena Isabel, León Cifuentes, César Israel, Muñoz Oña, José Miguel, Valdivieso Caraguay, Angel Leonardo, Benalcázar, Marco E
• Format: Book Chapter
• Language: English
• Published: Cham Springer Nature Switzerland 01.01.2023
• Series: Lecture Notes in Computer Science
• Subjects: Artificial Neural Networks; Electromyography; Hand Gesture Recognition; Inertial Measurement Unit; Quaternion; Real Time
• ISBN: 9783031477645; 3031477642
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-031-47765-2_3

In this work, a hand gesture recognition system was created for 11 different gestures. The system employed CNN-LSTM artificial neural networks and followed the CRISP-ML(Q) process model. The aim was to incorporate software engineering practices into machine learning projects. The system uses Electromyography (EMG) and Inertial Measurement Unit (IMU) signals as input to compute a gesture label and the time of occurrence in the signal. The system is integrated with a video game that utilizes hand gestures as input. A system usability scale (SUS) survey was done by ten final users in order to measure the interaction with the video game using gestures as the main way of interaction. The complete application evaluation obtained a SUS score of 75, or a B grade.