Real-Time Marker-Based Finger Tracking with Neural Networks

Hands in virtual reality applications represent our primary means for interacting with the environment. Although marker-based motion capture with inverse kinematics (IK) works for body tracking, it is less reliable for fingers often occluded when captured with cameras. Many computer vision and virtu...

Full description

Saved in:
Bibliographic Details
Published in2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) pp. 651 - 652
Main Authors Pavllo, Dario, Porssut, Thibault, Herbelin, Bruno, Boulic, Ronan
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.03.2018
Subjects
Computing methodologies-Neural networks
Human-centered computing-Virtual reality
Kinematics
Machine learning
Prediction algorithms
Predictive models
Real-time systems
Tracking
Virtual reality
Online AccessGet full text
DOI10.1109/VR.2018.8446173

Cover

More Information
Summary:Hands in virtual reality applications represent our primary means for interacting with the environment. Although marker-based motion capture with inverse kinematics (IK) works for body tracking, it is less reliable for fingers often occluded when captured with cameras. Many computer vision and virtual reality applications circumvent the problem by using an additional system (e.g. inertial trackers). We explore an alternative solution that tracks hands and fingers using solely a motion capture system based on cameras and active markers with machine learning techniques. Our animation of fingers is performed by a predictive model based on neural networks, which is trained on a movements dataset acquired from several subjects with a complementary capture system (inertial). The system is as efficient as a traditional IK algorithm, provides a natural reconstruction of postures, and handles occlusions.
DOI:10.1109/VR.2018.8446173