A Deep Learning Technique as a Sensor Fusion for Enhancing the Position in a Virtual Reality Micro-Environment

• Published in: 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) pp. 4935 - 4940
• Main Authors: Prieto Prada, John David, Luna, Miguel, Park, Sang Hyun, Song, Cheol
• Format: Conference Proceeding
• Language: English
• Published: IEEE 23.10.2022
• Subjects: Deep learning; Dynamics; Heuristic algorithms; Solid modeling; System performance; Tracking; Virtual environments
• ISSN: 2153-0866
• DOI: 10.1109/IROS47612.2022.9981239

Most virtual reality (VR) applications use a commercial controller for interaction. However, a typical virtual reality controller (VRC) lacks positional precision and accu-racy in millimeter-scale scenarios. This lack of precision and accuracy is caused by built-in sensors drift. Therefore, the tracking performance of a VRC needs to be enhanced for millimeter-scale scenarios. Herein, we introduce a novel way of enhancing the tracking performance of a commercial VRC in a millimeter-scale environment using a deep learning (DL) al-gorithm. Specifically, we use a long short-term memory (LSTM) model trained with data collected from a linear motor, an IMU sensor, and a VRC. We integrate the virtual environment developed in Unity software with the LSTM model running in Python. We designed three experimental conditions: the VRC, Kalman filter (KF), and LSTM modes. Furthermore, we evaluate tracking performances in the three conditions and two other experimental scenarios, namely stationary and dynamic. In the stationary experimental scenario, the system is left motionless for 10 s. By contrast, in the dynamic experimental scenarios, the linear stage moves the system by 12 mm along the X, Y, and Z axes. The experimental results indicate that the deep learning model outperforms the standard controllers positional performance by 85.69 % and 92.14 % in static and dynamic situations, respectively.