Walking-in-place for omnidirectional VR locomotion using a single RGB camera

• Published in: Virtual reality : the journal of the Virtual Reality Society Vol. 26; no. 1; pp. 173 - 186
• Main Authors: Kim, Woojoo, Sung, Jaeho, Xiong, Shuping
• Format: Journal Article
• Language: English
• Published: London Springer London 01.03.2022; Springer Nature B.V
• Subjects: Artificial Intelligence; Cameras; Computer Graphics; Computer Science; Image Processing and Computer Vision; Locomotion; Low cost; Original Article; Pose estimation; Sensors; User Interfaces and Human Computer Interaction; Virtual environments; Virtual reality; Walking; Kinect; Virtual environment; Walking in place; Navigation control; VR locomotion; OpenPose
• ISSN: 1359-4338; 1434-9957
• DOI: 10.1007/s10055-021-00551-0

Locomotion is a fundamental interaction element allowing navigation inside the virtual environment, and the walking-in-place (WIP) techniques have been actively developed as a balanced compromise between naturalness and efficiency. One popular method to implement the WIP technique was to use a low-cost, easy to set up, and markerless Kinect, but required integration of multiple sensors or covered limited directions due to the poor tracking capability when facing non-frontal sides of the user. This study aimed to propose a WIP technique for omnidirectional VR locomotion based on a single RGB camera, utilizing an open-source 2D human pose estimation system called OpenPose. Three WIP techniques (existing Kinect-based technique, proposed Kinect-based technique, and proposed OpenPose-based technique) were compared in terms of variation of virtual walking speed and subjective evaluation through a user study with walking tasks in different directions. Experimental results showed that the proposed OpenPose-based technique performed comparably when the user faced the front of the camera, but it induced lower variation of virtual walking speed and higher subjective evaluation ratings at non-forward directions compared to other techniques. The proposed OpenPose-based WIP technique can be used in VR applications to provide a fully unobstructed VR locomotion experience. It can achieve stable WIP-based omnidirectional VR locomotion through a single low-cost easily accessible RGB camera, without the need for additional sensors, and at the same time, both hands are free for other interactions.