Toward physically realistic vision in teleoperation: A user study with light‐field head mounted display and 6‐DoF head motion

• Published in: Journal of the Society for Information Display Vol. 31; no. 12; pp. 663 - 674
• Main Authors: Bechtel, Nicolai, Weber, Bernhard, Severin, Pascal, Sancho Aragon, Jaime, Van Bogaert, Laurie, Panzirsch, Michael
• Format: Journal Article
• Language: English
• Published: Campbell Wiley Subscription Services, Inc 01.12.2023
• Subjects: Accommodation; Asthenopia; Display devices; Environment models; Head movement; Helmet mounted displays; light‐field displays; vergence‐accommodation conflict
• ISSN: 1071-0922; 1938-3657
• DOI: 10.1002/jsid.1262

Besides haptics, the visual channel provides the most essential feedback to the operator in teleoperation setups. For optimal performance, the view on the remote scene must provide 3D information, be sharp, and of high resolution. Head‐mounted displays (HMD) are applied to improve the immersion of the operator into the remote environment. Still, so far, no near‐eye display technology was available that provides a natural view on objects within the typical manipulation distance (up to 1.2 m). The main limitation is a mismatch of the 3D distance and the focal distance of the visualized objects (vergence‐accommodation conflict) in displays with fixed focal distance. This conflict potentially leads to eye strain after extended use. Here, we apply a light‐field HMD providing close‐to‐continuous depth information to the user, thus avoiding the vergence‐accommodation conflict. Furthermore, we apply a time‐of‐flight sensor to generate a 2.5D environment model. The displayed content is processed with image‐based rendering allowing a 6 degree‐of‐freedom head motion in the visualized scene. The main objective of the presented study is evaluating the effects of view perspective and light‐field on performance and workload in a teleoperation setup. The reduction of visual effort for the user is confirmed in an depth‐matching task. The presented study with light‐field acquisition and rendering was conducted with n=28 test‐subjects performing a tele‐robotic depth matching task based on a 2.5D visualization of the robotic environment. Light‐field: The user study showed that light‐field (LF) head‐mounted displays reduce workload. View perspective: Rotational and translational perspective changes enabled through real‐time view synthesis improve depth matching accuracy.