Mixed Reality Human Teleoperation

• Published in: 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) pp. 375 - 383
• Main Authors: Black, David, Salcudean, Septimiu
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.03.2023
• Subjects: Computer systems organization-Embedded and cyber-physical systems-Robotics-Robotic Control; Human computer interaction; Human-centered computing-Human Computer Interaction-Interaction Devices-Haptic devices; Human-centered computing-Human Computer Interaction-Interaction Paradigms-Mixed / augmented reality; Maintenance engineering; Medical services; Mixed reality; Three-dimensional displays; Training; Virtual reality
• DOI: 10.1109/VRW58643.2023.00083

For many applications, remote guidance and telerobotics provide great advantages. For example, tele-ultrasound can bring much-needed expert healthcare to isolated communities. However, existing tele-guidance methods have serious limitations. A new concept called human teleoperation leverages mixed reality, haptics, and high-speed communication to provide tele-guidance that is more tightly coupled than existing methods yet more accessible than telerobotics. This paper provides an overview of the human teleoperation concept and its application to tele-ultrasound. The concept and its impact are discussed, the graphics, communications, controls, and haptics subsystems are explained, and results are presented that show the system's efficacy. These include tests of the communication architecture, of human performance in tracking mixed reality signals, and of human teleoperation in a limited clinical use-case. The results show good potential for teleultrasound, as well as possible other applications of human teleoperation including remote maintenance, inspection, and training.