Digital Twin Enabled Teleoperated Driving Under Network Delay Using Ego Vehicle Tracking

• Published in: 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC) pp. 1943 - 1948
• Main Authors: Kremer, Philipp, Nourani-Vatani, Navid, Park, Sangyoung
• Format: Conference Proceeding
• Language: English
• Published: IEEE 24.09.2023
• Subjects: Bandwidth; Digital twins; Roads; Safety; Streams; Teleoperators; Workstations
• ISSN: 2153-0017
• DOI: 10.1109/ITSC57777.2023.10422456

Vehicle teleoperation is increasingly seen as a complementary technology to autonomous driving allowing a human teleoperator to be the ultimate guarantor of safety. The major technical hurdles for teleoperated vehicles on public roads are network bandwidth and latency between the vehicle and the teleoperator workstation. A teleoperator requires a sufficiently high update rate, e.g. one update every 0.5 m of travel, and low network latency, e.g. < 300 ms, in order to comfortably control the vehicle. In order to achieve such requirements, we propose to use a real-time digital twin of the vehicular environment based on perception information from the vehicle and a delay-compensating ego vehicle tracking algorithm. Our experiments in a realistic driving simulator show that we can significantly reduce the tracking error under network latency, indicating that higher driving speeds can be achieved with lower update rates and less bandwidth consumption compared to conventional teleoperation solutions based on multiple HD video streams.