The Role of Audio Feedback and Gamification Elements for Remote Boom Operation

• Published in: Multimodal technologies and interaction Vol. 8; no. 8; p. 69
• Main Authors: Burova, Alissa, Mäkelä, John, Keskinen, Tuuli, Kallioniemi, Pekka, Ronkainen, Kimmo, Turunen, Markku
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.08.2024
• Subjects: audio design; audio feedback; Control systems; Feedback; Gamification; Hazardous areas; Human error; Operator performance; Remote control; remote operations; Remote sensing; Situational awareness; Sound design; teleoperations; Unmanned aerial vehicles; Vehicles
• ISSN: 2414-4088; 2414-4088
• DOI: 10.3390/mti8080069

Remote operations have been greatly enhanced by advancements in technology, enabling remote control of machinery in hazardous environments. However, it is still a challenge to design remote control interfaces and provide feedback in a way that would enhance situational awareness without negatively affecting cognitive load. This study investigates how different audio feedback designs can support remote boom operation and, additionally, explores the potential impact of gamification elements on operator performance and motivation. Due to COVID-19 restrictions, this study was conducted remotely with 16 participants using a simulated environment featuring a virtual excavator. Participants performed digging tasks using two audio feedback designs: frequency-modulated beeping and realistic spatialized steam sounds. The findings indicate that both audio designs are beneficial for remote boom operations: the beeping sound was perceived as more comfortable and efficient in determining the proximity of a hidden object and helped in avoiding collisions, whereas spatial sounds enhanced the sense of presence. Therefore, we suggest combining both audio designs for optimal performance and emphasize the importance of customizable feedback in remote operations. This study also revealed that gamification elements could both positively and negatively affect performance and motivation, highlighting the need for careful design tailored to specific task requirements.