Exploring the Human-Centric Interaction Paradigm: Augmented Reality-Assisted Head-Up Display Design for Collaborative Human-Machine Interface in Cockpit

• Published in: Advanced engineering informatics Vol. 62; p. 102656
• Main Authors: YUAN, Xin, NG, Kam K.H., LI, Qinbiao, YIU, Cho Yin, LAU, Chun Kit, FUNG, Ka Hei, NG, Lok Hei
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.10.2024
• Subjects: Collaborative human–machine interface; Human factors and ergonomics; Human-centric design; Human-cybernetic interface; Single pilot operations; Single pilot operations; Human factors and ergonomics; Collaborative human–machine interface; Human-centric design; Human-cybernetic interface
• ISSN: 1474-0346
• DOI: 10.1016/j.aei.2024.102656

•An exploratory interaction paradigm, the Augmented Reality assisted Head-up display (AR-HUD), has been developed utilising Microsoft HoloLens 2, introducing collaborative human–machine interaction (CHMI) in cockpit design tailored to diverse flight scenarios.•A multidimensional assessment approach was proposed to examine the effect of the AR-HUD for pilots’ Situational awareness (SA), workload level and attention allocation.•The AR-HUD helps maintain better SA and workload levels while reducing the number of gaze points for looking down at individual instruments.•Implementing AR in the CHMI of cockpit design introduces a human-centric pilot-support system that will contribute to aviation safety and allows the implementation of SPO to provide a feasible solution. Human-cybernetic interfaces (HCI) focuses on the integration and synergy of human-centric design, human factors (HFs) and economics, and human–machine interaction (HMI). The scrutiny of single pilot operations (SPO) approval from civil aviation authorities (CAAs) is subject to the aviation safety and pilot capability during exceptional handling and flight emergency. Airliners advocate the possibility from dual pilot operations (DPO) to SPO because of the financial considerations and captain shortage in the post-pandemic era. One could expect that SPO can only realise when reduced crew operations initiative is proven to be safe and airworthy. Public concerns pilot mental workload and potentially low situational awareness (SA) when handling overwhelmed multi-source information from the HMI in a cockpit, leading to degraded skills and flying performance. Advanced head up display (HUD) design, an integrated transparent display, can assist pilots in maintaining their optimal performance when SPO exercised. In this study, we are interested to design a SPO-favoured cockpit design with HUD and investigate the potential of integrating augmented reality (AR) and HUD in SPO in flight operations. An experiment involving 21 pilots was conducted to assess the effectiveness of AR-HUD on SA and workload in DPO, SPO, and SPO with AR-HUD scenarios. The Situation Awareness Present Assessment Method (SPAM) and a self-rated workload scale were used to evaluate SA and workload levels. Eye-tracking data was also collected to analyse behavioural performance. Results indicated that the AR-HUD scenario achieved optimal workload and SA levels and demonstrated superior performance in handling emergencies. Eye-tracking data revealed that AR-HUD facilitated a more equitable distribution of gaze between instruments and external view. In conclusion, the integration of AR in cockpit design introduces a human-centric interaction paradigm, contributing to aviation safety, SPO implementation and adaptive HMI design.