A practical augmented reality-assisted maintenance system framework for adaptive user support

• Published in: Robotics and computer-integrated manufacturing Vol. 59; pp. 115 - 129
• Main Authors: Siew, C.Y., Ong, S.K., Nee, A.Y.C.
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.10.2019; Elsevier BV
• Subjects: Adaptive systems; Augmented reality; Feedback; Human-computer interaction; Image detection; Maintenance; Man-machine interfaces; Modules; Operators; User services; Viability; Wrist; Human-computer-interaction; Augmented reality; Maintenance
• ISSN: 0736-5845; 1879-2537
• DOI: 10.1016/j.rcim.2019.03.010

•A practical wrist tracking methodology for user tracking and evaluation.•User guidance via an adaptive augmented reality human-machine interface.•An augmented reality feedback framework to achieve closed-loop maintenance. This paper describes an Augmented Reality Maintenance System (ARMS) that can provide adaptive support and feedback to a user during maintenance. An innovative wrist-based haptic tracking methodology is implemented to provide maintenance step awareness to the maintenance operator when he/she is working within the maintenance workspace. This is integrated with an image-based step detection module to facilitate the determination on whether a step has been conducted correctly. To further adapt to the user's visualisation requirements during maintenance, a novel adaptive human-machine interface module has been incorporated. The result is a seamless AR-assisted maintenance platform that can better cater to the needs of the operators during maintenance by providing useful feedback to the operators. The system has been demonstrated through a case study to highlight the viability of the methodology in actual industrial scenarios.