"I Think you Need Help! Here's why": Understanding the Effect of Explanations on Automatic Facial Expression Recognition

• Published in: International Conference on Affective Computing and Intelligent Interaction and workshops pp. 108 - 115
• Main Authors: Nahulanthran, Sanjeev, Vered, Mor, Tian, Leimin, Kulic, Dana
• Format: Conference Proceeding
• Language: English
• Published: IEEE 15.09.2024
• Subjects: Affective computing; Control systems; Emotion recognition; explainable artificial intelligence; Face recognition; facial expression recognition; Games; human-computer interaction; Intelligent systems; Navigation; Surveys
• DOI: 10.1109/ACII63134.2024.00017

Facial expression recognition (FER) has emerged as a promising approach to the development of emotion-aware intelligent systems. The performance of FER in multiple domains is continuously being improved, especially through advancements in data-driven learning approaches. However, a key challenge remains in utilizing FER in real-world contexts, namely ensuring user understanding of these systems and establishing a suitable level of user trust towards this technology. We conducted an empirical user study to investigate how explanations of FER can improve trust, understanding and performance in a human-computer interaction task that uses FER to trigger helpful hints during a navigation game. Our results showed that users provided with explanations of the FER system demonstrated improved control in using the system to their advantage, leading to a significant improvement in their understanding of the system, reduced collisions in the navigation game, as well as increased trust towards the system.