The impact of individual AI proficiency on human–agent collaboration: Higher sensitivity to discern the comprehension ability of intelligent agents for users with higher AI proficiency levels

• Published in: International journal of industrial ergonomics Vol. 107; p. 103745
• Main Authors: Yu, Ruifeng, Xu, Xinran, Peng, Shuo
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.05.2025
• Subjects: AI proficiency; Human–agent collaboration; Perceived anthropomorphism; Perceived intelligence; Trust; AI proficiency; Perceived anthropomorphism; Trust; Human–agent collaboration; Perceived intelligence
• ISSN: 0169-8141
• DOI: 10.1016/j.ergon.2025.103745

This study explored the impact of individual artificial intelligence (AI) proficiency on task scores, human perceptions of the agent's intelligence and anthropomorphism, trust, and mental workload from the perspective of human–agent collaboration. A 2 (Individual AI Proficiency, between-subjects) × 2 (Intelligent Agent's Comprehension Ability for Human Intentions, within-subjects) mixed experimental design was implemented in a task in which the participants and intelligent agents collaborated to navigate an unknown map. Forty participants participated in this study. The results revealed significant interaction effects between human AI proficiency and the agents' comprehension abilities on human trust, perceived intelligence, and perceived anthropomorphism. Users with higher AI proficiency demonstrated greater trust, perceived intelligence, and anthropomorphism when interacting with intelligent agents that interpreted human intention based on both immediate and previous feedback, compared to intelligent agents based solely on immediate feedback, displaying a higher sensitivity to the change in agents' ability. Additionally, intelligent agents that interpret human intentions based on both immediate and previous feedback significantly reduce users' mental workload compared with those that rely solely on immediate feedback. •AI proficiency impacts trust, perceived intelligence, and mental workload.•High-proficiency users are more sensitive to agents' comprehension abilities.•Advanced intention understanding reduces cognitive load in human-agent teams.•Findings inform user-adaptive AI design for effective human-agent collaboration.