Measuring Game Engagement Multiple Methods and Construct Complexity

• Published in: Simulation & gaming Vol. 45; no. 4-5; pp. 528 - 547
• Main Authors: Martey, Rosa Mikeal, Kenski, Kate, Folkestad, James, Feldman, Laurie, Gordis, Elana, Shaw, Adrienne, Stromer-Galley, Jennifer, Clegg, Ben, Zhang, Hui, Kaufman, Nissim, Rabkin, Ari N., Shaikh, Samira, Strzalkowski, Tomek
• Format: Journal Article
• Language: English
• Published: Los Angeles, CA SAGE Publications 01.08.2014; SAGE PUBLICATIONS, INC
• Subjects: Attention; Complexity; Construction; Content analysis; Correlation analysis; Customizing; Education; Educational Games; Engagement; Game theory; Games; Learning; Measurement; Movements; Narratives; Online entertainment; Quantitative psychology; Simulation; Studies; Transport; User behavior; art; narrative complexity; EDA; engagement; self-report; narrative; art complexity; measurement; transportation; character customization; content analyses; educational games; multiple methods; customized character; game clicks; mouse movements; electro-dermal activity; attention; interactivity; presence
• ISSN: 1046-8781; 1552-826X
• DOI: 10.1177/1046878114553575

Background. Engagement has been identified as a crucial component of learning in games research. However, the conceptualization and operationalization of engagement vary widely in the literature. Many valuable approaches illuminate ways in which presence, flow, arousal, participation, and other concepts constitute or contribute to engagement. However, few studies examine multiple conceptualizations of engagement in the same project. Method. This article discusses the results of two experiments that measure engagement in five different ways: survey self-report, content analyses of player videos, electro-dermal activity, mouse movements, and game click logs. We examine the relationships among these measures and assess how they are affected by the technical characteristics of a 30-minute, custom-built, educational game: use of a customized character, level of narrative complexity, and level of art complexity. Results. We found that the five measures of engagement correlated in limited ways, and that they revealed substantially different relationships with game characteristics. We conclude that engagement as a construct is more complex than is captured in any of these measures individually and that using multiple methods to assess engagement can illuminate aspects of engagement not detectable by a single method of measurement.