Assessment with computer agents that engage in conversational dialogues and trialogues with learners

• Published in: Computers in human behavior Vol. 76; pp. 607 - 616
• Main Authors: Graesser, Arthur C., Cai, Zhiqiang, Morgan, Brent, Wang, Lijia
• Format: Journal Article
• Language: English
• Published: Elmsford Elsevier Ltd 01.11.2017; Elsevier Science Ltd
• Subjects: Assessments; Avatars; Conversation-based assessment; Conversational agents; Emotions; Intelligent tutoring systems; Interactive computer systems; Multimedia; Natural language (computers); Personal communication; Social factors; Talking heads; Trialogues; Tutoring; Conversational agents; Conversation-based assessment; Intelligent tutoring systems; Trialogues
• ISSN: 0747-5632; 1873-7692
• DOI: 10.1016/j.chb.2017.03.041

This article describes conversation-based assessments with computer agents that interact with humans through chat, talking heads, or embodied animated avatars. Some of these agents perform actions, interact with multimedia, hold conversations with humans in natural language, and adaptively respond to a person’s actions, verbal contributions, and emotions. Data are logged throughout the interactions in order to assess the individual’s mastery of subject matters, skills, and proficiencies on both cognitive and noncognitive characteristics. There are different agent-based designs that focus on learning and assessment. Dialogues occur between one agent and one human, as in the case of intelligent tutoring systems. Three-party conversations, called trialogues, involve two agents interacting with a human. The two agents can take on different roles (such as tutors and peers), model actions and social interactions, stage arguments, solicit help from the human, and collaboratively solve problems. Examples of assessment with these agent-based environments are presented in the context of intelligent tutoring, educational games, and interventions to help struggling adult readers. Most of these involve assessment at varying grain sizes to guide the intelligent interaction, but conversation-based assessment with agents is also currently being used in high stakes assessments. •Computer agents interact with humans in dialogues and trialogues with two agents.•Intelligent agents reliably analyze many but not all aspects of natural language.•Human tutoring has systematic discourse patterns that can be simulated by agents.•Intelligent tutoring systems with agents improve learning of difficult content.•Computer agents are used in conversation-based formative and summative assessments.