EISEval, a generic reconfigurable environment for evaluating agent-based interactive systems

• Published in: International journal of human-computer studies Vol. 71; no. 6; pp. 725 - 761
• Main Authors: Tran, Chi Dung, Ezzedine, Houcine, Kolski, Christophe
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.06.2013; Elsevier
• Subjects: Agent-based architecture; Algorithmics. Computability. Computer arithmetics; Applied sciences; Artificial intelligence; Computer Science; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Electronic informer; Ergonomic criteria; Evaluation; Exact sciences and technology; Human-computer interaction (HCI); Interactive systems; Multiagent Systems; Software; Theoretical computing; User interface; Evaluation; Agent-based architecture; Interactive systems; User interface; Human-computer interaction (HCI); Ergonomic criteria; Electronic informer; System architecture; Man machine system; Modeling; Time response; Interactive system; Multiagent system; Ergonomics; Response time; Preference; Urban transportation; Reconfigurable architectures; Time complexity; Artificial intelligence; multi-agent system; artificial intelligence
• ISSN: 1071-5819; 1095-9300
• DOI: 10.1016/j.ijhcs.2013.02.001

The evaluation of interactive systems has been an active subject of research for many years. Many methods have been proposed, but most of them do not take the architectural specificities of an agent-based interactive system into account, nor do they focus on the link between architecture and evaluation. In this paper, we present an agent-based architecture model for interactive systems. Then, based on this architecture, we propose a generic, reconfigurable evaluation environment, called EISEval, designed and developed to help evaluators analyze and evaluate certain aspects of interactive systems in general and of agent-based architecture interactive systems in particular: User Interface (UI), non-functional properties (e.g., response time, complexity) and user characteristics (e.g., abilities, preferences, progress). System designers can draw useful conclusions from the evaluation results to improve the system. This environment was applied to evaluate an agent-based interactive system used to supervise an urban transport network in a study organized in laboratory. [Display omitted] ► We consider agent-based architecture model in evaluation. ► A new type of evaluation tool is described. ► The feasibility of this approach has been successfully showed in a study in transport domain.