Consolidation of Usability Problems With Novice Evaluators Re-Examined in Individual vs. Collaborative Settings

Abstract Usability testing can involve multiple users and evaluators. In such cases, consolidating usability problems (UPs) constitutes an essential part of data analysis. In a between-subjects design, this study aims to re-examine a previous study by comparing the results of novice evaluators mergi...

Full description

Saved in:
Bibliographic Details
Published inInteracting with computers Vol. 31; no. 6; pp. 525 - 538
Main Authors Hoffmann, Rebekka, Jónsdóttir, Anna Helga, Hvannberg, Ebba Thora
Format Journal Article
LanguageEnglish
Published Oxford University Press 23.04.2020
Subjects
usability testing
usability problems
HCI design and evaluation methods
human-centered computing
consolidation
group decision effects
evaluator effect
Online AccessGet full text

Cover

More Information
Summary:Abstract Usability testing can involve multiple users and evaluators. In such cases, consolidating usability problems (UPs) constitutes an essential part of data analysis. In a between-subjects design, this study aims to re-examine a previous study by comparing the results of novice evaluators merging UPs individually vs. collaboratively and to assess the quality of the final UP lists, by computing the merging rate and the accuracy rate, respectively. Law and Hvannberg compared the results of evaluators merging UPs individually vs. collaboratively in a within-subjects design, revealing a tendency towards merging UPs in collaborative settings. In the present study, 45 novice evaluators consolidated four UP lists into a single UP master list while working alone or with a partner. The results showed no significant difference between evaluators in the two settings, suggesting that the UP consolidation process does not benefit from positive group decision effects.
ISSN:0953-5438
1873-7951
DOI:10.1093/iwc/iwz034