Controlled user evaluations of information visualization interfaces for text retrieval: Literature review and meta-analysis

• Published in: Journal of the American Society for Information Science and Technology Vol. 59; no. 6; pp. 1012 - 1024
• Main Authors: Julien, Charles-Antoine, Leide, John E., Bouthillier, France
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc., A Wiley Company 01.04.2008; Wiley; Wiley Periodicals Inc
• Subjects: Browsing; College students; Comparative analysis; Design; Design of experiments; Design Requirements; Exact sciences and technology; Information and communication sciences; Information retrieval; Information science. Documentation; Infrared analysis; Interfaces; Library and information science. General aspects; Literature reviews; Meta Analysis; Online information retrieval; Research design; Sciences and techniques of general use; Scientific visualization; Studies; Undergraduate Students; Usability; Use and user studies. Information needs; User interface; Visualization; Websites; Information retrieval; Full text; Comparative study; User study; Information representation; Result
• ISSN: 1532-2882; 2330-1635; 1532-2890; 2330-1643
• DOI: 10.1002/asi.20786

This review describes experimental designs (users, search tasks, measures, etc.) used by 31 controlled user studies of information visualization (IV) tools for textual information retrieval (IR) and a meta‐analysis of the reported statistical effects. Comparable experimental designs allow research designers to compare their results with other reports, and support the development of experimentally verified design guidelines concerning which IV techniques are better suited to which types of IR tasks. The studies generally use a within‐subject design with 15 or more undergraduate students performing browsing to known‐item tasks on sets of at least 1,000 full‐text articles or Web pages on topics of general interest/news. Results of the meta‐analysis (N = 8) showed no significant effects of the IV tool as compared with a text‐only equivalent, but the set shows great variability suggesting an inadequate basis of comparison. Experimental design recommendations are provided which would support comparison of existing IV tools for IR usability testing.