Has Retrieval Technology in Vertical Site Search Systems Improved over the Years? A Holistic Evaluation for Real Web Systems

Evaluation of retrieval systems is mostly limited to laboratory settings and rarely considers changes of performance over time. This article presents an evaluation of retrieval systems for internal Web site search systems between the years 2006 and 2011. A holistic evaluation methodology for real We...

Full description

Saved in:
Bibliographic Details
Published inJournal of information science theory and practice Vol. 3; no. 4; pp. 19 - 34
Main Authors Mandl, Thomas, Womser-Hacker, Christa, Gatzke, Natalia
Format Journal Article
LanguageKorean
Published 2015
Subjects
Evaluation
Site Search
Information Retrieval
Online AccessGet full text

Cover

More Information
Summary:Evaluation of retrieval systems is mostly limited to laboratory settings and rarely considers changes of performance over time. This article presents an evaluation of retrieval systems for internal Web site search systems between the years 2006 and 2011. A holistic evaluation methodology for real Web sites was developed which includes tests for functionality, search quality, and user interaction. Among other sites, one set of 20 Web site search systems was evaluated three times in different years and no substantial improvement could be shown. It is surprising that the communication between site and user still leads to very poor results in many cases. Overall, the quality of these search systems could be improved, and several areas for improvement are apparent from our evaluation. For a comparison, Google's site search function was also tested with the same tasks.
Bibliography:KISTI1.1003/JNL.JAKO201536459386908
ISSN:2287-9099
2287-4577