Evaluating image retrieval

We present a comprehensive strategy for evaluating image retrieval algorithms. Because automated image retrieval is only meaningful in its service to people, performance characterization must be grounded in human evaluation. Thus we have collected a large data set of human evaluations of retrieval r...

Full description

Saved in:
Bibliographic Details
Published in2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) Vol. 1; pp. 955 - 961 vol. 1
Main Authors Shirahatti, N.V., Barnard, K.
Format Conference Proceeding
LanguageEnglish
Published IEEE 2005
Subjects
Biomedical imaging
Calibration
Computer science
Computer Society
Computer vision
Content based retrieval
Humans
Image retrieval
Information retrieval
Vocabulary
Online AccessGet full text

Cover

More Information
Summary:We present a comprehensive strategy for evaluating image retrieval algorithms. Because automated image retrieval is only meaningful in its service to people, performance characterization must be grounded in human evaluation. Thus we have collected a large data set of human evaluations of retrieval results, both for query by image example and query by text. The data is independent of any particular image retrieval algorithm and can be used to evaluate and compare many such algorithms without further data collection. The data and calibration software are available on-line. We develop and validate methods for generating sensible evaluation data, calibrating for disparate evaluators, mapping image retrieval system scores to the human evaluation results, and comparing retrieval systems. We demonstrate the process by providing grounded comparison results for several algorithms.
ISBN:0769523722
9780769523729
ISSN:1063-6919
1063-6919
DOI:10.1109/CVPR.2005.147