Learning query-dependent prefilters for scalable image retrieval

• Published in: 2009 IEEE Conference on Computer Vision and Pattern Recognition pp. 2615 - 2622
• Main Authors: Torresani, Lorenzo, Szummer, Martin, Fitzgibbon, Andrew
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2009
• Subjects: Dictionaries; Histograms; Image databases; Image retrieval; Image sampling; Information filtering; Information filters; Information retrieval; Testing; Visual databases
• ISBN: 9781424439928; 1424439922
• ISSN: 1063-6919
• DOI: 10.1109/CVPR.2009.5206582

We describe an algorithm for similar-image search which is designed to be efficient for extremely large collections of images. For each query, a small response set is selected by a fast prefilter, after which a more accurate ranker may be applied to each image in the response set. We consider a class of prefilters comprising disjunctions of conjunctions ("ORs of ANDs") of Boolean features. AND filters can be implemented efficiently using skipped inverted files, a key component of Web-scale text search engines. These structures permit search in time proportional to the response set size. The prefilters are learned from training examples, and refined at query time to produce an approximately bounded response set. We cast prefiltering as an optimization problem: for each test query, select the OR-of-AND filter which maximizes training-set recall for an adjustable bound on response set size. This may be efficiently implemented by selecting from a large pool of candidate conjunctions of Boolean features using a linear program relaxation. Tests on object class recognition show that this relatively simple filter is nevertheless powerful enough to capture some semantic information.