Evaluating Server Selection for Federated Search

• Published in: Advances in Information Retrieval pp. 607 - 610
• Main Authors: Thomas, Paul, Shokouhi, Milad
• Format: Book Chapter
• Language: English
• Published: Berlin, Heidelberg Springer Berlin Heidelberg 2010
• Series: Lecture Notes in Computer Science
• ISBN: 3642122744; 9783642122743
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-642-12275-0_58

Previous evaluations of server selection methods for federated search have either used metrics which are unconnected with user satisfaction, or have not been able to account for confounding factors due to other search components. We propose a new framework for evaluating federated search server selection techniques. In our model, we isolate the effect of other confounding factors such as server summaries and result merging. Our results suggest that state-of-the-art server selection techniques are generally effective but result merging methods can be significantly improved. Furthermore, we show that the performance differences among server selection techniques can be obscured by ineffective merging.