PHIRST: A distributed architecture for P2P information retrieval

• Published in: Information systems (Oxford) Vol. 34; no. 2; pp. 290 - 303
• Main Authors: Rosenfeld, Avi, Goldman, Claudia V., Kaminka, Gal A, Kraus, Sarit
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2009
• Subjects: Distributed databases; Full text searching; Information retrieval; Online data bases; Peer to peer systems; Information retrieval; Peer to peer systems; Distributed databases
• ISSN: 0306-4379; 1873-6076
• DOI: 10.1016/j.is.2008.08.002

Recent progress in peer to peer (P2P) search algorithms has presented viable structured and unstructured approaches for full-text search. We posit that these existing approaches are each best suited for different types of queries. We present PHIRST, the first system to facilitate effective full-text search within P2P databases. PHIRST works by effectively leveraging between the relative strengths of these approaches. Similar to structured approaches, agents first publish terms within their stored documents. However, frequent terms are quickly identified and not exhaustively stored, resulting in a significant reduction in the system's storage requirements. During query lookup, agents use unstructured search to compensate for the lack of fully published terms. Additionally, they explicitly weigh between the costs involved in structured and unstructured approaches, allowing for a significant reduction in query costs. Finally, we address how node failures can be effectively addressed through storing multiple copies of selected data. We evaluated the effectiveness of our approach using both real-world and artificial queries. We found that in most situations our approach yields near perfect recall. We discuss the limitations of our system, as well as possible compensatory strategies.