Using a Layered Markov Model for Distributed Web Ranking Computation

• Published in: 25th IEEE International Conference on Distributed Computing Systems (ICDCS'05) pp. 533 - 542
• Main Authors: Jie Wu, Aberer, K.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 2005
• Subjects: Computer architecture; Content based retrieval; Distributed computing; Peer to peer computing; Proposals; Search engines; Service oriented architecture; Vectors; Web pages; Web search
• ISBN: 9780769523316; 0769523315
• ISSN: 1063-6927; 2575-8411
• DOI: 10.1109/ICDCS.2005.84

The link structure of the Web graph is used in algorithms such as Kleinberg's HITS and Google's PageRank to assign authoritative weights to Web pages and thus rank them. Both require a centralized computation of the ranking if used to rank the complete Web graph. In this paper, we propose a new approach based on a Layered Markov Model to distinguish transitions among Web sites and Web documents. Based on this model, we propose two different approaches for computation of ranking of Web documents, a centralized one and a decentralized one. Both produce a well-defined ranking for a given Web graph. We then formally prove that the two approaches are equivalent. This provides a theoretical foundation for decomposing link-based rank computation and makes the computation for a Web-scale graph feasible in a decentralized fashion, such as required for Web search engines having a peer-to-peer architecture. Furthermore, personalized rankings can be produced by adapting the computation at both the local layer and the global layer. Our empirical results show that the ranking generated by our model is qualitatively comparable to or even better than the ranking produced by PageRank