Asynchronous iterative computations with Web information retrieval structures: The PageRank case

TL;DR

This paper explores asynchronous iterative algorithms for computing PageRank on large-scale, distributed web data, aiming to improve efficiency and scalability in heterogeneous computing environments.

Contribution

It introduces asynchronous iterative schemes for PageRank computation, eliminating synchronization phases to enhance performance on large, distributed web datasets.

Findings

Asynchronous schemes reduce synchronization overhead.

Distributed memory platforms are necessary for large Web graphs.

Potential for scalable PageRank computation on Grid architectures.

Abstract

There are several ideas being used today for Web information retrieval, and specifically in Web search engines. The PageRank algorithm is one of those that introduce a content-neutral ranking function over Web pages. This ranking is applied to the set of pages returned by the Google search engine in response to posting a search query. PageRank is based in part on two simple common sense concepts: (i)A page is important if many important pages include links to it. (ii)A page containing many links has reduced impact on the importance of the pages it links to. In this paper we focus on asynchronous iterative schemes to compute PageRank over large sets of Web pages. The elimination of the synchronizing phases is expected to be advantageous on heterogeneous platforms. The motivation for a possible move to such large scale distributed platforms lies in the size of matrices representing Web structure. In orders of magnitude: $10^{10}$ pages with $10^{11}$ nonzero elements and $10^{12}$ bytes just to store a small percentage of the Web (the already crawled); distributed memory machines are necessary for such computations. The present research is part of our general objective, to explore the potential of asynchronous computational models as an underlying framework for very large scale computations over the Grid. The area of ``internet algorithmics'' appears to offer many occasions for computations of unprecedent dimensionality that would be good candidates for this framework.