Delayed Asynchronous Iterative Graph Algorithms

TL;DR

This paper introduces a hybrid asynchronous-synchronous approach for iterative graph algorithms that buffers updates locally to reduce memory contention and improve convergence speed on multi-threaded systems.

Contribution

It proposes a novel hybrid method that balances asynchronous update speed with contention control, outperforming purely synchronous or asynchronous methods.

Findings

Achieves up to 19.4% speedup over asynchronous Pagerank

Attains 2.56x better performance than synchronous algorithms

Effective on graphs with less clustered connectivity

Abstract

Iterative graph algorithms often compute intermediate values and update them as computation progresses. Updated output values are used as inputs for computations in current or subsequent iterations; hence the number of iterations required for values to converge can potentially reduce if the newest values are asynchronously made available to other updates computed in the same iteration. In a multi-threaded shared memory system, the immediate propagation of updated values can cause memory contention that may offset the benefit of propagating updates sooner. In some cases, the benefit of a smaller number of iterations may be diminished by each iteration taking longer. Our key idea is to combine the low memory contention that synchronous approaches have with the faster information sharing of asynchronous approaches. Our hybrid approach buffers updates from threads locally before committing them to the global store to control how often threads may cause conflicts for others while still sharing data within one iteration and hence speeding convergence. On a 112-thread CPU system, our hybrid approach attains up to 4.5% - 19.4% speedup over an asynchronous approach for Pagerank and up to 1.9% - 17% speedup over asynchronous Bellman Ford SSSP. Further, our hybrid approach attains 2.56x better performance than the synchronous approach. Finally, we provide insights as to why delaying updates is not helpful on certain graphs where connectivity is clustered on the main diagonal of the adjacency matrix.