ThunderRW: An In-Memory Graph Random Walk Engine (Complete Version)
Shixuan Sun, Yuhang Chen, Shengliang Lu, Bingsheng He and, Yuchen Li

TL;DR
ThunderRW is an in-memory graph random walk engine that significantly improves performance and reduces CPU stalls by employing a novel programming model and execution technique, enabling efficient massive parallel random walks on large graphs.
Contribution
The paper introduces ThunderRW, a novel in-memory engine with a step-centric programming model and step interleaving technique for efficient parallel random walks.
Findings
ThunderRW outperforms existing approaches by an order of magnitude.
Step interleaving reduces CPU pipeline stalls from 73.1% to 15.0%.
Demonstrated effectiveness on PPR, DeepWalk, Node2Vec, and MetaPath algorithms.
Abstract
As random walk is a powerful tool in many graph processing, mining and learning applications, this paper proposes an efficient in-memory random walk engine named ThunderRW. Compared with existing parallel systems on improving the performance of a single graph operation, ThunderRW supports massive parallel random walks. The core design of ThunderRW is motivated by our profiling results: common RW algorithms have as high as 73.1% CPU pipeline slots stalled due to irregular memory access, which suffers significantly more memory stalls than the conventional graph workloads such as BFS and SSSP. To improve the memory efficiency, we first design a generic step-centric programming model named Gather-Move-Update to abstract different RW algorithms. Based on the programming model, we develop the step interleaving technique to hide memory access latency by switching the executions of different…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGraph Theory and Algorithms · Caching and Content Delivery · Advanced Graph Neural Networks
