NXgraph: An Efficient Graph Processing System on a Single Machine

TL;DR

NXgraph is a high-performance single-machine graph processing system that employs novel data structures and adaptive strategies to outperform existing systems on large-scale graphs, enabling fast computations like PageRank.

Contribution

The paper introduces NXgraph, featuring vertex intervals, edge sub-shards, and adaptive update strategies, achieving efficient and scalable graph processing on a single machine.

Findings

Outperforms GraphChi, TurboGraph, VENUS, and GridGraph in various benchmarks.

Completes PageRank on a 1.5 billion edge graph in 2.05 seconds.

Achieves high parallelism and data locality through novel data structures and strategies.

Abstract

Recent studies show that graph processing systems on a single machine can achieve competitive performance compared with cluster-based graph processing systems. In this paper, we present NXgraph, an efficient graph processing system on a single machine. With the abstraction of vertex intervals and edge sub-shards, we propose the Destination-Sorted Sub-Shard (DSSS) structure to store a graph. By dividing vertices and edges into intervals and sub-shards, NXgraph ensures graph data access locality and enables fine-grained scheduling. By sorting edges within each sub-shard according to their destination vertices, NXgraph reduces write conflicts among different threads and achieves a high degree of parallelism. Then, three updating strategies, i.e., Single-Phase Update (SPU), Double-Phase Update (DPU), and Mixed-Phase Update (MPU), are proposed in this paper. NXgraph can adaptively choose the fastest strategy for different graph problems according to the graph size and the available memory resources to fully utilize the memory space and reduce the amount of data transfer. All these three strategies exploit streamlined disk access pattern. Extensive experiments on three real-world graphs and five synthetic graphs show that NXgraph can outperform GraphChi, TurboGraph, VENUS, and GridGraph in various situations. Moreover, NXgraph, running on a single commodity PC, can finish an iteration of PageRank on the Twitter graph with 1.5 billion edges in 2.05 seconds; while PowerGraph, a distributed graph processing system, needs 3.6s to finish the same task.