Distributed Graph Layout for Scalable Small-world Network Analysis

TL;DR

This paper introduces DGL, a distributed graph layout strategy optimized for small-world networks, significantly enhancing the performance of various graph analytics workloads through improved memory locality and load balancing.

Contribution

The paper presents a novel, scalable graph layout method tailored for small-world networks, combining label propagation and BFS ordering to boost distributed graph processing efficiency.

Findings

DGL layout improves performance of PageRank, BFS, and shortest paths.

Significant reduction in communication and computation time observed.

Enhanced scalability and efficiency in distributed graph analytics.

Abstract

The in-memory graph layout or organization has a considerable impact on the time and energy efficiency of distributed memory graph computations. It affects memory locality, inter-task load balance, communication time, and overall memory utilization. Graph layout could refer to partitioning or replication of vertex and edge arrays, selective replication of data structures that hold meta-data, and reordering vertex and edge identifiers. In this work, we present DGL, a fast, parallel, and memory-efficient distributed graph layout strategy that is specifically designed for small-world networks (low-diameter graphs with skewed vertex degree distributions). Label propagation-based partitioning and a scalable BFS-based ordering are the main steps in the layout strategy. We show that the DGL layout can significantly improve end-to-end performance of five challenging graph analytics workloads: PageRank, a parallel subgraph enumeration program, tuned implementations of breadth-first search and single-source shortest paths, and RDF3X-MPI, a distributed SPARQL query processing engine. Using these benchmarks, we additionally offer a comprehensive analysis on how graph layout affects the performance of graph analytics with variable computation and communication characteristics.