The GAP Benchmark Suite

TL;DR

The GAP Benchmark Suite provides a standardized set of graph processing benchmarks with optimized baselines to facilitate fair comparison and evaluation of graph algorithms, implementations, and hardware platforms.

Contribution

It introduces a comprehensive benchmark suite with optimized baselines, standardized inputs, and evaluation methodologies for graph processing research.

Findings

Baseline implementations are representative of state-of-the-art performance.

The benchmark suite enables consistent comparison across different systems.

It supports various use cases including algorithm development and platform evaluation.

Abstract

We present a graph processing benchmark suite with the goal of helping to standardize graph processing evaluations. Fewer differences between graph processing evaluations will make it easier to compare different research efforts and quantify improvements. The benchmark not only specifies graph kernels, input graphs, and evaluation methodologies, but it also provides optimized baseline implementations. These baseline implementations are representative of state-of-the-art performance, and thus new contributions should outperform them to demonstrate an improvement. The input graphs are sized appropriately for shared memory platforms, but any implementation on any platform that conforms to the benchmark's specifications could be compared. This benchmark suite can be used in a variety of settings. Graph framework developers can demonstrate the generality of their programming model by implementing all of the benchmark's kernels and delivering competitive performance on all of the benchmark's graphs. Algorithm designers can use the input graphs and the baseline implementations to demonstrate their contribution. Platform designers and performance analysts can use the suite as a workload representative of graph processing.