The Anatomy of Large-Scale Distributed Graph Algorithms

TL;DR

This paper examines the challenges of performance analysis in large-scale distributed graph algorithms, emphasizing the importance of understanding runtime properties and proposing recommendations for better experimental reporting.

Contribution

It highlights the critical role of runtime analysis in DGA performance studies and offers initial guidelines for more comprehensive experimental descriptions.

Findings

Runtime significantly influences DGA performance interpretation

Current research often overlooks detailed runtime characterization

Recommendations aim to improve experimental transparency and reproducibility

Abstract

The increasing complexity of the software/hardware stack of modern supercomputers results in explosion of parameters. The performance analysis becomes a truly experimental science, even more challenging in the presence of massive irregularity and data dependency. We analyze how the existing body of research handles the experimental aspect in the context of distributed graph algorithms (DGAs). We distinguish algorithm-level contributions, often prioritized by authors, from runtime-level concerns that are harder to place. We show that the runtime is such an integral part of DGAs that experimental results are difficult to interpret and extrapolate without understanding the properties of the runtime used. We argue that in order to gain understanding about the impact of runtimes, more information needs to be gathered. To begin this process, we provide an initial set of recommendations for describing DGA results based on our analysis of the current state of the field.