Streaming Graph Algorithms in the Massively Parallel Computation Model

TL;DR

This paper develops efficient massively parallel algorithms for processing evolving large-scale graphs with edge updates, achieving optimal resource utilization in terms of time and memory.

Contribution

It introduces MPC algorithms capable of handling dynamic graphs with minimal memory per machine and total memory, improving upon previous methods that used more space.

Findings

Algorithms process large batches of updates in constant rounds.

Achieves sublinear local memory and total memory usage.

Handles fundamental graph problems like connectivity and minimum spanning forest.

Abstract

We initiate the study of graph algorithms in the streaming setting on massive distributed and parallel systems inspired by practical data processing systems. The objective is to design algorithms that can efficiently process evolving graphs via large batches of edge insertions and deletions using as little memory as possible. We focus on the nowadays canonical model for the study of theoretical algorithms for massive networks, the Massively Parallel Computation (MPC) model. We design MPC algorithms that efficiently process evolving graphs: in a constant number of rounds they can handle large batches of edge updates for problems such as connectivity, minimum spanning forest, and approximate matching while adhering to the most restrictive memory regime, in which the local memory per machine is strongly sublinear in the number of vertices and the total memory is sublinear in the graph size. These results improve upon earlier works in this area which rely on using larger total space, proportional to the size of the processed graph. Our work demonstrates that parallel algorithms can process dynamically changing graphs with asymptotically optimal utilization of MPC resources: parallel time, local memory, and total memory, while processing large batches of edge updates.