Optimal Multi-Pass Lower Bounds for MST in Dynamic Streams

TL;DR

This paper proves that the existing multi-pass streaming algorithm for minimum spanning trees (MSTs) in dynamic graph streams is optimal, establishing tight space-pass trade-offs through new communication complexity lower bounds.

Contribution

It introduces the first tight lower bounds for multi-pass dynamic streaming algorithms for MST, matching the performance of the AGM algorithm and explaining its optimality.

Findings

Any p-pass dynamic streaming algorithm for MST requires n^{1+Ω(1/p)} space.

Semi-streaming algorithms need at least Ω(log n / log log n) passes.

New communication complexity lower bounds and composition theorems are developed.

Abstract

The seminal work of Ahn, Guha, and McGregor in 2012 introduced the graph sketching technique and used it to present the first streaming algorithms for various graph problems over dynamic streams with both insertions and deletions of edges. This includes algorithms for cut sparsification, spanners, matchings, and minimum spanning trees (MSTs). These results have since been improved or generalized in various directions, leading to a vastly rich host of efficient algorithms for processing dynamic graph streams. A curious omission from the list of improvements has been the MST problem. The best algorithm for this problem remains the original AGM algorithm that for every integer $p \geq 1$, uses $n^{1+O(1/p)}$ space in $p$ passes on $n$-vertex graphs, and thus achieves the desired semi-streaming space of $\tilde{O}(n)$ at a relatively high cost of $O(\frac{\log{n}}{\log\log{n}})$ passes. On the other hand, no lower bounds beyond a folklore one-pass lower bound is known for this problem. We provide a simple explanation for this lack of improvements: The AGM algorithm for MSTs is optimal for the entire range of its number of passes! We prove that even for the simplest decision version of the problem -- deciding whether the weight of MSTs is at least a given threshold or not -- any $p$-pass dynamic streaming algorithm requires $n^{1+\Omega(1/p)}$ space. This implies that semi-streaming algorithms do need $\Omega(\frac{\log{n}}{\log\log{n}})$ passes. Our result relies on proving new multi-round communication complexity lower bounds for a variant of the universal relation problem that has been instrumental in proving prior lower bounds for single-pass dynamic streaming algorithms. The proof also involves proving new composition theorems in communication complexity, including majority lemmas and multi-party XOR lemmas, via information complexity approaches.