Spectral Lower Bounds on the I/O Complexity of Computation Graphs

TL;DR

This paper introduces a spectral method to derive lower bounds on the I/O complexity of arbitrary computation graphs, extending previous problem-specific bounds to a general, efficiently computable framework.

Contribution

We develop a novel spectral approach using Laplacian eigenvalues to bound I/O complexity for any computation graph, including parallel and random graphs.

Findings

Spectral bounds are tighter than existing empirical bounds.

Method applies to various graphs including TSP and FFT.

Spectral bounds are efficiently computable via power iteration.

Abstract

We consider the problem of finding lower bounds on the I/O complexity of arbitrary computations in a two level memory hierarchy. Executions of complex computations can be formalized as an evaluation order over the underlying computation graph. However, prior methods for finding I/O lower bounds leverage the graph structures for specific problems (e.g matrix multiplication) which cannot be applied to arbitrary graphs. In this paper, we first present a novel method to bound the I/O of any computation graph using the first few eigenvalues of the graph's Laplacian. We further extend this bound to the parallel setting. This spectral bound is not only efficiently computable by power iteration, but can also be computed in closed form for graphs with known spectra. We apply our spectral method to compute closed-form analytical bounds on two computation graphs (the Bellman-Held-Karp algorithm for the traveling salesman problem and the Fast Fourier Transform), as well as provide a probabilistic bound for random Erdos Renyi graphs. We empirically validate our bound on four computation graphs, and find that our method provides tighter bounds than current empirical methods and behaves similarly to previously published I/O bounds.