Fine-Grained Computation in 3-Space: Matrix Multiplication and Graph Problems

TL;DR

This paper establishes optimal algorithms and bounds for matrix multiplication and graph problems in 3-dimensional space, considering physical constraints like volume and communication speed, revealing fundamental limits and advantages of 3D computation.

Contribution

It introduces optimal fine-grained algorithms for matrix multiplication and graph problems in 3-space, considering physical constraints, and compares their efficiency to lower bounds and 2D approaches.

Findings

Matrix multiplication in 3D space takes at least Ω(n^{2/3}) time.

Achieves faster matrix multiplication in 3D than parallel Strassen's algorithm.

Shows 2D space algorithms are less efficient for these problems.

Abstract

Obeying constraints imposed by classical physics, we give optimal fine-grained algorithms for matrix multiplication and problems involving graphs and mazes, where all calculations are done in 3-dimensional space. We assume that whatever the technology is, a bit requires a minimum volume and communication travels at a bounded speed. These imply that multiplying $n \times n$ matrices takes $\Omega(n^{2/3})$ time, and we show that this can be achieved by a fine-grained 3-d mesh of $n^2$ processors. While the constants are impractically large, this is asymptotically faster than parallel implementations of Strassen's algorithm, while the lower bound shows that some claims about parallelizing faster serial algorithms are impossible in 3-space. If the matrices are not over a ring then multiplication can be done in $\Theta(n^{3/4})$ time by expanding to a mesh larger than the input. In 2-d (such as the surface of a chip) this approach is useless and $\Theta(n)$ systolic algorithms are optimal even when the matrices are over a ring. Similarly, for path and maze problems there are approaches useful in 3-d but not 2-d.