Limits on All Known (and Some Unknown) Approaches to Matrix Multiplication

TL;DR

This paper investigates the fundamental limits of existing matrix multiplication algorithms, introducing new methods to prove lower bounds on the exponent of matrix multiplication, and demonstrating inherent barriers in current approaches.

Contribution

It defines the Solar and Galactic methods as generalizations of known techniques and proves universal bounds on their effectiveness in improving matrix multiplication complexity.

Findings

A universal constant >2 limits the effectiveness of the Galactic method.

Certain tensor classes cannot surpass specific bounds on  using these methods.

Previous techniques are insufficient to establish these fundamental limits.

Abstract

We study the known techniques for designing Matrix Multiplication algorithms. The two main approaches are the Laser method of Strassen, and the Group theoretic approach of Cohn and Umans. We define a generalization based on zeroing outs which subsumes these two approaches, which we call the Solar method, and an even more general method based on monomial degenerations, which we call the Galactic method. We then design a suite of techniques for proving lower bounds on the value of $\omega$, the exponent of matrix multiplication, which can be achieved by algorithms using many tensors $T$ and the Galactic method. Some of our techniques exploit `local' properties of $T$, like finding a sub-tensor of $T$ which is so `weak' that $T$ itself couldn't be used to achieve a good bound on $\omega$, while others exploit `global' properties, like $T$ being a monomial degeneration of the structural tensor of a group algebra. Our main result is that there is a universal constant $\ell>2$ such that a large class of tensors generalizing the Coppersmith-Winograd tensor $CW_q$ cannot be used within the Galactic method to show a bound on $\omega$ better than $\ell$, for any $q$. We give evidence that previous lower-bounding techniques were not strong enough to show this. We also prove a number of complementary results along the way, including that for any group $G$, the structural tensor of $\mathbb{C}[G]$ can be used to recover the best bound on $\omega$ which the Coppersmith-Winograd approach gets using $CW_{|G|-2}$ as long as the asymptotic rank of the structural tensor is not too large.