Stability Improvements for Fast Matrix Multiplication

TL;DR

This paper introduces an optimization approach to discover more stable and faster matrix multiplication algorithms by finding new tensor decompositions through an augmented Lagrangian method.

Contribution

The paper presents a novel application of an augmented Lagrangian method to optimize tensor decompositions for matrix multiplication, enabling the discovery of improved algorithms.

Findings

New discrete tensor decompositions identified

Parameter families of decompositions constructed

Enhanced stability and speed in matrix multiplication algorithms

Abstract

We implement an Augmented Lagrangian method to minimize a constrained least-squares cost function designed to find polyadic decompositions of the matrix multiplication tensor. We use this method to obtain new discrete decompositions and parameter families of decompositions. Using these parametrizations, faster and more stable matrix multiplication algorithms can be discovered.