The Matrix Chain Algorithm to Compile Linear Algebra Expressions

TL;DR

This paper extends the classic matrix chain algorithm to handle complex linear algebra expressions involving various kernels, properties, and cost functions, aiming to automate and optimize the compilation of such expressions.

Contribution

It introduces generalized algorithms for mapping complex linear algebra expressions to computational kernels, addressing practical challenges in compiler automation.

Findings

Developed algorithms for kernel-based expression mapping.

Addressed optimization beyond scalar operation minimization.

Provided solutions for numerical stability and execution time.

Abstract

The matrix chain problem consists in finding the parenthesization of a matrix product $M := A_1 A_2 \cdots A_n$ that minimizes the number of scalar operations. In practical applications, however, one frequently encounters more complicated scenarios, where expressions involve transposition, inversion, matrices with given properties, and sequences. The computation of such expressions makes use of a set of computational kernels that offer functionality well beyond the simple matrix product. The challenge then shifts from finding an optimal parenthesization to finding an optimal mapping of the input expression to the available kernels. Furthermore, it is often the case that a solution based on the minimization of scalar operations does not result in the optimal solution in terms of execution time, and/or might be numerically unstable. In this paper, we introduce a number of generalizations of the matrix chain problem--including kernels, properties, sequences, and cost functions--and present corresponding algorithmic solutions. The motivation for this work comes from the fact that--despite great advances in the development of compilers--the task of mapping linear algebra problems to optimized kernels is still to be done manually. In order to relieve the user from this complex task, new techniques for the compilation of linear algebra expressions have to be developed.