Automatic Generation of Efficient Linear Algebra Programs

TL;DR

This paper introduces Linnea, a code generator that translates high-level linear algebra descriptions into efficient kernel sequences, significantly outperforming existing high-level languages and libraries in speed.

Contribution

Linnea is a novel code generator that optimizes the translation from high-level linear algebra problems to efficient kernel sequences, improving performance.

Findings

Linnea outperforms Matlab, Julia, Eigen, and Armadillo in 25 tested problems.

Speedups of up to and exceeding 10x were achieved.

Generated code is consistently more efficient than existing high-level solutions.

Abstract

The level of abstraction at which application experts reason about linear algebra computations and the level of abstraction used by developers of high-performance numerical linear algebra libraries do not match. The former is conveniently captured by high-level languages and libraries such as Matlab and Eigen, while the latter expresses the kernels included in the BLAS and LAPACK libraries. Unfortunately, the translation from a high-level computation to an efficient sequence of kernels is a task, far from trivial, that requires extensive knowledge of both linear algebra and high-performance computing. Internally, almost all high-level languages and libraries use efficient kernels; however, the translation algorithms are too simplistic and thus lead to a suboptimal use of said kernels, with significant performance losses. In order to both achieve the productivity that comes with high-level languages, and make use of the efficiency of low level kernels, we are developing Linnea, a code generator for linear algebra problems. As input, Linnea takes a high-level description of a linear algebra problem and produces as output an efficient sequence of calls to high-performance kernels. In 25 application problems, the code generated by Linnea always outperforms Matlab, Julia, Eigen and Armadillo, with speedups up to and exceeding 10x.