Program Generation for Small-Scale Linear Algebra Applications

TL;DR

SLinGen is a program generation system that produces optimized C code for small-scale linear algebra applications, improving performance over traditional library-based implementations in control, signal processing, and related domains.

Contribution

The paper introduces SLinGen, a novel system that synthesizes high-performance C code from a linear algebra-inspired language for small-scale applications, surpassing existing methods.

Findings

Significant speed-ups over straightforward C implementations.

Outperforms library-based and template-based approaches.

Effective optimization for fixed-size operands in small-scale computations.

Abstract

We present SLinGen, a program generation system for linear algebra. The input to SLinGen is an application expressed mathematically in a linear-algebra-inspired language (LA) that we define. LA provides basic scalar/vector/matrix additions/multiplications and higher level operations including linear systems solvers, Cholesky and LU factorizations. The output of SLinGen is performance-optimized single-source C code, optionally vectorized with intrinsics. The target of SLinGen are small-scale computations on fixed-size operands, for which a straightforward implementation using optimized libraries (e.g., BLAS or LAPACK) is known to yield suboptimal performance (besides increasing code size and introducing dependencies), but which are crucial in control, signal processing, computer vision, and other domains. Internally, SLinGen uses synthesis and DSL-based techniques to optimize at a high level of abstraction. We benchmark our program generator on three prototypical applications: the Kalman filter, Gaussian process regression, and an L1-analysis convex solver, as well as basic routines including Cholesky factorization and solvers for the continuous-time Lyapunov and Sylvester equations. The results show significant speed-ups compared to straightforward C with Intel icc and clang with a polyhedral optimizer, as well as library-based and template-based implementations.