Hierarchical Performance Modeling for Ranking Dense Linear Algebra Algorithms

TL;DR

This paper introduces a hierarchical modeling approach to predict and rank the performance of various dense linear algebra algorithms and their variants on specific hardware and problem sizes, without executing each algorithm.

Contribution

It presents novel tools that measure, model, and predict algorithm performance, enabling optimal algorithm selection and parameter tuning for dense linear algebra operations.

Findings

Accurately predicts algorithm performance rankings.

Effectively tunes algorithmic parameters like block size.

Reduces need for extensive benchmarking.

Abstract

A large class of dense linear algebra operations, such as LU decomposition or inversion of a triangular matrix, are usually performed by blocked algorithms. For one such operation, typically, not only one but many algorithmic variants exist; depending on computing architecture, libraries and problem size, each variant attains a different performances. We propose methods and tools to rank the algorithmic variants according to their performance for a given scenario without executing them. For this purpose, we identify the routines upon which the algorithms are built. A first tool - the Sampler - measures the performance of these routines. Using the Sampler, a second tool models their performance. The generated models are then used to predict the performance of the considered algorithms. For a given scenario, these predictions allow us to correctly rank the algorithms according to their performance without executing them. With the help of the same tools, algorithmic parameters such as block-size can be optimally tuned.