Just-in-Time autotuning

TL;DR

This paper presents a just-in-time autotuning system that dynamically optimizes computational kernels at runtime, improving performance stability and accuracy after initial overhead, suitable for repeated kernel calls with similar parameters.

Contribution

It extends JIT compilation infrastructure to include runtime autotuning, enabling dynamic optimization of kernels based on execution parameters.

Findings

Overhead occurs during initial iterations but is offset by performance gains later

Optimal parameters found are stable and accurate across executions

Approach improves performance portability for repeated kernel calls

Abstract

Performance portability is a major concern on current architectures. One way to achieve it is by using autotuning. In this paper, we are presenting how we exten ded a just-in-time compilation infrastructure to introduce autotuning capabiliti es triggered at run-time. When a function is executed, the first iterations optimize it, and once the best solution has been found, it is used for subsequent calls to the function. This just-in-time autotuning infrastructure is relevant for optimizing computation kernels that will be called numerous times with similar parameters through the execution, re-optimizes kernels when they are called with other parameters, and the programmer can obtain the optimal parameters to use them for other kernels. We present an experimental performance evaluation of our approach. Compiling the code introduces an overhead on the first iterations, and this overhead is compensated for during subsequent iterations. We also determined that the optimum found seems stable and accurate.