KLARAPTOR: A Tool for Dynamically Finding Optimal Kernel Launch Parameters Targeting CUDA Programs

TL;DR

KLARAPTOR is a tool that dynamically finds optimal CUDA kernel launch parameters by combining static compile-time analysis with runtime performance prediction, improving GPU program efficiency.

Contribution

The paper introduces a novel technique that builds a rational program at compile time to dynamically optimize CUDA kernel launch parameters based on performance models.

Findings

Successfully implemented on CUDA GPU kernels.

Achieved improved kernel performance through dynamic parameter tuning.

Demonstrated applicability to parallel programs and hardware parameters.

Abstract

In this paper we present KLARAPTOR (Kernel LAunch parameters RAtional Program estimaTOR), a new tool built on top of the LLVM Pass Framework and NVIDIA CUPTI API to dynamically determine the optimal values of kernel launch parameters of a CUDA program P. To be precise, we describe a novel technique to statically build (at the compile time of P) a so-called rational program R. Using a performance prediction model, and knowing particular data and hardware parameters of P at runtime, the program R can automatically and dynamically determine the values of launch parameters of P that will yield optimal performance. Our technique can be applied to parallel programs in general, as well as to generic performance prediction models which account for program and hardware parameters. We are particularly interested in programs targeting manycore accelerators. We have implemented and successfully tested our technique in the context of GPU kernels written in CUDA using the MWP-CWP performance prediction model.