KEET: Explaining Performance of GPU Kernels Using LLM Agents

TL;DR

KEET leverages LLMs to interpret GPU kernel performance profiles, providing natural language explanations and optimization suggestions to aid developers in performance tuning.

Contribution

Introduces KEET, an LLM-based framework that interprets Nsight Compute profiles to generate explanations and optimization advice for GPU kernels.

Findings

Generated explanations improve code optimization quality.

Tool interprets large sets of profiles for better suggestions.

Explanations enhance understanding of GPU kernel performance.

Abstract

Performance profiles of GPU kernels generated by tools such as Nsight Compute are rich in detail but are often challenging to interpret. To achieve the best performance possible on a given GPU architecture, kernel developers need to spend significant time analyzing and comparing profiles in the tool's graphical interface to identify and understand kernel performance bottlenecks. Large Language Models (LLMs) have shown promise in understanding complex data and generating natural language explanations. In this paper, we propose the Kernel Execution Explanation Toolkit (KEET), an LLM-based agentic framework for interpreting Nsight Compute profiles to generate useful and data-grounded natural language explanations of performance issues in GPU kernels, and suggestions for optimizations. We evaluate \toolname using several CUDA kernels of varying complexity on NVIDIA H100 GPUs. We find that the generated explanations, when provided as context, improve the quality of LLM code optimization and multiple-choice question answering in downstream tasks. We further demonstrate that the tool can be used to interpret performance data from large sets of profiles to improve the quality of optimization suggestions.