OptiML: An End-to-End Framework for Program Synthesis and CUDA Kernel Optimization

TL;DR

OptiML is an end-to-end framework that synthesizes and optimizes CUDA kernels by combining language models, search, and verification to achieve high performance guided by hardware feedback.

Contribution

This work introduces OptiML, a novel framework that systematically explores and verifies CUDA kernel optimizations using LLMs, search algorithms, and profiler feedback.

Findings

OptiML outperforms baseline LLM approaches in kernel performance.

It produces verified, interpretable optimization trajectories.

OptiML effectively balances synthesis and optimization for diverse CUDA kernels.

Abstract

Generating high-performance CUDA kernels remains challenging due to the need to navigate a combinatorial space of low-level transformations under noisy and expensive hardware feedback. Although large language models can synthesize functionally correct CUDA code, achieving competitive performance requires systematic exploration and verification of optimization choices. We present OptiML, an end-to-end framework that maps either natural-language intent or input CUDA code to performance-optimized CUDA kernels by formulating kernel optimization as search under verification. OptiML consists of two decoupled stages. When the input is natural language, a Mixture-of-Thoughts generator (OptiML-G) acts as a proposal policy over kernel implementation strategies, producing an initial executable program. A search-based optimizer (OptiML-X) then refines either synthesized or user-provided kernels using Monte Carlo Tree Search over LLM-driven edits, guided by a hardware-aware reward derived from profiler feedback. Each candidate transformation is compiled, verified, and profiled with Nsight Compute, and evaluated by a composite objective that combines runtime with hardware bottleneck proxies and guardrails against regressions. We evaluate OptiML in both synthesis-and-optimize and optimization-only settings on a diverse suite of CUDA kernels. Results show that OptiML consistently discovers verified performance improvements over strong LLM baselines and produces interpretable optimization trajectories grounded in profiler evidence.