REASONING COMPILER: LLM-Guided Optimizations for Efficient Model Serving

TL;DR

This paper introduces REASONING COMPILER, a novel framework that uses large language models and structured Monte Carlo tree search to improve compiler optimizations for neural model serving, achieving faster results with fewer samples.

Contribution

It presents a new LLM-guided, context-aware compiler optimization method that enhances sample efficiency without retraining, outperforming existing neural compiler techniques.

Findings

Achieves significant speedups with fewer samples.

Leverages LLM reasoning for context-aware optimization.

Outperforms existing neural compiler methods.

Abstract

While model serving has unlocked unprecedented capabilities, the high cost of serving large-scale models continues to be a significant barrier to widespread accessibility and rapid innovation. Compiler optimizations have long driven substantial performance improvements, but existing compilers struggle with neural workloads due to the exponentially large and highly interdependent space of possible transformations. Although existing stochastic search techniques can be effective, they are often sample-inefficient and fail to leverage the structural context underlying compilation decisions. We set out to investigate the research question of whether reasoning with large language models (LLMs), without any retraining, can leverage the context-aware decision space of compiler optimizations to significantly improve sample efficiency. To that end, we introduce a novel compilation framework (dubbed REASONING COMPILER) that formulates optimization as a sequential, context-aware decision process guided by a large language model and structured Monte Carlo tree search (MCTS). The LLM acts as a proposal mechanism, suggesting hardware-informed transformations that reflect the current program state and accumulated performance feedback. MCTS incorporates the LLM-generated proposals to balance exploration and exploitation, facilitating a structured, context-sensitive traversal of the expansive compiler optimization space. By achieving substantial speedups with markedly fewer samples than leading neural compilers, our approach demonstrates the potential of LLM-guided reasoning to transform the landscape of compiler optimization.