Magellan: Autonomous Discovery of Novel Compiler Optimization Heuristics with AlphaEvolve

TL;DR

Magellan is an innovative framework that autonomously evolves compiler heuristics using AI techniques, significantly improving optimization performance and reducing manual engineering effort across multiple compiler tasks.

Contribution

It introduces a novel AI-driven approach combining LLMs and evolutionary search to automatically generate effective compiler heuristics.

Findings

Outperforms manual heuristics in LLVM inlining and register allocation.

Synthesizes compact, effective heuristics that match or surpass expert-designed policies.

Demonstrates portability and reduced engineering effort on XLA problems.

Abstract

Modern compilers rely on hand-crafted heuristics to guide optimization passes. These human-designed rules often struggle to adapt to the complexity of modern software and hardware and lead to high maintenance burden. To address this challenge, we present Magellan, an agentic framework that evolves the compiler pass itself by synthesizing executable C++ decision logic. Magellan couples an LLM coding agent with evolutionary search and autotuning in a closed loop of generation, evaluation on user-provided macro-benchmarks, and refinement, producing compact heuristics that integrate directly into existing compilers. Across several production optimization tasks, Magellan discovers policies that match or surpass expert baselines. In LLVM function inlining, Magellan synthesizes new heuristics that outperform decades of manual engineering for both binary-size reduction and end-to-end performance. In register allocation, it learns a concise priority rule for live-range processing that matches intricate human-designed policies on a large-scale workload. We also report preliminary results on XLA problems, demonstrating portability beyond LLVM with reduced engineering effort.