AI Coding Agents Need Better Compiler Remarks

TL;DR

This paper argues that improving compiler remark interfaces with structured, precise feedback can significantly enhance AI-driven program optimization and autonomous performance engineering.

Contribution

It demonstrates that structured, precise compiler remarks improve AI agent effectiveness, highlighting the need for future compilers to expose actionable feedback.

Findings

Precise remarks increase success rate by 3.3x

Ambiguous remarks cause semantic-breaking hallucinations

Replacing ambiguous remarks with precise ones enhances model capabilities

Abstract

Modern AI agents optimize programs by refactoring source code to trigger trusted compiler transformations. This preserves program semantics and reduces source code pollution, making the program easier to maintain and portable across architectures. However, this collaborative workflow is limited by legacy compiler interfaces, which obscure analysis behind unstructured, lossy optimization remarks that have been designed for human intuition rather than machine logic. Using the TSVC benchmark, we evaluate the efficacy of existing optimization feedback. We find that while precise remarks provide actionable feedback (3.3x success rate), ambiguous remarks are actively detrimental, triggering semantic-breaking hallucinations. By replacing ambiguous remarks with precise ones, we show that structured, precise analysis information unlocks the capabilities of small models, proving that the bottleneck is the interface, not the agent. We conclude that future compilers must expose structured, actionable feedback designed specifically for the future of autonomous performance engineering.