Reinforcement Learning from Compiler and Language Server Feedback

TL;DR

This paper introduces RLCSF, a framework that leverages compiler and language server feedback to improve coding agents through reinforcement learning, with a CLI tool for robust interaction and supervision.

Contribution

It presents RLCSF and Lanser-CLI, novel tools that utilize compiler diagnostics and language server signals for reinforcement learning in coding agents, ensuring deterministic and replayable supervision.

Findings

RLCSF treats tool interactions as transitions with shaped rewards.

Lanser-CLI converts ephemeral sessions into replayable bundles.

The framework guarantees non-negative rewards for componentwise improvements.

Abstract

Coding agents fail when text-level guesses outrun program facts: they hallucinate APIs, drift to the wrong symbol, and apply edits without evidence that the workspace remains valid. Compilers, type checkers, and language servers already compute the missing supervision signal, in the form of diagnostics, symbol resolution, type information, references, and refactoring preconditions, but expose it through interfaces designed for human-driven IDEs rather than learning loops. We introduce Reinforcement Learning from Compiler and Language Server Feedback (RLCSF) together with Lanser-CLI, a CLI-first orchestration layer that exposes this signal to agents and CI. RLCSF treats each tool interaction as a transition and computes a shaped process reward from deterministic changes in diagnostics, selector confidence, and edit safety. Lanser-CLI, in turn, converts ephemeral LSP sessions into replayable Analysis Bundles with pinned environment metadata and stable content hashes. Its core mechanisms are robust selectors that go beyond file:line:col, deterministic bundle normalization, preview-first guarded mutations, and a reward functional whose potential-based component is replayable under frozen snapshots. We formalize determinism for canonical bundles and prove that componentwise-improving transitions receive non-negative reward in the undiscounted setting. Together, these pieces yield a practical substrate for process supervision of coding agents.