Code Evolution for Control: Synthesizing Policies via LLM-Driven Evolutionary Search

TL;DR

This paper introduces a novel approach that combines large language models with evolutionary search to synthesize interpretable, executable control policies for autonomous systems, addressing challenges of opacity and scalability in traditional methods.

Contribution

The paper presents a new method using LLM-driven evolutionary search to generate human-readable control policies, integrating foundation models with evolutionary algorithms for policy synthesis.

Findings

Produces compact, interpretable control policies

Policies can be inspected, modified, and verified

Framework demonstrates effective policy synthesis

Abstract

Designing effective control policies for autonomous systems remains a fundamental challenge, traditionally addressed through reinforcement learning or manual engineering. While reinforcement learning has achieved remarkable success, it often suffers from high sample complexity, reward shaping difficulties, and produces opaque neural network policies that are hard to interpret or verify. Manual design, on the other hand, requires substantial domain expertise and struggles to scale across diverse tasks. In this work, we demonstrate that LLM-driven evolutionary search can effectively synthesize interpretable control policies in the form of executable code. By treating policy synthesis as a code evolution problem, we harness the LLM's prior knowledge of programming patterns and control heuristics while employing evolutionary search to explore the solution space systematically. We implement our approach using EvoToolkit, a framework that seamlessly integrates LLM-driven evolution with customizable fitness evaluation. Our method iteratively evolves populations of candidate policy programs, evaluating them against task-specific objectives and selecting superior individuals for reproduction. This process yields compact, human-readable control policies that can be directly inspected, modified, and formally verified. This work highlights the potential of combining foundation models with evolutionary computation for synthesizing trustworthy control policies in autonomous systems. Code is available at https://github.com/pgg3/EvoControl.