AlgoPilot: Fully Autonomous Program Synthesis Without Human-Written Programs

TL;DR

AlgoPilot introduces a fully autonomous program synthesis method that generates algorithms from scratch using reinforcement learning guided by a trajectory language model, without relying on human-written programs or prior knowledge.

Contribution

It presents a novel RL-based framework guided by a TLM trained on random functions, enabling the discovery of classical algorithms like Bubble Sort without prior algorithmic knowledge.

Findings

Successfully generated interpretable sorting algorithms

Demonstrated ability to synthesize classical algorithms from scratch

Established a new paradigm for autonomous algorithm discovery

Abstract

Program synthesis has traditionally relied on human-provided specifications, examples, or prior knowledge to generate functional algorithms. Existing methods either emulate human-written algorithms or solve specific tasks without generating reusable programmatic logic, limiting their ability to create novel algorithms. We introduce AlgoPilot, a groundbreaking approach for fully automated program synthesis without human-written programs or trajectories. AlgoPilot leverages reinforcement learning (RL) guided by a Trajectory Language Model (TLM) to synthesize algorithms from scratch. The TLM, trained on trajectories generated by random Python functions, serves as a soft constraint during the RL process, aligning generated sequences with patterns likely to represent valid algorithms. Using sorting as a test case, AlgoPilot demonstrates its ability to generate trajectories that are interpretable as classical algorithms, such as Bubble Sort, while operating without prior algorithmic knowledge. This work establishes a new paradigm for algorithm discovery and lays the groundwork for future advancements in autonomous program synthesis.