KAPSO: A Knowledge-grounded framework for Autonomous Program Synthesis and Optimization

TL;DR

KAPSO is a modular framework that combines knowledge integration, experiment management, and iterative learning to improve autonomous program synthesis and optimization over long horizons, addressing common failures in coding agents.

Contribution

KAPSO introduces a novel integrated system that enhances long-horizon program synthesis by combining a git-based experimentation engine, a structured knowledge system, and a cognitive memory layer.

Findings

Achieved improved performance on MLE-Bench and ALE-Bench tasks.

Demonstrated effective reuse of knowledge and lessons to accelerate convergence.

Reduced repeated errors and enhanced reproducibility in program synthesis.

Abstract

We introduce KAPSO, a modular framework for autonomous program synthesis and optimization. Given a natural language goal and an evaluation method, KAPSO iteratively performs ideation, code synthesis and editing, execution, evaluation, and learning to improve a runnable artifact toward measurable objectives. Rather than treating synthesis as the endpoint, KAPSO uses synthesis as an operator within a long-horizon optimization loop, where progress is defined by evaluator outcomes. KAPSO targets long-horizon failures common in coding agents, including lost experimental state, brittle debugging, and weak reuse of domain expertise, by integrating three tightly coupled components. First, a git-native experimentation engine isolates each attempt as a branch, producing reproducible artifacts and preserving provenance across iterations. Second, a knowledge system ingests heterogeneous sources, including repositories, internal playbooks, and curated external resources such as documentation, scientific papers, and web search results, and organizes them into a structured representation that supports retrieval over workflows, implementations, and environment constraints. Third, a cognitive memory layer coordinates retrieval and maintains an episodic store of reusable lessons distilled from experiment traces (run logs, diffs, and evaluator feedback), reducing repeated error modes and accelerating convergence. We evaluated KAPSO on MLE-Bench (Kaggle-style ML competitions) and ALE-Bench (AtCoder heuristic optimization), and report end-to-end performance. Code Available at: https://github.com/Leeroo-AI/kapso