Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI

TL;DR

SOAR introduces a self-improving evolutionary approach that iteratively enhances language models for program synthesis, significantly improving performance on the ARC-AGI benchmark through combined search and fine-tuning.

Contribution

The paper presents SOAR, a novel method integrating evolutionary search with self-supervised fine-tuning of language models for improved program synthesis.

Findings

Achieves 52% solve rate on ARC-AGI test set.

Demonstrates effective transfer between search and refinement tasks.

Enables model improvement through iterative self-supervised learning.

Abstract

Many program synthesis tasks prove too challenging for even state-of-the-art language models to solve in single attempts. Search-based evolutionary methods offer a promising alternative by exploring solution spaces iteratively, but their effectiveness remain limited by the fixed capabilities of the underlying generative model. We propose SOAR, a method that learns program synthesis by integrating language models into a self-improving evolutionary loop. SOAR alternates between (1) an evolutionary search that uses an LLM to sample and refine candidate solutions, and (2) a hindsight learning phase that converts search attempts into valid problem-solution pairs used to fine-tune the LLM's sampling and refinement capabilities\, -- \,enabling increasingly effective search in subsequent iterations. On the challenging ARC-AGI benchmark, SOAR achieves significant performance gains across model scales and iterations, leveraging positive transfer between the sampling and refinement finetuning tasks. These improvements carry over to test-time adaptation, enabling SOAR to solve 52\% of the public test set. Our code is open-sourced at: https://github.com/flowersteam/SOAR