CrossBeam: Learning to Search in Bottom-Up Program Synthesis

TL;DR

CrossBeam introduces a neural-guided search policy for bottom-up program synthesis, significantly reducing search space and improving efficiency in string manipulation and logic programming tasks.

Contribution

It presents a novel neural model trained on-policy to guide bottom-up synthesis, outperforming traditional combinatorial search methods.

Findings

Explores smaller search spaces than state-of-the-art methods.

Learns efficient search strategies in two different domains.

Demonstrates improved synthesis performance and scalability.

Abstract

Many approaches to program synthesis perform a search within an enormous space of programs to find one that satisfies a given specification. Prior works have used neural models to guide combinatorial search algorithms, but such approaches still explore a huge portion of the search space and quickly become intractable as the size of the desired program increases. To tame the search space blowup, we propose training a neural model to learn a hands-on search policy for bottom-up synthesis, instead of relying on a combinatorial search algorithm. Our approach, called CrossBeam, uses the neural model to choose how to combine previously-explored programs into new programs, taking into account the search history and partial program executions. Motivated by work in structured prediction on learning to search, CrossBeam is trained on-policy using data extracted from its own bottom-up searches on training tasks. We evaluate CrossBeam in two very different domains, string manipulation and logic programming. We observe that CrossBeam learns to search efficiently, exploring much smaller portions of the program space compared to the state-of-the-art.