Just-in-Time Learning for Bottom-Up Enumerative Synthesis

TL;DR

This paper introduces a just-in-time learning approach for program synthesis that learns from partial solutions during synthesis, improving efficiency without sacrificing solution quality.

Contribution

It proposes a novel guided bottom-up search algorithm and a tool called Probe that enhances synthesis performance by bootstrap learning during the process.

Findings

Significant performance improvements over unguided search.

Outperforms state-of-the-art probability-guided synthesizers.

Generated programs are nearly as concise as shortest solutions.

Abstract

A key challenge in program synthesis is the astronomical size of the search space the synthesizer has to explore. In response to this challenge, recent work proposed to guide synthesis using learned probabilistic models. Obtaining such a model, however, might be infeasible for a problem domain where no high-quality training data is available. In this work we introduce an alternative approach to guided program synthesis: instead of training a model ahead of time we show how to bootstrap one just in time, during synthesis, by learning from partial solutions encountered along the way. To make the best use of the model, we also propose a new program enumeration algorithm we dub guided bottom-up search, which extends the efficient bottom-up search with guidance from probabilistic models. We implement this approach in a tool called Probe, which targets problems in the popular syntax-guided synthesis (SyGuS) format. We evaluate Probe on benchmarks from the literature and show that it achieves significant performance gains both over unguided bottom-up search and over a state-of-the-art probability-guided synthesizer, which had been trained on a corpus of existing solutions. Moreover, we show that these performance gains do not come at the cost of solution quality: programs generated by Probe are only slightly more verbose than the shortest solutions and perform no unnecessary case-splitting.