Optimal Neural Program Synthesis from Multimodal Specifications

TL;DR

This paper introduces an optimal neural program synthesis method that combines natural language and input-output examples to generate programs satisfying user constraints, using a top-down neural model and search optimization.

Contribution

It presents a novel top-down neural model for multimodal program synthesis that efficiently searches for model-optimal programs satisfying complex user constraints.

Findings

Outperforms prior methods in accuracy and efficiency

Finds model-optimal programs more frequently

Leverages automated program analysis for pruning

Abstract

Multimodal program synthesis, which leverages different types of user input to synthesize a desired program, is an attractive way to scale program synthesis to challenging settings; however, it requires integrating noisy signals from the user, like natural language, with hard constraints on the program's behavior. This paper proposes an optimal neural synthesis approach where the goal is to find a program that satisfies user-provided constraints while also maximizing the program's score with respect to a neural model. Specifically, we focus on multimodal synthesis tasks in which the user intent is expressed using a combination of natural language (NL) and input-output examples. At the core of our method is a top-down recurrent neural model that places distributions over abstract syntax trees conditioned on the NL input. This model not only allows for efficient search over the space of syntactically valid programs, but it allows us to leverage automated program analysis techniques for pruning the search space based on infeasibility of partial programs with respect to the user's constraints. The experimental results on a multimodal synthesis dataset (StructuredRegex) show that our method substantially outperforms prior state-of-the-art techniques in terms of accuracy and efficiency, and finds model-optimal programs more frequently.