Program Synthesis using Natural Language

TL;DR

This paper introduces a versatile framework for building program synthesizers that convert natural language instructions into domain-specific programs, demonstrating high accuracy across multiple application domains.

Contribution

It presents a novel, generalizable approach for creating NL-to-DSL synthesizers using NLP features and training data, applicable to various domains.

Findings

Achieved 80% top-1 accuracy on 1200+ descriptions

Achieved 90% top-3 accuracy on 1200+ descriptions

Framework successfully applied to three diverse domains

Abstract

Interacting with computers is a ubiquitous activity for millions of people. Repetitive or specialized tasks often require creation of small, often one-off, programs. End-users struggle with learning and using the myriad of domain-specific languages (DSLs) to effectively accomplish these tasks. We present a general framework for constructing program synthesizers that take natural language (NL) inputs and produce expressions in a target DSL. The framework takes as input a DSL definition and training data consisting of NL/DSL pairs. From these it constructs a synthesizer by learning optimal weights and classifiers (using NLP features) that rank the outputs of a keyword-programming based translation. We applied our framework to three domains: repetitive text editing, an intelligent tutoring system, and flight information queries. On 1200+ English descriptions, the respective synthesizers rank the desired program as the top-1 and top-3 for 80% and 90% descriptions respectively.