Program Synthesis with Pragmatic Communication

TL;DR

This paper introduces a pragmatic communication model for program synthesis that improves user interaction by considering how a rational speaker would communicate specifications, leading to more effective synthesis.

Contribution

It proposes a novel inductive bias based on rational communication models, enhancing program synthesis by modeling specification communication as pragmatic reasoning.

Findings

End-user communication improves with the pragmatic synthesizer.

The approach effectively scores candidate programs based on communication rationality.

User study shows increased communication effectiveness with the pragmatic model.

Abstract

Program synthesis techniques construct or infer programs from user-provided specifications, such as input-output examples. Yet most specifications, especially those given by end-users, leave the synthesis problem radically ill-posed, because many programs may simultaneously satisfy the specification. Prior work resolves this ambiguity by using various inductive biases, such as a preference for simpler programs. This work introduces a new inductive bias derived by modeling the program synthesis task as rational communication, drawing insights from recursive reasoning models of pragmatics. Given a specification, we score a candidate program both on its consistency with the specification, and also whether a rational speaker would chose this particular specification to communicate that program. We develop efficient algorithms for such an approach when learning from input-output examples, and build a pragmatic program synthesizer over a simple grid-like layout domain. A user study finds that end-user participants communicate more effectively with the pragmatic program synthesizer over a non-pragmatic one.