Efficient Pragmatic Program Synthesis with Informative Specifications

TL;DR

This paper introduces an efficient pragmatic program synthesizer that models user-provided examples as a factored distribution, improving performance on natural human inputs by approximating pragmatic communication.

Contribution

It proposes a novel factored approximation for pragmatic inference in program synthesis, enabling efficient and effective synthesis from human-like examples.

Findings

Factored distribution approximates the joint distribution well with pragmatic examples.

The factored synthesizer outperforms the exact joint distribution model on human inputs.

Humans may communicate programs assuming a factored distribution.

Abstract

Providing examples is one of the most common way for end-users to interact with program synthesizers. However, program synthesis systems assume that examples consistent with the program are chosen at random, and do not exploit the fact that users choose examples pragmatically. Prior work modeled program synthesis as pragmatic communication, but required an inefficient enumeration of the entire program space. In this paper, we show that it is possible to build a program synthesizer that is both pragmatic and efficient by approximating the joint distribution of programs with a product of independent factors, and performing pragmatic inference on each factor separately. This factored distribution approximates the exact joint distribution well when the examples are given pragmatically, and is compatible with a basic neuro-symbolic program synthesis algorithm. Surprisingly, we find that the synthesizer assuming a factored approximation performs better than a synthesizer assuming an exact joint distribution when evaluated on natural human inputs. This suggests that humans may be assuming a factored distribution while communicating programs.