Metric Program Synthesis

TL;DR

Metric program synthesis introduces a novel approach that relaxes equivalence to similarity, enabling more efficient search and repair of programs from input-output examples across multiple domains.

Contribution

It proposes a domain-agnostic synthesis method using a distance metric to cluster programs and guide repairs, improving efficiency and effectiveness over traditional equivalence-based methods.

Findings

Outperforms other domain-agnostic synthesizers in multiple domains.

Achieves results comparable to domain-specific synthesizers.

Reduces search space by clustering programs based on similarity.

Abstract

We present a new domain-agnostic synthesis technique for generating programs from input-output examples. Our method, called metric program synthesis, relaxes the well-known observational equivalence idea (used widely in bottom-up enumerative synthesis) into a weaker notion of observational similarity, with the goal of reducing the search space that the synthesizer needs to explore. Our method clusters programs into equivalence classes based on a distance metric and constructs a version space that compactly represents "approximately correct" programs. Then, given a "close enough" program sampled from this version space, our approach uses a distance-guided repair algorithm to find a program that exactly matches the given input-output examples. We have implemented our proposed metric program synthesis technique in a tool called SyMetric and evaluate it in three different domains considered in prior work. Our evaluation shows that SyMetric outperforms other domain-agnostic synthesizers that use observational equivalence and that it achieves results competitive with domain-specific synthesizers that are either designed for or trained on those domains.