Are There Good Mistakes? A Theoretical Analysis of CEGIS

TL;DR

This paper provides a theoretical analysis of CEGIS, exploring how different types of counterexamples, like minimal and history-bounded, affect the synthesis power and success of program synthesis.

Contribution

It introduces a formal comparison of CEGIS variants using minimal and history-bounded counterexamples, revealing their relative synthesis powers.

Findings

MinCEGIS has the same synthesis power as CEGIS.

HCEGIS has different power than CEGIS, with no dominance.

Counterexample type influences synthesis success in infinite candidate spaces.

Abstract

Counterexample-guided inductive synthesis CEGIS is used to synthesize programs from a candidate space of programs. The technique is guaranteed to terminate and synthesize the correct program if the space of candidate programs is finite. But the technique may or may not terminate with the correct program if the candidate space of programs is infinite. In this paper, we perform a theoretical analysis of counterexample-guided inductive synthesis technique. We investigate whether the set of candidate spaces for which the correct program can be synthesized using CEGIS depends on the counterexamples used in inductive synthesis, that is, whether there are good mistakes which would increase the synthesis power. We investigate whether the use of minimal counterexamples instead of arbitrary counterexamples expands the set of candidate spaces of programs for which inductive synthesis can successfully synthesize a correct program. We consider two kinds of counterexamples: minimal counterexamples and history bounded counterexamples. The history bounded counterexample used in any iteration of CEGIS is bounded by the examples used in previous iterations of inductive synthesis. We examine the relative change in power of inductive synthesis in both cases. We show that the synthesis technique using minimal counterexamples MinCEGIS has the same synthesis power as CEGIS but the synthesis technique using history bounded counterexamples HCEGIS has different power than that of CEGIS, but none dominates the other.