Generational Computation Reduction in Informal Counterexample-Driven Genetic Programming

TL;DR

This paper introduces informal CDGP, a method that uses user-provided data instead of formal constraints in genetic programming, leading to faster solutions and improved success rates in software synthesis tasks.

Contribution

It extends counterexample-driven genetic programming to operate solely with user data, introduces two new variants, and demonstrates performance improvements over standard GP.

Findings

Informal CDGP finds solutions faster than standard GP.

One variant of informal CDGP yields more successful runs.

Adding counterexamples improves training effectiveness.

Abstract

Counterexample-driven genetic programming (CDGP) uses specifications provided as formal constraints to generate the training cases used to evaluate evolving programs. It has also been extended to combine formal constraints and user-provided training data to solve symbolic regression problems. Here we show how the ideas underlying CDGP can also be applied using only user-provided training data, without formal specifications. We demonstrate the application of this method, called ``informal CDGP,'' to software synthesis problems. Our results show that informal CDGP finds solutions faster (i.e. with fewer program executions) than standard GP. Additionally, we propose two new variants to informal CDGP, and find that one produces significantly more successful runs on about half of the tested problems. Finally, we study whether the addition of counterexample training cases to the training set is useful by comparing informal CDGP to using a static subsample of the training set, and find that the addition of counterexamples significantly improves performance.