Recent Developments in Program Synthesis with Evolutionary Algorithms

TL;DR

This paper reviews recent advances in program synthesis using evolutionary algorithms, analyzing their performance on benchmark problems and highlighting challenges with complex mappings.

Contribution

It provides an in-depth analysis of key evolutionary program synthesis approaches and discusses their strengths and limitations on various problem types.

Findings

Stack-based, grammar-guided, and linear genetic programming perform well on simple problems.

Performance declines on complex problems involving sub-problems or recursion.

Future research should evaluate solution processes, not just outputs.

Abstract

The automatic generation of computer programs is one of the main applications with practical relevance in the field of evolutionary computation. With program synthesis techniques not only software developers could be supported in their everyday work but even users without any programming knowledge could be empowered to automate repetitive tasks and implement their own new functionality. In recent years, many novel program synthesis approaches based on evolutionary algorithms have been proposed and evaluated on common benchmark problems. Therefore, we identify in this work the relevant evolutionary program synthesis approaches and provide an in-depth analysis of their performance. The most influential approaches we identify are stack-based, grammar-guided, as well as linear genetic programming. Further, we find that these approaches perform well on benchmark problems if there is a simple mapping from the given input to the correct output. On problems where this mapping is complex, e.g., if the problem consists of several sub-problems or requires iteration/recursion for a correct solution, results tend to be worse. Consequently, for future work, we encourage researchers not only to use a program's output for assessing the quality of a solution but also the way towards a solution (e.g., correctly solved sub-problems).