Effective Adaptive Mutation Rates for Program Synthesis

TL;DR

This paper introduces an adaptive bandit-based mutation scheme for evolutionary algorithms in program synthesis, effectively removing the need for fixed mutation rates and outperforming traditional methods.

Contribution

It proposes a novel bandit-based adaptive mutation scheme that overcomes issues of fixed and self-adaptive mutation rates in evolutionary algorithms.

Findings

The scheme outperforms fixed mutation methods on benchmark problems.

It prevents mutation rate decay issues common in self-adaptive schemes.

Validated on software synthesis and symbolic regression tasks.

Abstract

The problem-solving performance of many evolutionary algorithms, including genetic programming systems used for program synthesis, depends on the values of hyperparameters including mutation rates. The mutation method used to produce some of the best results to date on software synthesis benchmark problems, Uniform Mutation by Addition and Deletion (UMAD), adds new genes into a genome at a predetermined rate and then deletes genes at a rate that balances the addition rate, producing no size change on average. While UMAD with a predetermined addition rate outperforms many other mutation and crossover schemes, we do not expect a single rate to be optimal across all problems or all generations within one run of an evolutionary system. However, many current adaptive mutation schemes such as self-adaptive mutation rates suffer from pathologies like the vanishing mutation rate problem, in which the mutation rate quickly decays to zero. We propose an adaptive bandit-based scheme that addresses this problem and essentially removes the need to specify a mutation rate. Although the proposed scheme itself introduces hyperparameters, we either set these to good values or ensemble them in a reasonable range. Results on software synthesis and symbolic regression problems validate the effectiveness of our approach.