Structured mutation inspired by evolutionary theory enriches population performance and diversity

TL;DR

This paper introduces Facilitated Mutation, a biologically inspired variation operator for genetic programming, which enhances neural network optimizer evolution by improving accuracy, diversity, and efficiency.

Contribution

The paper proposes Facilitated Mutation (FM), a novel mutation method inspired by evolutionary biology, that improves performance and diversity in genetic programming for neural network optimization.

Findings

FMX improves test accuracy by 0.48% on average

FM increases solution diversity by 400 high-quality behaviors per run

FM and FMX reduce the number of fitness evaluations in some cases

Abstract

Grammar-Guided Genetic Programming (GGGP) employs a variety of insights from evolutionary theory to autonomously design solutions for a given task. Recent insights from evolutionary biology can lead to further improvements in GGGP algorithms. In this paper, we apply principles from the theory of Facilitated Variation and knowledge about heterogeneous mutation rates and mutation effects to improve the variation operators. We term this new method of variation Facilitated Mutation (FM). We test FM performance on the evolution of neural network optimizers for image classification, a relevant task in evolutionary computation, with important implications for the field of machine learning. We compare FM and FM combined with crossover (FMX) against a typical mutation regime to assess the benefits of the approach. We find that FMX in particular provides statistical improvements in key metrics, creating a superior optimizer overall (+0.48\% average test accuracy), improving the average quality of solutions (+50\% average population fitness), and discovering more diverse high-quality behaviors (+400 high-quality solutions discovered per run on average). Additionally, FM and FMX can reduce the number of fitness evaluations in an evolutionary run, reducing computational costs in some scenarios.