A probabilistic and multi-objective analysis of lexicase selection and epsilon-lexicase selection

TL;DR

This paper provides a theoretical analysis of lexicase and epsilon-lexicase selection methods, explaining their performance and limitations, and introduces variants that improve selection in continuous error spaces, validated on regression problems.

Contribution

It derives an analytical formula for selection probabilities and relates lexicase to many-objective optimization, proposing epsilon-lexicase variants that enhance performance in continuous spaces.

Findings

Lexicase selection performs better on certain population sizes and training case counts.

Epsilon-lexicase outperforms diversity strategies on regression tasks.

Theoretical insights explain performance issues in continuous error spaces.

Abstract

Lexicase selection is a parent selection method that considers training cases individually, rather than in aggregate, when performing parent selection. Whereas previous work has demonstrated the ability of lexicase selection to solve difficult problems in program synthesis and symbolic regression, the central goal of this paper is to develop the theoretical underpinnings that explain its performance. To this end, we derive an analytical formula that gives the expected probabilities of selection under lexicase selection, given a population and its behavior. In addition, we expand upon the relation of lexicase selection to many-objective optimization methods to describe the behavior of lexicase selection, which is to select individuals on the boundaries of Pareto fronts in high-dimensional space. We show analytically why lexicase selection performs more poorly for certain sizes of population and training cases, and show why it has been shown to perform more poorly in continuous error spaces. To address this last concern, we propose new variants of epsilon-lexicase selection, a method that modifies the pass condition in lexicase selection to allow near-elite individuals to pass cases, thereby improving selection performance with continuous errors. We show that epsilon-lexicase outperforms several diversity-maintenance strategies on a number of real-world and synthetic regression problems.