The Sampling-and-Learning Framework: A Statistical View of Evolutionary Algorithms

TL;DR

This paper introduces a statistical framework for analyzing evolutionary algorithms, focusing on sampling-and-learning methods, and demonstrates their potential for polynomial and super-polynomial speedups over uniform search under specific conditions.

Contribution

It presents a unified sampling-and-learning framework for EAs, providing a general analysis of query complexity and comparing SAC algorithms with uniform search.

Findings

SAC algorithms can achieve polynomial speedup under error-target independence.

Super-polynomial speedup is possible under one-side-error conditions.

The framework offers a new perspective on the power of EAs from a statistical viewpoint.

Abstract

Evolutionary algorithms (EAs), a large class of general purpose optimization algorithms inspired from the natural phenomena, are widely used in various industrial optimizations and often show excellent performance. This paper presents an attempt towards revealing their general power from a statistical view of EAs. By summarizing a large range of EAs into the sampling-and-learning framework, we show that the framework directly admits a general analysis on the probable-absolute-approximate (PAA) query complexity. We particularly focus on the framework with the learning subroutine being restricted as a binary classification, which results in the sampling-and-classification (SAC) algorithms. With the help of the learning theory, we obtain a general upper bound on the PAA query complexity of SAC algorithms. We further compare SAC algorithms with the uniform search in different situations. Under the error-target independence condition, we show that SAC algorithms can achieve polynomial speedup to the uniform search, but not super-polynomial speedup. Under the one-side-error condition, we show that super-polynomial speedup can be achieved. This work only touches the surface of the framework. Its power under other conditions is still open.