Generalization and Completeness of Stochastic Local Search Algorithms

TL;DR

This paper introduces a formal model for Stochastic Local Search algorithms, demonstrating their Turing-completeness and implications for undecidability in analyzing their properties.

Contribution

It provides a unified formal framework for SLS heuristics and proves their Turing-completeness, revealing fundamental limits of analyzing these algorithms.

Findings

Unified formal model for SLS heuristics

Proof of Turing-completeness for SLS algorithms

Implication of undecidability in analyzing SLS properties

Abstract

We generalize Stochastic Local Search (SLS) heuristics into a unique formal model. This model has two key components: a common structure designed to be as large as possible and a parametric structure intended to be as small as possible. Each heuristic is obtained by instantiating the parametric part in a different way. Particular instances for Genetic Algorithms (GA), Ant Colony Optimization (ACO), and Particle Swarm Optimization (PSO) are presented. Then, we use our model to prove the Turing-completeness of SLS algorithms in general. The proof uses our framework to construct a GA able to simulate any Turing machine. This Turing-completeness implies that determining any non-trivial property concerning the relationship between the inputs and the computed outputs is undecidable for GA and, by extension, for the general set of SLS methods (although not necessarily for each particular method). Similar proofs are more informally presented for PSO and ACO.