Linear Convergence of Comparison-based Step-size Adaptive Randomized Search via Stability of Markov Chains

TL;DR

This paper proves the global linear convergence of comparison-based step-size adaptive randomized search algorithms on scaling-invariant functions by analyzing the stability of associated Markov chains, linking stochastic optimization and Markov chain theory.

Contribution

It introduces a general methodology for establishing linear convergence of CB-SARS algorithms using Markov chain stability analysis on scaling-invariant functions.

Findings

Existence of a homogeneous Markov chain due to invariance properties.

Sufficient conditions for global linear convergence based on Markov chain stability.

Connection established between comparison-based algorithms and Markov chain Monte Carlo methods.

Abstract

In this paper, we consider comparison-based adaptive stochastic algorithms for solving numerical optimisation problems. We consider a specific subclass of algorithms that we call comparison-based step-size adaptive randomized search (CB-SARS), where the state variables at a given iteration are a vector of the search space and a positive parameter, the step-size, typically controlling the overall standard deviation of the underlying search distribution.We investigate the linear convergence of CB-SARS on\emph{scaling-invariant} objective functions. Scaling-invariantfunctions preserve the ordering of points with respect to their functionvalue when the points are scaled with the same positive parameter (thescaling is done w.r.t. a fixed reference point). This class offunctions includes norms composed with strictly increasing functions aswell as many non quasi-convex and non-continuousfunctions. On scaling-invariant functions, we show the existence of ahomogeneous Markov chain, as a consequence of natural invarianceproperties of CB-SARS (essentially scale-invariance and invariance tostrictly increasing transformation of the objective function). We thenderive sufficient conditions for \emph{global linear convergence} ofCB-SARS, expressed in terms of different stability conditions of thenormalised homogeneous Markov chain (irreducibility, positivity, Harrisrecurrence, geometric ergodicity) and thus define a general methodologyfor proving global linear convergence of CB-SARS algorithms onscaling-invariant functions. As a by-product we provide aconnexion between comparison-based adaptive stochasticalgorithms and Markov chain Monte Carlo algorithms.