Analysing the Robustness of Evolutionary Algorithms to Noise: Refined Runtime Bounds and an Example Where Noise is Beneficial

TL;DR

This paper refines the understanding of how noise affects the runtime of evolutionary algorithms on LeadingOnes, identifying precise thresholds for polynomial versus superpolynomial performance and demonstrating that noise can sometimes be beneficial.

Contribution

It provides refined runtime bounds for (1+1)EA and (1+λ)EA on LeadingOnes under noise, and presents an example where noise helps escape local optima.

Findings

Expected runtime is Θ(n^2) * exp(Θ(min{pn^2, n})) for all p ≤ 1/2.

Threshold between polynomial and superpolynomial times is at p = Θ((log n)/n^2).

Noise can be beneficial by helping escape local optima in rugged landscapes.

Abstract

We analyse the performance of well-known evolutionary algorithms (1+1)EA and (1+$\lambda$)EA in the prior noise model, where in each fitness evaluation the search point is altered before evaluation with probability $p$. We present refined results for the expected optimisation time of the (1+1)EA and the (1+$\lambda$)EA on the function LeadingOnes, where bits have to be optimised in sequence. Previous work showed that the (1+1)EA on LeadingOnes runs in polynomial expected time if $p = O((\log n)/n^2)$ and needs superpolynomial expected time if $p = \omega((\log n)/n)$, leaving a huge gap for which no results were known. We close this gap by showing that the expected optimisation time is $\Theta(n^2) \cdot \exp(\Theta(\min\{pn^2, n\}))$ for all $p \le 1/2$, allowing for the first time to locate the threshold between polynomial and superpolynomial expected times at $p = \Theta((\log n)/n^2)$. Hence the (1+1)EA on LeadingOnes is much more sensitive to noise than previously thought. We also show that offspring populations of size $\lambda \ge 3.42\log n$ can effectively deal with much higher noise than known before. Finally, we present an example of a rugged landscape where prior noise can help to escape from local optima by blurring the landscape and allowing a hill climber to see the underlying gradient. We prove that in this particular setting noise can have a highly beneficial effect on performance.

Figures

Tables

Table 1. Table 1: Overview of results on the expected optimisation time on LeadingOnes with prior noise. Results for the (1+1) EA also hold for asymmetric one-bit noise, for which no results on LeadingOnes are available, with the caveat that for p = ω ​ ( 1 / n ) 𝑝 𝜔 1 𝑛 p=\omega(1/n) we only have an upper bound of O ​ ( n 2 ) ⋅ e O ​ ( p ​ n 2 ) ⋅ 𝑂 superscript 𝑛 2 superscript 𝑒 𝑂 𝑝 superscript 𝑛 2 O(n^{2})\cdot e^{O(pn^{2})} . The bound O ​ ( λ ​ n + n 2 ) 𝑂 𝜆 𝑛 superscript 𝑛 2 O(\lambda n+n^{2}) from  [ 22 ] was simplified to O ​ ( n 2 ) 𝑂 superscript 𝑛 2 O(n^{2}) using their condition λ = o ​ ( n ) 𝜆 𝑜 𝑛 \lambda=o(n) .

Setting	Previous work		This work
(1+1) EA, one-bit noise $p$	$O\left(n^2\right)$ if $p\le 1/\left(6e{n}^2\right)$ [22, Cor. 18]	\rdelim}131em	$\Theta \left(n^2\right)\cdot e^{\Theta \left(\min \{p{n}^2,n\}\right)}$ if $p\le 1/2$
	$2^{\Omega \left(n\right)}$ if $p=1/2$ [22, Thm. 20]
	polynomial if $p=O\left(\left(\log n\right)/n^2\right)$ [42, Thm. 14]
	superpolynomial if $p=\omega \left(\left(\log n\right)/n\right)\cap o\left(1\right)$ [42, Thm. 14]
	exponential if $p=\Omega \left(1\right)$ [42, Thm. 14]
(1+1) EA, bit-wise noise $(p,1/n)$	polynomial if $p=O\left(\left(\log n\right)/n^2\right)$ [42, Thm. 8]
	superpolynomial if $p=\omega \left(\left(\log n\right)/n\right)\cap o\left(1\right)$ [42, Thm. 9]
	exponential if $p=\Omega \left(1\right)$ [42, Thm. 10]
(1+1) EA, bit-wise noise $(1,p/n)$	polynomial if $p=O\left(\left(\log n\right)/n^2\right)$ [42, Thm. 11]
	superpolynomial if $p=\omega \left(\left(\log n\right)/n\right)\cap o\left(1\right)$ [42, Thm. 12]
	exponential if $p=\Omega \left(1\right)$ [42, Thm. 13]
(1+1) EA, bit-wise noise $(p^{\prime },q/n)$	polynomial if $p:=p^{\prime }\min \{q,1\}=O\left(\left(\log n\right)/n^2\right)$ [5, Thm. 5]
	superpolynomial if $p:=p^{\prime }\min \{q,1\}=\omega \left(\left(\log n\right)/n\right)$ [5, Thm. 6]
(1+$\lambda$) EA,	$O\left(\lambda n+n^2\right)$ if $p\le 0.028/n$		$O\left(n^2\cdot e^{O\left(pn/\lambda \right)}\right)$ if $p\le 1/2$
one-bit noise $p$	and $72\log n\le \lambda =o\left(n\right)$ [22, Cor. 24]		and $3.42\log n\le \lambda =O\left(n\right)$

Equations

$$\frac{1}{2}\min \{p\ell ,1\}\le \frac{p\ell }{1+p\ell }\le 1-(1-p{)}^{\ell }\le \min \{p\ell ,1\}$$

$$\frac{p}{2}\min \{q,1\}\le p(1-(1-q/n{)}^n)\le p\min \{q,1\}.$$

$$E_{\mathcal{A},f}:=\underset{x}{\max }E(T_{\mathcal{A},f}(x))$$

$$M_{\mathcal{A},f}(x):=\min \{t\mid \mathrm{Pr}(T_{\mathcal{A},f}(x)\le t)\ge 1/2\}$$

$$M_{\mathcal{A},f}:=\underset{x}{\max }{M}_{\mathcal{A},f}(x).$$

$$2M(1-p{)}^{-M}\le 2M\cdot e^{pM/(1-p)}.$$

$$2M(1-p{)}^{-\nu M}\le 2M\cdot e^{\nu pM/(1-p)}$$

$$O\big{(}n^{2}\cdot e^{O(pn^{2})}\big{)}.$$

$$\mathrm{Pr}(i,j>\ell +1)\ge \mathrm{Pr}(i=\ell +1).$$

$$\pi (x)\cdot \mathrm{Pr}(x\to y)=\pi (y)\cdot \mathrm{Pr}(y\to x),\text{for all}x,y\in \{0,1{\}}^n$$

$$||p_t-\pi ||:=\underset{A\subset \Omega }{\max }|p_t(A)-\pi (A)|.$$

$$||p_t-\pi ||\le \mathrm{Pr}(\underset{x,y}{\max }{T}_{x,y}>t).$$

$$\frac{1/(3en)}{q(1-q{)}^{i-1}+1/(3en)}=\frac{1}{1+3enq(1-q{)}^{i-1}}.$$

$$\prod _{i=1}^n\frac{1}{1+3enq(1-q{)}^{i-1}}=\prod _{i=0}^{n-1}\frac{1}{1+3enq(1-q{)}^i}.$$

$$\prod _{i=0}^{n-1}(1+3enq(1-q{)}^i)\le$$

$$=$$

$$\le$$

$$\Theta (n^2)\cdot e^{\Theta (\min \{p{n}^2,n\})}$$

$$\mathrm{Pr}(x^t=1)\le \frac{1}{2}\left(1+{\left(1-\frac{2}{n}\right)}^t\right).$$

$$\frac{1}{2}\left(1+{\left(1-\frac{2}{n}\right)}^{n/2}\right)\le \frac{1}{2}\left(1+\frac{1}{e}\right)=\frac{e+1}{2e}.$$

$${\left(1-\Omega (p)\right)}^{c{n}^2}\le e^{-\Omega (p{n}^2)}.$$

$$p{\left(1-{\left(1-\frac{1}{n}\right)}^n(1-p)\right)}^{\lambda }=p{\left(\frac{e-(1-p)}{e}\right)}^{\lambda }\cdot \exp (O(\lambda /n)).$$

$$\sum _{i=0}^{\lambda }\left(\genfrac{}{}{0ex}{}{\lambda }{i}\right)q^i(1-q{)}^{\lambda -i}{p}^{i+1}=$$

$$=$$

$${\left(1-{\left(1-\frac{1}{n}\right)}^n(1-p)\right)}^{\lambda }$$

$$\le$$

$$=$$

$$\le$$

$$O\left(n^2\cdot e^{O(pn/\lambda )}\right)$$

$$p_1:=O\left(p{\left(\frac{e-(1-p)}{e}\right)}^{\lambda }\right)\le O\left(p{\left(\frac{e-1/2}{e}\right)}^{\lambda }\right)=O\left(\frac{p}{n}\right).$$

$$p_2:=\frac{p}{n}$$

$$p_3\le \frac{2p\lambda }{n^2}=O\left(\frac{p}{n}\right).$$

Full text

Finally, we present an example of a rugged landscape where prior noise can help to escape from local optima by blurring the landscape and allowing a hill climber to see the underlying gradient. We prove that in this particular setting noise can have a highly beneficial effect on performance.

Keywords:

Evolutionary algorithms; noisy optimisation; robustness; runtime analysis; theory; uncertainty

1 Introduction

Many real-world problems suffer from sources of uncertainty, such as noise in the fitness evaluation, changing constraints, or dynamic changes to the fitness function [26]. Evolutionary algorithms are well suited for dealing with these challenges due to their use of a population, and because they can often recover quickly from setbacks resulting from noise or dynamic changes. They have proven to work well in many applications to combinatorial problems [6].

However, our theoretical understanding of how evolutionary algorithms deal with noise is limited. It is often not clear how noise affects the performance of evolutionary algorithms, and how much noise an evolutionary algorithm can cope with. For evolution strategies in continuous optimisation there exists a rich body of work (see, e. g. [4, 32, 25] and the references therein), but there are only few rigorous theoretical analyses on the performance of noisy evolutionary optimisation in discrete spaces.

The first runtime analysis for discrete evolutionary algorithms in a noisy setting was given by Droste [16] in the context of a simple algorithm called (1+1) EA on the well-known function OneMax $(x):=\sum_{i=1}^{n}x_{i}$, which simply counts the number of bits set to 1. He considered a setting now known as one-bit prior noise, where with probability $p$ a uniform random bit is flipped before evaluation. Hence, instead of returning the fitness of the evaluated search point, the fitness function may return the fitness of a random Hamming neighbour. He proved that, when $p=O((\log n)/n)$ the (1+1) EA can still optimise OneMax efficiently. But when $p=\omega((\log n)/n)$ the expected optimisation time becomes superpolynomial.

Gießen and Kötzing [22] studied a more general class of algorithms, including the (1+1) EA, the (1+$\lambda$) EA that generates $\lambda$ new solutions (offspring) in parallel and picks the best one, and the ($\mu$+1) EA that keeps a population of $\mu$ search points. They considered prior noise and posterior noise, where posterior noise means that noise is added to the fitness value, and presented an elegant approach that gives results in both noise models. They showed that the (1+1) EA on OneMax runs in expected time $O(n\log n)$ if $p=O(1/n)$, polynomial time if $p=O((\log n)/n)$, and superpolynomial time if $p=\omega((\log n)/n)\cap 1-\omega((\log n)/n)$. The same results hold in the bit-wise noise model, where each bit is flipped independently before evaluation with probability $p/n$. They also considered the function LeadingOnes that counts the length of the longest prefix that only contains bits set to 1. For LeadingOnes they show a time bound of $O(n^{2})$ if $p\leq 1/(6en^{2})$ and an exponential lower bound if $p=1/2$.

The authors also found that using parent populations in a ($\mu$+1) EA can drastically improve robustness as survival selection removes one of the worst individuals, and a population increases the chances that a low-fitness individual will be correctly identified as having low fitness. Offspring populations also increase robustness as they amplify the probability that a clone of the current search point will be evaluated truthfully, thus lowering the chance of losing the best fitness. For LeadingOnes they showed a time bound for the (1+$\lambda$) EA of $O(\lambda n+n^{2})$ if $p\leq 0.028/n$ and $72\log n\leq\lambda=o(n)$. Note that their bound simplifies to $O(n^{2})$ since $\lambda=o(n)$.

Dang and Lehre [9] gave general results for prior and posterior noise in non-elitist evolutionary algorithms, that is, evolutionary algorithms where the best fitness in the population may decrease. The same authors [10] also considered noise resulting from only partially evaluating search points.

In terms of posterior noise, Sudholt and Thyssen [51] considered the performance of a simple ant colony optimiser (ACO) for computing shortest paths when path lengths are obscured by positive posterior noise modelling traffic delays. They showed that noise can make the ants risk-seeking, tricking them onto a suboptimal path and leading to exponential optimisation times. Doerr, Hota, and Kötzing [13] showed that this problem can be avoided if the parent is reevaluated in each iteration. Feldmann and Kötzing [18] further analysed the performance of fitness-proportional updates. Friedrich, Kötzing, Krejca, and Sutton [20] showed that the compact Genetic Algorithm and ACO [19] are both efficient under extreme Gaussian posterior noise, while a simple ($\mu$+1) EA is not.

Prugel-Bennett, Rowe, and Shapiro [40] considered a population-based algorithm using only selection and crossover, and showed that the algorithm can optimise OneMax with a large amount of noise. Qian, Yu, and Zhou [45] showed that noise can be handled efficiently by combining reevaluation and threshold selection. Akimoto, Astete-Morales, and Teytaud [1] as well as Qian, Yu, Tang, Jin, Yao, and Zhou [44] showed that resampling can essentially eliminate the effect of noise.

[FIGURE:]