Exponential Slowdown for Larger Populations: The $(\mu+1)$-EA on Monotone Functions

TL;DR

This paper demonstrates that increasing the population size in the $(+1)$-EA can cause exponential slowdown on monotone functions, due to decreased selective pressure and accumulation of unfavorable mutations, contrasting with other benchmarks.

Contribution

It proves that for any mutation rate, a sufficiently large population size can cause superpolynomial optimization time on monotone functions, highlighting a counter-intuitive negative effect of larger populations.

Findings

Larger populations can slow down optimization on monotone functions.

Increased population size can lead to accumulation of unfavorable mutations.

Counter-intuitive effect where larger populations hinder rather than help optimization.

Abstract

Pseudo-Boolean monotone functions are unimodal functions which are trivial to optimize for some hillclimbers, but are challenging for a surprising number of evolutionary algorithms (EAs). A general trend is that EAs are efficient if parameters like the mutation rate are set conservatively, but may need exponential time otherwise. In particular, it was known that the $(1+1)$-EA and the $(1+\lambda)$-EA can optimize every monotone function in pseudolinear time if the mutation rate is $c/n$ for some $c<1$, but they need exponential time for some monotone functions for $c>2.2$. The second part of the statement was also known for the $(\mu+1)$-EA. In this paper we show that the first statement does not apply to the $(\mu+1)$-EA. More precisely, we prove that for every constant $c>0$ there is a constant integer $\mu_0$ such that the $(\mu+1)$-EA with mutation rate $c/n$ and population size $\mu_0\le\mu\le n$ needs superpolynomial time to optimize some monotone functions. Thus, increasing the population size by just a constant has devastating effects on the performance. This is in stark contrast to many other benchmark functions on which increasing the population size either increases the performance significantly, or affects performance mildly. The reason why larger populations are harmful lies in the fact that larger populations may temporarily decrease selective pressure on parts of the population. This allows unfavorable mutations to accumulate in single individuals and their descendants. If the population moves sufficiently fast through the search space, such unfavorable descendants can become ancestors of future generations, and the bad mutations are preserved. Remarkably, this effect only occurs if the population renews itself sufficiently fast, which can only happen far away from the optimum. This is counter-intuitive since usually optimization gets harder as we approach the optimum.

Equations

$$\ell (x):=\max \left\{{\ell }^{\prime }\in [L]:d(B_{{\ell }^{\prime }},x)\le \epsilon \right\},$$

$$\text{textsc}HT(x):=\ell (x)\cdot n^2+\sum _{i\in A_{\ell (x)+1}}{x}_i\cdot n+\sum _{i\in R_{\ell (x)+1}}{x}_i,$$

$$\mathrm{Pr}[S\le (1-\delta )E[S]]\le e^{-{\delta }^{2}E[S]/2}$$

$$\mathrm{Pr}[S\ge (1+\delta )E[S]]\le e^{-\min \{{\delta }^2,\delta \}E[S]/3}.$$

$$\mathrm{Pr}[S\ge k]\le 2^{-k}.$$

$$\Pr[S\leq\operatorname{\mathbb{E}}[S]-\delta]\leq\exp\Big{(}-\frac{\delta^{2}}{2s}\Big{)}.$$

$$\Pr[S\geq\operatorname{\mathbb{E}}[S]+\delta]\leq\exp\Big{(}-\frac{\delta}{4}\min\Big{\{}\frac{\delta}{s},h\Big{\}}\Big{)}.$$

$$\mathrm{Pr}[\mathrm{rk}(y)\ge i]\le 2^{-{\epsilon }^{\prime }\alpha n}\text{or}\frac{\mathrm{Pr}[\mathrm{rk}(y)\ge i+1]}{\mathrm{Pr}[\mathrm{rk}(y)\ge i]}\le 2{\epsilon }^{\prime }\alpha c.$$

$$\mathrm{Pr}[\text{flip 1 zero-bit and 0 one-bits in A}]$$

$$\displaystyle=\varepsilon(x)\alpha c\Big{(}e^{-\alpha c}-O\Big{(}\frac{1}{n}\Big{)}\Big{)}$$

$$\ge \frac{1}{2}\epsilon (x)\alpha c{e}^{-\alpha c}.$$

$$\mathrm{Pr}[\text{flip at least 1 zero-bit}]\le \sum _{i=1}^{\epsilon (x)\alpha n}\mathrm{Pr}[\text{flip the i-th zero-bit}]=\epsilon (x)\alpha c,$$

$$\mathrm{Pr}[\mathrm{rk}(y)\ge i+1]$$

$$=2{\epsilon }^{\prime }\alpha c\cdot \mathrm{Pr}[\mathrm{rk}(y)\ge i].\text{qed}$$

$$\mathrm{Pr}[|Y_{i+1}-Y_i|\ge y\mid Y_0,\dots ,Y_i]\le c^{\prime }(1+{\delta }^{\prime }{)}^{-y}.$$

$$E[Y_{i+1}-Y_i\mid Y_0,\dots ,Y_i]\le r,$$

$$\Pr\Big{[}\max_{0\leq j\leq i}(Y_{j}-Y_{0})\geq ri+y\Big{]}\leq\exp\Big{(}-\frac{y}{2}\min\Big{(}\delta^{\prime\prime},\frac{y}{c^{\prime\prime}i}\Big{)}\Big{)}.$$

$$d(A_{\ell +1},x^{\ast })\in [\epsilon /2,2\epsilon ]\text{and}d(R_{\ell +1},x^{\ast })\in [\epsilon /2,2\epsilon ],$$

$$f_{\ell }(x):=n\sum _{j\in A_{\ell +1}}{x}_j+\sum _{j\in R_{\ell +1}}{x}_j.$$

$$\frac{1}{2}(\kappa -\eta )\mu \log \mu \le T^{\kappa }\le 4\kappa e^{\alpha c}\mu \log \mu .$$

$$E[T^{\kappa }]\le 3\kappa e^{\alpha c}\mu \log \mu .$$

$$E[{\bar{T}}^{\kappa }]=\sum _{j=1}^{{\mu }^{\kappa }-1}E[{\bar{T}}_j]\le 2e^{\alpha c}\mu \sum _{j=1}^{{\mu }^{\kappa }}\frac{1}{j}.$$

$$E[T^{\kappa }]\le E[{\bar{T}}^{\kappa }]\le 2e^{\alpha c}\mu (\log ({\mu }^{\kappa })+1)\le 3\kappa e^{\alpha c}\mu \log \mu .$$

$$s\le 4e^{2\alpha c}{\mu }^2\sum _{j=1}^{{\mu }^{\kappa }}\frac{1}{j^2}\le \frac{2e^{2\alpha c}{\pi }^2}{3}{\mu }^2,$$

$$\mathrm{Pr}\left[{\bar{T}}^{\kappa }\ge E[{\bar{T}}^{\kappa }]+\delta \right]\le e^{-\Omega (\log \mu )}={\mu }^{-\Omega (1)}.$$

$$\mathrm{Pr}\left[y^t\in X_{\ge i}\right]$$

$$\le (\mu -j)/\mu \cdot p_U+j/\mu \cdot 1\le j/\mu +p_U,$$

$$E[{\hat{T}}^{\kappa }]$$

$$\ge \sum _{j=1}^{{\mu }^{\kappa }-1}\frac{\mu }{j+\lceil \mu p_U\rceil }=\sum _{j=1}^{{\mu }^{\kappa }-1+\lceil \mu p_U\rceil }\frac{\mu }{j}-\sum _{j^{\prime }=1}^{\lceil \mu p_U\rceil }\frac{\mu }{j^{\prime }}$$

$$\displaystyle>\mu\log\big{(}\mu^{\kappa}+\lceil\mu p_{U}\rceil\big{)}-\mu\big{(}\log\lceil\mu p_{U}\rceil+1\big{)}.$$

$$\displaystyle\operatorname{\mathbb{E}}[T^{\kappa}]\geq\operatorname{\mathbb{E}}[\hat{T}^{\kappa}]\geq\big{(}1-O\big{(}\log^{-1}\mu\big{)}\big{)}(\kappa-\eta)\mu\log\mu.$$

$$\mathrm{Pr}\left[{\hat{T}}^{\kappa }\le E[{\hat{T}}^{\kappa }]-{\delta }^{\prime }\right]\le e^{-\Omega ({\log }^2\mu )}={\mu }^{-\Omega (1)}.$$

$$T^{\kappa }\ge \frac{1}{2}(\kappa -\eta )\mu \log \mu$$

Full text

In this paper we show that the first statement does not apply to the $(\mu+1)$-EA. More precisely, we prove that for every constant $c>0$ there is a constant $\mu_{0}\in\mathbb{N}$ such that the $(\mu+1)$-EA with mutation rate $c/n$ and population size $\mu_{0}\leq\mu\leq n$ needs superpolynomial time to optimize some monotone functions. Thus, increasing the population size by just a constant has devastating effects on the performance. This is in stark contrast to many other benchmark functions on which increasing the population size either increases the performance significantly, or affects performance only mildly.

The reason why larger populations are harmful lies in the fact that larger populations may temporarily decrease the selective pressure on parts of the population. This allows unfavorable mutations to accumulate in single individuals and their descendants. If the population moves sufficiently fast through the search space, then such unfavorable descendants can become ancestors of future generations, and the bad mutations are preserved. Remarkably, this effect only occurs if the population renews itself sufficiently fast, which can only happen far away from the optimum. This is counter-intuitive since usually optimization becomes harder as we approach the optimum. Previous work missed the effect because it focused on monotone functions that are only deceptive close to the optimum.

††journal: Theoretical Computer Science

1 Introduction

Population-based evolutionary algorithms (EAs) are general-purpose heuristics for optimization. Having a population may be helpful, because it allows for diversity in the algorithm’s states. Such diversity may be helpful for escaping local minima, and it is a necessary ingredient for crossover operations as they are used in genetic algorithms (GAs). Theoretical and practical analysis of population-based algorithms have indeed mostly found positive or neutral effects, and showed a general trend that larger populations are better [1], or at least not worse than a population size of one [2]. The only (mild) observed negative effect is, intuitively speaking, that maintaining a population of size $\mu$ may slow down the optimization time by a factor of at most $\mu$. Only few, highly artificial examples are known [3, 4] in which a $(\mu+1)$-EA or $(\mu+1)$-GA with time budget $\mu t$ performs significantly worse than a $(1+1)$-EA with time budget $t$. In this sense, it is easy to believe that a $(\mu+1)$ algorithm is at least as good as a $(1+1)$ algorithm, except for the runtime increase that comes from each individual only having probability $1/\mu$ per round of creating an offspring.

Our results challenge this belief, and show that it is highly wrong for some monotone functions. Our main results show that increasing $\mu$ from $1$ to a larger constant can increase the runtime from quasilinear to exponential.

A monotone555Following [5, 6], we call them monotone functions, although strictly monotone functions would be slightly more accurate. pseudo-Boolean function is a function $f:\{0,1\}^{n}\to\mathbb{R}$ such that for every $x,y\in\{0,1\}^{n}$ with $x\neq y$ and $x_{i}\geq y_{i}$ for all $1\leq i\leq n$ it holds $f(x)>f(y)$. Monotone functions are easy benchmark functions for optimization techniques, since they always have a unique local and global optimum at the all-ones string. Moreover, from every search point there are short, fitness-increasing paths to the optimum, by flipping zero-bits into one-bits. Consequently, there are many algorithms which can easily optimize every monotone function. A particular example is random local search (RLS), which is the $(1+1)$ algorithm that flips in each round exactly one bit, uniformly at random. RLS can never increase the distance from the optimum for a monotone function, and it optimizes any such function in time $O(n\log n)$ by a coupon collector argument. Thus monotone functions are regarded as an easy benchmark for evolutionary algorithms. Nevertheless it was shown in [5, 6, 7, 8] that a surprising number of evolutionary algorithms need exponential time to optimize some monotone functions, especially if they mutate too aggressively, i.e., the mutation parameter $c$ is too large (see Section 1.2 for a detailed discussion). However, in all considered cases the algorithms were efficient if the mutation parameter satisfied $c<1$.

1.1 Our Results

We show that the $(\mu+1)$-Evolutionary Algorithm, $(\mu+1)$-EA, becomes inefficient even if the mutation strength is smaller than 1. More precisely, we show that for every $c>0$ there is a $\mu_{0}=\mu_{0}(c)\in\mathbb{N}$ such that for all $\mu_{0}\leq\mu\leq n$ there are some monotone functions for which the $(\mu+1)$-EA with mutation rate $c/n$ needs superpolynomial time to find the optimum. If $\mu$ is $O(1)$ then this time is even exponential in $n$. Note that for $0<c\leq 1$, it is known that the $(1+1)$-EA finds the optimum in quasilinear time for any monotone functions  [8, 9, 10]. Thus when we increase the population size only slightly (from $1$ to $\mu_{0}$), the optimization time explodes, from quasilinear to exponential.

The monotone functions that are hard to optimize are due to Lengler and Steger [5], and were dubbed HotTopic functions in [6]. These functions look locally like linear functions in which all bits have some positive weights. However, in each region of the search space there is a specific subset of bits (the ‘hot topic’), which have very large weights, while all other bits have only small weights. If an algorithm improves in the hot topic, then it will accept the offspring regardless of whether the other bits deteriorate. In [5, 6, 11] it was shown that an algorithm like the $(1+1)$-EA with mutation rate $c>2.13..$ will mutate too many of these bits outside of the hot topic, and will thus not make progress towards the global optimum.

The key insight of our paper is that for such weighted linear functions with imbalanced weights, populations may also lead to an accumulation of bad mutations, even if the mutation rate is small. Here is the intuition. For a search point $x$, we call the number of one-bits in the hot topic in $x$ the rank of $x$. Consider a $(\mu+1)$-EA close to the optimum, and assume for simplicity that all search points in the population $S_{0}$ have the same rank $i$. At some point one of them will improve in the hot topic by flipping a zero-bit there. Let us call the offspring $x$, and let us assume that its rank is $i+1$. Then $x$ is fitter than all other search points in the population because it has a higher rank. Moreover, every offspring or descendant of $x$ will also be fitter than all the other points in the population, as long as they maintain rank $i+1$. Thus for a while the $(\mu+1)$-EA will accept all (or most) descendants of $x$, and remove search points of rank $i$ from the population. This goes on until some time $t_{0}$ at which search points of rank $i$ are completely eliminated from the population. Note that at time $t_{0}$, most descendants $x^{\prime}$ of $x$ have considerably smaller fitness than $x$, since the algorithm accepts every type of mutation outside of the hot topic, and most mutations are detrimental. If some descendant $x^{\prime}$ of $x$ creates an offspring $y$ of even higher rank, then $y$ is accepted and the cycle repeats with $y$ instead of $x$. The crucial point is that $y$ is an offspring of $x^{\prime}$, which has accumulated a lot of bad mutations compared to $x$. So typically, $x^{\prime}$ is considerably less fit than $x$, but still it passes on its bad genes.

The above effect needs that the probability of improving in the hot topic has the right order. If the probability is too large (close to one), then $x$ will already spawn an offspring of rank $i+1$ before it has spawned many descendants with the same rank. On the other hand, if the probability is too small then there will be no rank-improving mutations until time $t_{0}$, and after time $t_{0}$ the algorithm starts to remove the worst individuals of rank $i+1$ from the population. We remark that this latter regime was already studied in [6], for the extreme case in which the improvement probability is so small that typically the population of rank $i+1$ collapses into copies of $x$ before a further improvement is made. (In the terminology of [6], it was the assumption that the parameter $\varepsilon$ of the HotTopic function was sufficiently small.) However, there is a rather large range of improvement probabilities that lead to the aforementioned effect, i.e., they typically yield an offspring $y$ from some inferior search point $x^{\prime}$ of rank $i+1$.

1.2 Related Work

The analysis of EAs on monotone functions started in 2010 by the work of Doerr, Jansen, Sudholt, Winzen and Zarges [7, 8]. Their contribution was twofold: firstly, they showed that the $(1+1)$-EA, which flips each bit independently with static mutation rate $c/n$, needs time $O(n\log n)$ on all monotone functions if the mutation parameter $c$ is a constant strictly smaller than one. This result was already implicit in [9].

On the other hand, it was also shown in [7, 8] that for large mutation rates, $c>16$, there are monotone functions for which the $(1+1)$-EA needs exponential time. The construction of hard monotone functions in [7, 8] was later simplified by Lengler and Steger [5], who improved the range for $c$ from $c>16$ to $c>c_{0}=2.13..$. Their construction was later called HotTopic functions in [6], and it will also be the basis for the results in this paper.

For a long time, it was an open question whether $c=1$ is a threshold at which the runtime switches from polynomial to exponential. On the presumed threshold $c=1$, a bound of $O(n^{3/2})$ was known due to Jansen [9], but it was unclear whether the runtime is quasilinear. Finally, Lengler, Martinsson and Steger [10] could show that $c=1$ is not a threshold, showing by an information compression argument an $O(n\log^{2}n)$ bound for all $c\in[1,1+\varepsilon]$ for some $\varepsilon>0$.

Recently, the limits of our understanding of monotone functions were pushed significantly by Lengler [6, 11], who analyzed monotone functions for a manifold of other evolutionary and genetic algorithms. In particular, he analyzed the algorithms on HotTopic functions, and found sharp thresholds in the parameters, such that on one side of the threshold the runtime on HotTopic was $O(n\log n)$, while on the other side of the threshold it was exponential. These algorithms include the $(1+1)$-EA, the $(1+\lambda)$-EA, the $(\mu+1)$-EA, for which the threshold condition was $c<c_{0}$, where $c_{0}=2.13..$, and it further included the $(1+(\lambda,\lambda))$-GA, and the so-called ‘fast $(1+1)$-EA’ and ‘fast $(1+\lambda)$-EA’.666The so-called “fast” versions draw the parameter $c$ randomly in each iteration from a heavy-tailed distribution. This avoids that the probability of flipping $k$ bits drops exponentially in $k$ [12]. Surprisingly, for the genetic algorithms $(\mu+1)$-GA and the ‘fast $(\mu+1)$-GA’, any parameter range leads to runtime $O(n\log n)$ on HotTopic if the population size $\mu$ is large enough, showing that crossover is strongly beneficial in these cases.

For some of the algorithms, Lengler in [6, 11] also complemented the results on HotTopic functions by statements asserting that for less aggressive choices of the parameters the algorithms optimize every monotone function efficiently. For example, he proved that for mutation parameter $c<1$ and for every constant $\lambda\in\mathbb{N}$, with high probability the $(1+\lambda)$-EA optimizes every monotone function in $O(n\log n)$ steps. Analogous statements were proven for the ‘fast $(1+1)$-EA’ and ‘fast $(1+\lambda)$-EA’, and for the $(1+(\lambda,\lambda))$-GA, but the condition $c<1$ needs to be replaced by analogous conditions on the parameters of the respective algorithms. Moreover, in the case of the ‘fast $(1+\lambda)$-EA’, the result was only proven if the algorithm starts sufficiently close to the optimum. Lengler did not prove any results for general monotone functions for the population-based algorithms $(\mu+1)$-EA and $(\mu+1)$-GA, and for their ‘fast’ counterparts. Our result shows that at least for the $(\mu+1)$-EA, this gap had a good reason. As mentioned before, we will show that for every (constant) mutation parameter $c>0$, there are monotone functions on which the $(\mu+1)$-EA needs superpolynomial time if the population size $\mu$ is larger than some constant $\mu_{0}=\mu_{0}(c)$. It also shows that the $(\mu+1)$-EA and the $(1+\lambda)$-EA behave completely differently on the class of monotone functions, since the $(1+\lambda)$-EA is efficient for all constant $\lambda$ whenever $c<1$.

Surprisingly, our instance of a hard monotone function is again a HotTopic function. This may appear contradictory to the result in [6, 11] that the $(\mu+1)$-EA is efficient on HotTopic functions if $c<c_{0}$. The reason why there is no contradiction is that all the results in [6, 11] on HotTopic come with an important catch. The HotTopic functions come with several parameters, and we will give the formal definition and a more detailed discussion in Section 2.3. For now it suffices to know that one of the parameters, $\varepsilon$, essentially determines how close the algorithm needs to come to the optimum before the fitness function starts switching between different hot topics. In [6, 11], only small values of $\varepsilon$ were considered. More precisely, it was shown that for every $\mu\in\mathbb{N}$ there is an $\varepsilon_{0}>0$ such that the results for the $(\mu+1)$-EA hold for all HotTopic functions with parameter $\varepsilon\leq\varepsilon_{0}$, and there were similar restrictions for other parameters of the HotTopic function. In a nutshell, the effect of switching hot topics was only studied close to the optimum. Arguably, this was a natural approach since usually the hardest region for optimization is close to the optimum. In this paper, we consider HotTopic functions in a different parameter regime: we study relatively large values of the parameter $\varepsilon$, which is a regime of the HotTopic functions in which the action happens far away from the optimum. Consequently, the results from [6, 11] on the $(\mu+1)$-EA on HotTopic do not carry over to the version of HotTopic functions that we consider in this paper. We stress this point to resolve the apparent contradiction between our results and the results in [6, 11].

The above discussion also shows a rather uncommon phenomenon. Consider a small mutation parameter, e.g., $c=1/2$. Our results show that the $(\mu+1)$-EA fails to make progress if the HotTopic function starts switching hot topics far away from the optimum. On the other hand, by the results in [6], the $(\mu+1)$-EA is not deceived if the HotTopic function starts switching hot topics close to the optimum. Thus, we have found an example where optimization close to the optimum is easier than optimization far away from the optimum, quite the opposite of the usual behavior of algorithms. This strange effect occurs because the problem of the $(\mu+1)$-EA arises from having a non-trivial population. However, close to the optimum, progress is so hard that the population tends to degenerate into multiple copies of a single search point, which effectively decreases the population size to one and thus eliminates the problem (see also the discussion in Section 1.1 above).

Most other work on population-based algorithms has shown benefits of larger population sizes, especially when crossover is used [13, 14, 15, 16]. Without crossover, the effect is often rather small [2]. The only exception in which a population has theoretically been proven to be severely disadvantageous is on Ignoble Trails. This rather specific function has been carefully designed to lead into a trap for crossover operators [3], and it is deceptive for $\mu=2$ if crossover is used, but not for $\mu=1$. Arguably, the HotTopic functions are also rather artificial, although they were not specifically designed to be deceptive for populations. However, regarding the larger and more natural framework of monotone functions, our results imply that a $(\mu+1)$-EA with mutation parameter $c=1$ does not optimize all monotone functions efficiently if $\mu$ is too large, while the corresponding $(1+1)$-EA is efficient.

Moreover, Lengler and Schaller pointed out an interesting connection between HotTopic functions and a dynamic optimization problem in [17], which is arguably more natural. In that paper, the algorithm should optimize a linear function with positive weights, but the weights of the objective function are re-drawn each round (independently and identically distributed). This setting is similar to monotone functions, since a one-bit is always preferable over a zero-bit, and the all-one string is always the global optimum. However, the weight of each bit changes from round to round, which somewhat resembles that the HotTopic function switches between different hot topics as the algorithm progresses. In [17] the $(1+1)$-EA was studied, and the behavior in the dynamic setting is very similar to the behavior on HotTopic functions. It remains open whether the effects observed in our paper carry over to this dynamic setting.

2 Preliminaries and Definitions

2.1 Notation

Throughout the paper we will assume that $f:\{0,1\}^{n}\to\mathbb{R}$ is a monotone function, i.e., for every $x,y\in\{0,1\}^{n}$ with $x\neq y$ and such that $x_{i}\geq y_{i}$ for all $1\leq i\leq n$ it holds $f(x)>f(y)$. We will consider algorithms that try to maximize $f$, and we will mostly focus on the runtime of an algorithm, which we define as the number of function evaluations before the first evaluation of the global maximum of $f$.

For $n\in\mathbb{N}$, we denote $[n]:=\{1,\ldots,n\}$. For a search point $x$, we write $\textsc{Om}(x)$ for the OneMax-value of $x$, i.e., the number of one-bits in $x$. For $x\in\{0,1\}^{n}$ and $\emptyset\neq I\subseteq[n]$, we denote by $d(I,x):=|\{i\in I\mid x_{i}=0\}|/|I|$ the density of zero-bits in $I$. In particular, $d([n],x)=1-\textsc{Om}(x)/n$. Landau notation like $O(n),o(n),\ldots$ is with respect to $n\to\infty$. An event $\mathcal{E}=\mathcal{E}(n)$ holds with high probability or whp if $\Pr[\mathcal{E}(n)]\to 1$ for $n\to\infty$. A function $f:\mathbb{N}\to\mathbb{R}$ grows stretched-exponentially if there is $\delta>0$ such that $f(x)=\exp\{\Omega(n^{\delta})\}$, and it grows quasilinearly if there is $C>0$ such that $f(x)=O(n\log^{C}n)$.

Throughout the paper we will use $n$ for the dimension of the search space, $\mu$ for the population size, and $c$ for the mutation parameter. We will always assume that the mutation parameter $c$ is a constant independent of $n$, but the population size $\mu=\mu(n)$ may depend on $n$.

2.2 Algorithm

We will consider the $(\mu+1)$-EA with population size $\mu\in\mathbb{N}$ and mutation parameter $c>0$ for maximizing a pseudo-boolean fitness function $f:\{0,1\}^{n}\to\mathbb{R}$. This algorithm maintains a population of $\mu$ search points. In each round, it picks one of these search points uniformly at random, the parent $x^{t}$ for this round. From this parent it creates an offspring $y^{t}$ by flipping each bit of $x^{t}$ independently with probability $c/n$, and adds it to the population. From the $\mu+1$ search points, it then discards the one with lowest fitness from the population, breaking ties randomly 777We break ties randomly for simplicity. Other selection schemes may give preference to offspring, or generally to more recent search points in case of ties. However, the tie-breaking scheme does not have an impact on our analysis..

2.3 HotTopic Functions

In this section we give the construction of hard monotone functions by Lengler and Steger [5], following closely the exposition in [6]. The functions come with five parameters $n\in\mathbb{N}$, $0<\beta<\alpha<1$, $0<\varepsilon<1$, and $L\in\mathbb{N}$, and they are given by a randomized construction. Following [6], we call the corresponding function $\textsc{HotTopic}_{n,\alpha,\beta,\varepsilon,L}=\textsc{HT}_{n,\alpha,\beta,\varepsilon,L}=\textsc{HT}$.

For $1\leq i\leq L$ we choose sets $A_{i}\subseteq[n]$ of size $\alpha n$ independently and uniformly at random, and we choose subsets $B_{i}\subseteq A_{i}$ of size $\beta n$ uniformly at random. We define the level $\ell(x)$ of a search point $x\in\{0,1\}^{n}$ by

[TABLE]

where we set $\ell(x)=0$, if no such $\ell^{\prime}$ exists. Then we define $f:\{0,1\}^{n}\to\mathbb{R}$ as follows:

[TABLE]

where $R_{\ell(x)+1}:=[n]\setminus A_{\ell(x)+1}$, and where we set $A_{L+1}:=B_{L+1}:=\emptyset$. One easily checks that this function is monotone [6].

So the set $A_{\ell+1}$ defines the hot topic while the algorithm is at level $\ell$, where the level is determined by the sets $B_{i}$. Following up on the discussion in the introduction, observe that the level $\ell$ increases if the density of zero-bits in $B_{\ell^{\prime}}$ drops below $\varepsilon$ for some $\ell^{\prime}>\ell$. From the analysis we will see that with high probability this only happens if the density of zero-bits in $A_{\ell+1}$ and in the whole string is also roughly $\varepsilon$, up to some constant factors. Hence, the parameter $\varepsilon$ determines how far away the algorithm is from the optimum when the level changes.

Throughout the paper we will assume that $\alpha$ and $\beta$ are independent of $n$, whereas we will choose small constants $\eta,\rho>0$ and set $\varepsilon=\mu^{-1+\eta}$ and $L=\exp\{\rho\varepsilon n/\log^{2}\mu\}$, i.e., $\varepsilon$ and $L$ may depend of $n$, since we also allow $\mu$ to depend on $n$.888In the papers [5, 6, 11] the parameter $L$ was replaced by a constant parameter $\rho$ such that $L=e^{\rho n}$. This had the advantage that their parameters were all independent of $n$, but since our parameters depend on $n$ anyway, it is more convenient to use the parameter $L$. However, both versions are equivalent.

2.4 Tools

To obtain good tail bounds, we often apply Chernoff’s inequality.

Theorem 2** (Chernoff Bound [18]).**

Let $Y_{1},\ldots,Y_{m}$ be independent random variables (not necessarily i.i.d.) that take values in $[0,1]$. Let $S:=\sum_{i=1}^{m}Y_{i}$, then for all $0\leq\delta\leq 1$,

[TABLE]

and for all $\delta\geq 0$,

[TABLE]

Finally, for all $k\geq 2e\operatorname{\mathbb{E}}[S]$,

[TABLE]

In addition, we will need the following theorem to bound the sum of geometrically distributed random variables.

Theorem 3** (Theorem 1 in [19]).**

Let $Y_{j}$, $1\leq j\leq m$, be independent random variables following the geometric distribution with success probability $p_{j}$, and let $S:=\sum_{j=1}^{m}Y_{j}$. If $\sum_{j=1}^{m}p_{j}^{-2}\leq s<\infty$ then for any $\delta>0$,

[TABLE]

For $h:=\min\{p_{j}\mid j\in[m]\}$,

[TABLE]

The following lemma estimates useful probabilities, e.g. the probability to improve on the current hot topic.

Lemma 4**.**

Let $\alpha,c>0$ be constants. Consider a set $A\subseteq[n]$ of size $\alpha n$ where $n$ is large enough, and consider a search point $x\in\{0,1\}^{n}$.

The probability that the number of one-bits in $A$ does not decrease after a standard bit mutation with rate $c/n$ on $x$ can be bounded from below by $p_{R}=e^{-\alpha c}/2$. 2. 2.

The probability that a standard bit mutation with rate $c/n$ strictly increases the number of one-bits in $A$ has a lower bound $p_{L}=\varepsilon(x)\alpha ce^{-\alpha c}/2$ and an upper bound $p_{U}=\varepsilon(x)\alpha c$, where $\varepsilon(x)=d(A,x)$. 3. 3.

Let $(1-\varepsilon^{\prime})\alpha n\leq i\leq\alpha n$ where $0<\varepsilon^{\prime}<1$ and $\varepsilon^{\prime}n\geq 2ec$. Let $\operatorname{rk}(x):=|\{j\in A\mid x_{j}=1\}|$ and let $y$ be an offspring of $x$. If $\operatorname{rk}(x)<i$, then at least one of the following inequalities holds.

[TABLE]

Proof of Lemma 4.

We show the statements one by one.

One way of creating an offspring with the same number of one-bits in $A$ is to flip no bits at all in $A$. This probability is $(1-c/n)^{\alpha n}=e^{-\alpha c}-O(1/n)\geq e^{-\alpha c}/2$ when $n$ is large enough. 2. 2.

We observe that the probability we consider is at least

[TABLE]

And it is at most

[TABLE]

where the second inequality follows from a union bound over all zero-bits in $A$. 3. 3.

Assume first that $\operatorname{rk}(x)<(1-2\varepsilon^{\prime})\alpha n$. Then for $\operatorname{rk}(y)\geq i$, at least $\varepsilon^{\prime}\alpha n$ zero-bits must be flipped in one mutation. The expected number of flipped zero-bits is at most $\alpha n\cdot c/n=\alpha c$, so that happens with probability $2^{-\varepsilon^{\prime}\alpha n}$ by the Chernoff bound. So let us consider the other case, $\operatorname{rk}(x)\geq(1-2\varepsilon^{\prime})\alpha n$. Let $P$ be a permutation on the $\alpha n$ bits in $A$ such that $P(j)<P(j^{\prime})$ for all $x_{j}=1$ and $x_{j^{\prime}}=0$. Consider mutating the bits in $x$ in the permuted order, and we track the number $G:=G_{0}-G_{1}$ during that process, where $G_{0}$ ($G_{1}$) is the number of flipped zero-bits (one-bits). Clearly, $G$ will be decreasing while we are at the one-bits and increasing afterwards. Then $\operatorname{rk}(y)\geq i$ if and only if $G\geq i-\operatorname{rk}(x)$ after flipping some zero-bit $j$, and $\operatorname{rk}(y)\geq i+1$ if and only if at least one more zero-bit is flipped after bit $j$. The number of remaining zero-bits is at most $\alpha n-\operatorname{rk}(x)-1<2\varepsilon^{\prime}\alpha n$, so the probability of flipping at least one remaining zero-bit is at most $2\varepsilon^{\prime}\alpha c$ by a union bound. Therefore,

[TABLE]

We will use the following two theorems to bound the running time of the $(\mu+1)$-EA. The first one states that a sequence of random variables whose differences are small with exponentially decaying tail bound are *sub-Gaussian.*999The reader can take the concept of being sub-Gaussian as a black box. Theorem 5 asserts that exponential tail bounds guarantee the property, Theorem 6 describes the consequences. For completeness, we also give the definition: a sequence of random variables $(Y_{i})_{i\geq 0}$ is $(c,\delta)$-sub-Gaussian if and only if $\operatorname{\mathbb{E}}[\exp(z(Y_{i+1}-Y_{i}))\mid Y_{0},\ldots,Y_{i}]\leq\exp(z^{2}c/2)$holds for all $i\geq 0$ and $z\in[0,\delta]$..

Theorem 5** (Timo Kötzing, Theorem 10 in [20]).**

Let $(Y_{i})_{i\geq 0}$ be a supermartingale such that there are $c^{\prime}>0$ and $\delta^{\prime}$ with $0<\delta^{\prime}<1$ and, for all $i\geq 0$ and for all $y\geq 0$,

[TABLE]

Then $(Y_{i})_{i\geq 0}$ is $(128c^{\prime}\delta^{\prime-3},\delta^{\prime}/4)$-sub-Gaussian.

The other theorem bounds first hitting times of sub-Gaussian supermartingales.

Theorem 6** (Timo Kötzing, Theorem 12 in [20]).**

Let $(Y_{i})_{i\geq 0}$ be a sequence of random variables and let $r\in\mathbb{R}$. If, for all $i\geq 0$,

[TABLE]

then $(Y_{i}-ri)_{i\geq 0}$ is a supermartingale. If further $(Y_{i}-ri)_{i\geq 0}$ is $(c^{\prime\prime},\delta^{\prime\prime})$-sub-Gaussian, then, for all $i\geq 0$ and all $y>0$,

[TABLE]

3 Formal Statement of the Result

The main result of this paper is the following.

Theorem 7**.**

For every constant $c>0$ and $0<\beta<\alpha<1$ there exist constants $\mu_{0}=\mu_{0}(c)\in\mathbb{N}$ and $\eta,\rho>0$ such that the following holds for all $\mu_{0}\leq\mu\leq n$ where $n$ is sufficiently large. Consider the $(\mu+1)$-EA with population size $\mu$ and mutation rate $c/n$ on the $n$-bit HotTopic function $\textsc{HT}_{n,\alpha,\beta,\varepsilon,L}$, where $\varepsilon=\mu^{-1+\eta}$ and $L=\lfloor\exp\{\rho\varepsilon n/\log^{2}\mu\}\rfloor$. Then with high probability the $(\mu+1)$-EA visits every level of the HT function at least once. In particular, it needs at least $L$ steps to find the optimum, with high probability and in expectation.

That is, if $\mu\geq\mu_{0}$ is a constant (independent of $n$) then with high probability the optimization time is exponential.

We remark that the requirement $\mu\leq n$ is not tight, and we conjecture that the runtime is always superpolynomial for $\mu\geq\mu_{0}$, also for much larger values of $\mu$. However, we did not undertake big efforts to extend the range of $\mu$ since we do not feel that it adds much to the statement. For larger values of $\mu$, e.g., $\mu=n^{2}$, our proof does not go through unmodified. With our definition of $\varepsilon=\mu^{-1+\eta}$, we only get error probabilities of the form $\exp\{-\Omega(\varepsilon n/\log^{2}\mu)\}$, which are not $o(1)$ if e.g. $\mu=n^{2}$. Hence we would need to choose larger values of $\varepsilon$, and then we lose a very convenient property, namely that for every fixed $i$, with high probability no individual of rank at most $i-1$ creates an individual of rank at least $i+1$. To avoid these complications, we only consider $\mu\leq n$.

4 Proof Overview

The next three sections are devoted to proving Theorem 7. The key ingredient is to analyze the drift of the density $d([n],x)$ for search points $x$ which have roughly density $\varepsilon$. We start by giving an informal overview, and by discussing similarities and differences to the situation in  [5] and [6].

We will analyze the algorithm in the regime where the fittest search point $x^{*}$ in the population satisfies

[TABLE]

where $\ell=\ell(x^{*})$ is the current level and $\varepsilon=\mu^{-1+\eta}$ is the parameter of the HotTopic function. It will turn out that for large $\mu$, the algorithm already needs stretched-exponential time to escape this situation.

The main idea is similar to [5, 6], in which the $(1+1)$-EA and other algorithms were analyzed. We first sketch the main argument for the $(1+1)$-EA, and explain afterwards which parts must be replaced by new arguments. The crucial ingredient is that while the density $d(A_{\ell+1},x)$ of zero-bits on the hot topic decreases from $2\varepsilon$ to $\varepsilon$, the total density $d([n],x)$ has a positive drift, i.e., a drift away from the optimum. Moreover, the probability to change $k$ bits in one step has a tail that decays exponentially with $k$. Therefore, it was shown that with high probability $d([n],x)$ stays above $\varepsilon+\gamma$ for an exponential number of steps, where $\gamma$ is a small constant. Then it was argued that as long as $d([n],x)$ stays bounded away from $\varepsilon$, it is exponentially unlikely that the level ever increases by more than one. Since there are an exponential number of levels, this implies an exponential runtime.

The analysis of $(\mu+1)$-EA and $(\mu+1)$-GA for constant $\mu$ in [6] was obtained by reducing it to the analysis of a related $(1+1)$ algorithm. This was possible since the choice of parameters in [6] (choosing the parameter $\varepsilon=\varepsilon(\mu)$ sufficiently small) made the algorithm operate close to the optimum. In this range, there are only few zero-bits, and thus it is rather unlikely that a mutation improves the fitness. On the other hand, there is always a constant probability (if $\mu$ is constant) to create a copy of the fittest individual. In such a situation, the population degenerates frequently into a collection of copies of a single search point. Thus, the population-based algorithms behave similarly to a $(1+1)$ algorithm. This $(1+1)$ algorithm has essentially the same mutation parameter as the $(\mu+1)$-EA, while for the $(\mu+1)$-GA it has a much smaller mutation parameter (less than one), which is the reason why the $(\mu+1)$-GA is efficient on all HotTopic instances with small parameter $\varepsilon$. For us, the situation is more complex since we consider larger values of $\varepsilon$. As a consequence, it is easier to find a search point with better fitness, and the population does not collapse. Hence, it is not possible to represent the population by a single point.

Instead, we proceed as follows. Fix a fitness level $\ell$, and consider the auxiliary fitness function

[TABLE]

We will first study the behavior of the $(\mu+1)$-EA on $f_{\ell}$. Considering this fitness function is essentially the same as assuming that the level remains the same. We will see in the end that this assumption is justified, by the same arguments as in [5, 6]. For a search point $x$, we define the rank $\operatorname{rk}(x):=\left|\{j\in A_{\ell+1}\mid x_{j}=1\}\right|$ of $x$ as the number of correct bits in the current hot topic. Note that by construction of $f_{\ell}$, a search point with higher rank is always fitter than a search point with smaller rank.

Now we define $\mathcal{X}_{i}$ to be the set of search points of rank $i$ that are visited by the $(\mu+1)$-EA, and we define $Z_{i}$ to be the OneMax-value (the number of one-bits) of the last search point in $\mathcal{X}_{i}$ that the algorithm deletes from its population. Note that due to elitist selection, this search point is also (one of) the fittest search point(s) in $\mathcal{X}_{i}$ that the algorithm ever visits, and hence it has the largest OneMax-value among all search points in $\mathcal{X}_{i}$ that the algorithm ever visits. Then our goal is to show that $\operatorname{\mathbb{E}}[Z_{i+1}-Z_{i}]=-\Omega(1)$, under the assumption that the population satisfies (3), i.e., that the density of the fittest search point is close to $\varepsilon$. This assumption can be justified by a coupling argument as in [5, 6]. Computing the drift of $Z_{i}$ is the heart of our proof, and the main technical contribution of this paper. In fact, to simplify the analysis we only prove the slightly weaker statement that $\operatorname{\mathbb{E}}[Z_{i+K}-Z_{i}]=-\Omega(1)$ for a suitable constant $K$, which is equally suited. Once we have established this negative drift, the remainder of the proof as in [5, 6] carries over almost unchanged.

To estimate the drift $\Delta:=\operatorname{\mathbb{E}}[Z_{i+K}-Z_{i}]$, we will assume for this exposition that $\mu=\omega(1)$, so that we may use $O$-notation. (In the formal proof we will use the weaker assumption $\mu\geq\mu_{0}$ for a sufficiently large constant $\mu_{0}=\mu_{0}(c)$.) We distinguish between good and bad events. Good events will represent the typical situation; they will occur with high probability, and if they occur $K$ times in a row, then it will deterministically follow that $Z_{i+K}-Z_{i}\leq-\log\mu$. On the other hand, bad events may lead to a positive difference, but they are unlikely and thus they contribute only a lower order term to the drift. We will discriminate two types of bad events. Firstly, we will show that the probability $\Pr[Z_{i+K}-Z_{i}>\lambda\log\mu]$ drops exponentially in $\lambda$. This implies that the events in which $Z_{i+K}-Z_{i}>\log^{2}\mu$ contribute at most a term $o(1)$ to the drift. Hence, we can restrict ourselves to the case that $Z_{i+K}-Z_{i}\leq\log^{2}\mu$. Now assume that we have any event of probability $o(\log^{-2}\mu)$. In the case $Z_{i+K}-Z_{i}\leq\log^{2}\mu$, this event can contribute at most a $o(1)$ term to the drift. Hence, we may declare any such event as a bad event, and conclude that all bad events together only contribute a $o(1)$ term to the drift.

As we have argued, we may neglect any event with probability $o(\log^{-2}\mu)$. This is a rather large error probability, which allows us to dub many events as ‘bad’, and to use rather coarse estimates on the error probability. We conclude this overview by describing how a good event, and thus a typical situation, looks like. In what follows, all claims hold with probability at least $1-o(\log^{-2}\mu)$.

Let us call $t_{i}$ the first round in which an individual of rank at least $i$ is created, and $T_{i}$ the round in which the last individual of rank at most $i$ is eliminated. Then typically $T_{i}-t_{i}=O(\mu\log\mu)\cap\Omega(\mu)$. Let $\left|X_{i}\right|=\left|X_{i}(t)\right|$ denote the number of search points in the population of rank $i$ at time $t$. We want to study the family forest $F_{i}$ of $X_{\geq i}$, which is closely related to the family trees and family graphs that have been used in other work on population-based EAs, e.g. [1, 2, 21, 22]. The vertices of this forest are all individuals of rank at least $i$ that are ever included into the population. A vertex is called a root if its parent has rank less than $i$. Otherwise, the forest structure reflects the creation of the search points, i.e., vertex $u$ is a child of vertex $v$ if the individual $u$ was created by a mutation of $v$.

As $X_{i}$ grows, eventually the first few search points of rank $i+1$ are created, and form the first roots of the family forest. Then the forest starts growing, both because new roots may appear and because the vertices in the forest may create offspring. At some point we have $|X_{i+1}|=\mu^{\delta}$ for some (suitably small) $\delta>0$. At this point, we still have typically $|X_{i}|=O(\mu^{\delta}/\varepsilon)=O(\mu^{1+\delta-\eta})=o(\mu)$, where the latter holds if $\delta$ is small enough. Moreover, at this point there are no search points of rank strictly larger than $i+1$. The sets $X_{i}$ and $X_{i+1}$ both continue to grow with roughly the same speed until the search points of rank at most $i-1$ are eliminated from the population. Afterwards, the search points of rank $i$ are eliminated from the population, until only search points of rank at least $i+1$ remain. Crucially, up to this point every search point of rank at least $i+1$ is accepted into the population. In other words, there is no selective pressure on the search points of rank $i+1$, and every mutation of a search point of rank $i+1$ enters the family tree, as long as the rank $i+1$ is preserved. Therefore, we can contain the family forest $F_{i+1}$ of rank $i+1$ up to this point in a random forests $F^{\prime}$ which is obtained by certain forest growth processes in which no vertex is ever eliminated and all vertices continue to spawn offspring with a fixed rate.

We want to understand the set of individuals in $X_{i+1}$ that spawn offspring in $X_{i+2}$, and thus spawn the roots for the family forest $F_{i+2}$. As before we can argue that no individuals of rank at least $i+2$ are created before the family forest of rank $i+1$ reaches size $\mu^{\delta}$. Moreover, we can show that the time $T_{i+1}$ at which all individuals of rank $i+1$ are eliminated from the population satisfies $T_{i+1}-t_{i+1}\leq C\mu\log\mu$ for a suitable constant $C>0$. Hence, $F_{i+1}$ is bounded from above by the random forest $F^{\prime}$ at time $t_{i+1}+C\mu\log\mu$. This forest is only polynomially large in $\mu$.

The recursive trees that we use to bound $F_{i+1}$ are well understood, see also Figure 1. In particular, it is known that even in $F^{\prime}$ only a small fraction $\mu^{\delta}$ of the vertices are in depth at most $\phi\log\mu$, where $\delta,\phi>0$ are suitable constants. Since each such vertex creates an offspring of strictly larger rank with probability $\varepsilon/\mu$ per round, the expected number of offspring of rank $i+2$ of these vertices is at most $O(\mu^{\delta}\varepsilon/\mu\cdot(T_{i+1}-t_{i+1}))$. With the right choice of parameters, this is $\mu^{-\Omega(1)}$, and we may conclude that no vertices of depth at most $\phi\log\mu$ create roots of rank $i+2$. On the other hand, since we do not truncate any vertices in the creation of $F^{\prime}$, they are obtained from their parents by unbiased mutations of $[n]\setminus A_{\ell}$, and we can show that most (all but at most $\mu^{\delta}$) vertices of depth at least $\phi\log\mu$ in $F^{\prime}$ have accumulated $c^{\prime}\log\mu$ more bad than good bit-flips when compared to their roots, for a suitable $c^{\prime}>0$. For the $\mu^{\delta}$ exceptional vertices, none of them will create a root of rank $i+2$ in $T_{i+1}-t_{i+1}$ rounds, even if they are in $F_{i+1}$.

To summarize, good events consist of the following four main points. Firstly, no vertex of rank at most $i$ creates an offspring of rank at least $i+2$. Secondly, every vertex in $X_{i+1}$ that creates an offspring in $X_{i+2}$ has at least depth $\phi\log\mu$ in the family forest. Thirdly, every vertex in $X_{i+1}$ of depth at least $\phi\log\mu$ that creates an offspring in $X_{i+2}$ has a OneMax value that is at least $c^{\prime}\log\mu$ smaller than that of its root. Finally, we also require that no vertex in $F^{\prime}$ exceeds the OneMax value of its root by more than $C\log\mu$, for some $C>0$. The complete list in the proof contains even more requirements, but these four already imply a decline in $Z_{i}$ if they hold over $K$ consecutive steps. In this case, inductively the OneMax values of all roots in $F_{i+K}$ are at most $Z_{i}-Kc^{\prime}\log\mu$. Moreover, $Z_{i+K}$ exceeds the OneMax value of the corresponding root in $X_{i+K}$ by at most $C\log\mu$, so we have $Z_{i+K}\leq Z_{i}-Kc^{\prime}\log\mu+C\log\mu$. Choosing $K$ sufficiently large shows that $Z_{i}$ must decrease in these typical situations.

5 Drift of $Z_{i}$

In this main section of the proof, we show that the random variable $Z_{i}$ has negative drift. We will use the same notation as in the proof outline. In particular, $X_{i}$ denotes the set of all search points of rank $i$ that the algorithm visits, and $Z_{i}$ denotes the OneMax-value of the last search point from $X_{i}$ that the algorithm keeps in its population. If $X_{i}$ is empty (which, as we will see, is very unlikely), then we set $Z_{i}:=Z_{i-1}$. Moreover, we define $X_{\geq i}:=\bigcup_{i^{\prime}\geq i}X_{i^{\prime}}$, and the definition of terms like $X_{>i}$ is analogous. For a given parent individual $x$, we denote by $p_{I}$ (by $p_{R}$) the probability that an offspring of $x$ has rank which is strictly larger than (at least as large as) the rank of $x$.

Throughout this section, we fix a level $\ell$ and consider the $(\mu+1)$-EA on the linear function $f_{\ell}$ defined in (4). In this section, we will study the case that $i\in[(1-2\varepsilon)\alpha n,(1-\varepsilon/2)\alpha n]$, where $\varepsilon=\mu^{-1+\eta}$. Note that this is a weaker form of Condition (3), i.e., we consider search points for which the density in $A$ is close to $\varepsilon$.

5.1 Preliminaries

In this section we first give bounds on the time that the set $X_{\geq i}$ needs to grow from size $1$ to size $\mu^{\kappa}$, and we will conclude that $X_{\geq i}$ is large at the latter point in time. We start by bounding the time.

Lemma 8**.**

For all $0<\alpha<1$, $c>0$, $0<\eta<\kappa\leq 1$, there exists a constant $\mu_{0}$ such that the following holds for all $\mu_{0}\leq\mu\leq n$. Let $i>(1-2\varepsilon)\alpha n$, where $\varepsilon=\mu^{-1+\eta}$. Consider the $(\mu+1)$-EA with mutation rate $c/n$ on the linear function $f_{\ell}$. Denote by $T_{i}^{\kappa}=T^{\kappa}$ the number of rounds until $|X_{\geq i}|$ reaches $\mu^{\kappa}$ after the algorithm visits the first point $x^{i}$ in $X_{\geq i}$. With probability $1-2\mu^{-\Omega(1)}$,

[TABLE]

Moreover,

[TABLE]

Proof.

By the definition of $f_{\ell}$, all individuals in $X_{\geq i}$ are fitter than those in $X_{<i}$. So no points in $X_{\geq i}$ will be discarded until $X_{<i}$ becomes empty, and we are interested in the growth of $|X_{\geq i}|$ during this period. Let $T_{j}$ be the time needed for $|X_{\geq i}|$ to grow from $j$ to $j+1$. By definition we have $T^{\kappa}=\sum_{j=1}^{\mu^{\kappa}-1}T_{j}$. Denote by $x^{t}$ the point selected as parent by the algorithm in round $t$ and denote by $y^{t}$ its offspring. The probability that both $x^{t}$ and $y^{t}$ belong to $X_{\geq i}$ is at least $p_{j}=j/\mu\cdot p_{R}$, where $j$ is the size of $X_{\geq i}$ at the beginning of round $t$ and $p_{R}=e^{-\alpha c}/2$ is defined in Lemma 4.1. It is clear that we can dominate $T_{j}$ by random variable $\bar{T}_{j}$ that follows a geometric distribution with parameter $p_{j}$. By Lemma 1.8.8 in [23], $T^{\kappa}$ is dominated by $\bar{T}^{\kappa}:=\sum_{j=1}^{\mu^{\kappa}-1}\bar{T}_{j}$. Next we apply Theorem 3 to bound $\bar{T}^{\kappa}$ from above.

The expectation of $\bar{T}^{\kappa}$ is

[TABLE]

For the Harmonic series, we have $\log(m+1)<\sum_{j=1}^{m}1/j\leq\log m+1$, where $\log$ denotes the natural logarithm. Therefore, for large enough $\mu$,

[TABLE]

Let $h:=\min\{p_{j}\mid j=1,\ldots,\mu^{\kappa}-1\}$, clearly $h=p_{1}=e^{-\alpha c}/(2\mu)$. Let $s:=\sum_{j=1}^{\mu^{\kappa}-1}p_{j}^{-2}$, we have

[TABLE]

where the last step follows from $\sum_{j=1}^{\infty}1/j^{2}=\pi^{2}/6$. Given $h$ and the bound on $s$, by Theorem 3 it holds for $\delta=\kappa e^{\alpha c}\mu\log\mu$ that

[TABLE]

Since $T^{\kappa}\preceq\bar{T}^{\kappa}$, together with equation (5) we conclude that $T^{\kappa}\leq 4\kappa e^{\alpha c}\mu\log\mu$ with probability $1-\mu^{-\Omega(1)}$.

We still need a lower bound of $T^{\kappa}$. Consider the probability that $X_{\geq i}$ gets a new offspring $y^{t}$ in a round where $|X_{\geq i}|=j$:

[TABLE]

where $p_{U}$ is defined in Lemma 4.2. Let $p_{j}^{\prime}=j/\mu+p_{U}$, similarly as for the upper bound on $T^{\kappa}$, we can subdominate $T^{\kappa}$ with a random variable $\hat{T}^{\kappa}=\sum_{j=1}^{\mu^{\kappa}-1}\hat{T}_{j}$ (Lemma 1.8.8 in [23]), where the $\hat{T}_{j}$ are independent and geometrically distributed with parameter $p_{j}^{\prime}$, respectively. Then

[TABLE]

Since $i\geq(1-2\varepsilon)\alpha n$, $P_{U}=O(\varepsilon)=O(\mu^{-1+\eta})$ for $0<\eta<\kappa$. So $\lceil\mu p_{U}\rceil=O(\mu^{\eta})$. Hence,

[TABLE]

Let $s^{\prime}:=\sum_{j=1}^{\mu^{\kappa}}p_{j}^{\prime-2}$. As $p_{j}^{\prime}>p_{j}$, it holds $s^{\prime}<s$ that. Applying Theorem 3 with $s^{\prime}$ and $\delta^{\prime}=\varepsilon^{\prime}\mu\log\mu$, we obtain

[TABLE]

Similarly, we have $\hat{T}^{\kappa}\preceq T^{\kappa}$, by picking a sufficiently small $\varepsilon^{\prime}$ we conclude that

[TABLE]

with probability $1-\mu^{-\Omega(1)}$. ∎

In the following lemma, we give a lower bound on $|X_{\geq i+1}|$ when $X_{\geq i}$ reaches a certain size.

Lemma 9**.**

Let $\alpha,\kappa\in(0,1)$, $c>0$, $\eta<1$ be constants such that $\kappa>1-\eta/2$. Consider the $(\mu+1)$-EA with $\mu\leq n$ and mutation rate $c/n$ on the linear function $f_{\ell}$. Let $\varepsilon=\mu^{-1+\eta}$ and let $i\leq(1-\varepsilon/2)\alpha n$. Denote by $Y_{i+1}^{\kappa}=Y^{\kappa}$ the size of $X_{\geq i+1}$ when $|X_{\geq i}|$ reaches $\mu^{\kappa}$. Then with probability 1-\exp\big{(}-\Omega\big{(}\mu^{2(\kappa-1)+\eta}\big{)}\big{)},

[TABLE]

Proof.

Note that we may assume that $\mu\geq\mu_{0}$ for a constant $\mu_{0}$ of our choice, since otherwise the probability may be zero and thus the statement is vacuous. In each round $|X_{\geq i}|$ increases by either 0 or 1, so after $X_{\geq i}$ reaches size $R:=\lfloor\mu^{\kappa}/2\rfloor$ there are at least $R$ more rounds until $|X_{\geq i}|=\mu^{\kappa}$. In each of the remaining $R$ rounds, the probability of a parent $x\in X_{\geq i}$ being selected and its offspring $y$ belonging to $X_{>i}$ is at least

[TABLE]

where $P_{L}$ is defined in Lemma 4. Let $Y_{j}$ be independent Bernoulli variables with parameters $R/\mu\cdot p_{L}$ for $j\in[R]$. Then $Y^{\kappa}$ dominates the sum of $Y_{j}$, i.e. $Y^{\kappa}\succeq\bar{Y}^{\kappa}:=\sum_{j=1}^{R}Y_{j}$. It holds that

[TABLE]

By Chernoff’s inequality (Theorem 2), we have for any constant $0<\delta<1$,

[TABLE]

The claim follows from $Y^{\kappa}\succeq\bar{Y}^{\kappa}$. ∎

5.2 Tail Bounds

In this section, we will give rather loose tail bounds to show that it is unlikely that $Z_{i}$ is much larger than $Z_{i-1}$. All constants in this section are independent of $\mu$. This includes all hidden constants in the $O$-notation.

5.2.1 Tail Bound on the Lifetime of $X_{i}$

As before, let $t_{i}$ be the first round in which an individual of rank at least $i$ is created, and let $T_{i}$ be the round in which the last individual of rank at most $i$ is eliminated.

Lemma 10**.**

For all $0<\alpha,\eta<1$, $c>0$, there is a constant $\mu_{0}\in\mathbb{N}$ such that the following holds for all $\mu_{0}\leq\mu\leq n$. Let $i\in[(1-2\varepsilon)\alpha n,(1-\varepsilon/2)\alpha n]$, where $\varepsilon=\mu^{-1+\eta}$. Consider the $(\mu+1)$-EA with mutation rate $c/n$ on the linear function $f_{\ell}$. Then with probability at least $1-\mu^{-\Omega(1)}$, $T_{i}-t_{i}\leq 8e^{\alpha c}\mu\log\mu$. Moreover, for all $\beta\geq 1$ and $C=16e^{\alpha c}$,

[TABLE]

Proof.

We first show that $\Pr[T_{i}-t_{i}\geq C^{\prime}\mu\log\mu]\leq 1/2$ for a suitable constant $C^{\prime}>0$. Let $x^{\geq i}$ be the first individual of rank at least $i$ and let $x^{j}$ with rank $j$ be the first individual of rank strictly larger than $i$. We can divide the process from $t_{i}$ to $T_{i}$ into two parts. The first part ends when $x^{j}$ is created, and we denote by $t_{j}$ the round when this happens. The second part starts after $t_{j}$ and ends when $X_{>i}$ reaches size $\mu$. Since we are proving an upper bound of the tail, we can consider the second part ends when $X_{\geq j}$ reaches $\mu$ for simplicity.

If $x^{\geq i}=x^{j}$, then we have $t_{j}=t_{i}$, namely the first part does not exist. So for the tail bound of the first part, we may assume that $x^{\geq i}\in X_{i}$. By Lemma 8, for some $1-\eta/2<\kappa<1$, we have $|X_{\geq i}|\geq\mu^{\kappa}$ at time $T:=t_{i}+4\kappa e^{\alpha c}\mu\log\mu$. By Lemma 9 we have $|X_{>i}|>0$ at this point, so $x_{j}$ must have been created before time $T$. For the second part, we apply Lemma 8 again for $X_{\geq j}$. By time $t_{j}+4e^{\alpha c}\mu\log\mu$, $X_{\geq j}$ reaches size $\mu$.

To summarize, we have applied Lemma 8 twice and Lemma 9 once. Therefore, with probability at least $1-5\mu^{-\Omega(1)}$, $T_{i}-t_{i}\leq 8e^{\alpha c}\mu\log\mu$. Since $\mu\geq\mu_{0}$, for large enough $\mu_{0}$ we obtain $\Pr[T_{i}-t_{i}\geq C^{\prime}\mu\log\mu]\leq 1/2$ for $C^{\prime}=8e^{\alpha c}$.

To conclude the proof, we set $C:=2C^{\prime}$. Then for all integral $\beta^{\prime}\in\mathbb{N}$ we consider $\beta^{\prime}$ phases and repeat the same argument. This shows $\Pr[T_{i}-t_{i}\geq\beta^{\prime}\cdot C^{\prime}\mu\log\mu]\leq 2^{-\beta^{\prime}}$. Hence, for $C=16e^{\alpha c}$ it holds for all $\beta\geq 1$,

[TABLE]

5.2.2 Family Forests

From now on we will be mostly working on family forests, so we introduce the definition and several related lemmas here. The main idea is to couple the algorithm with a process that is not subject to selection. This idea has been used before to analyze population-based algorithms [1, 2, 21, 22].

We denote the family forest for search points with rank at least $i$ by $F_{i}$. The vertex set of $F_{i}$ are the vertices in $X_{\geq i}$ that are (once) in the population, while the roots of the trees are vertices whose parents are in $X_{<i}$. Moreover, any path connecting a root and a vertex in $F_{i}$ corresponds to a series of mutations that create this vertex. Note that the size of $F_{i}$ increase over time.

As analysing $F_{i}$ directly can be complicated, we couple it with a simpler random forest $F^{\prime}$, which is generated by the following process. In round 0 there is a single root in $F^{\prime}$. In each subsequent round, each vertex in $F^{\prime}$ creates a new child with probability $1/\mu$ and a new root is added. Lemma 11 shows that $F_{i}$ can be coupled to a subgraph in $F^{\prime}$.

Lemma 11**.**

The family forest $F_{i}$ can be coupled to $F^{\prime}$ such that $F^{\prime}$ contains $F_{i}$ as a subgraph at any round.

Proof.

Throughout the coupling process we maintain that $F_{i}$ is a subgraph of $F^{\prime}$. The first point $x^{\geq i}$ that the algorithm visits in $X_{\geq i}$ (in round $t_{i}$) corresponds to the only root $r_{0}$ in round 0 in $F^{\prime}$. In every round $t>t_{i}$, a point $x^{t}$ in the current population is selected to create an offspring $y^{t}$. For each $x\in F_{i}$, if $x^{t}=x$ (which happens with probability $1/\mu$ if $x$ is still in the current population, and with probability zero otherwise) then we attach a child to $x$ in $F^{\prime}$: if $y^{t}\in X_{\geq i}$ then we attach $y^{t}$ to $x$ in $F^{\prime}$, otherwise we attach a dummy child to $x$ in $F^{\prime}$. In this case, we still associate the offspring with the dummy child, and in our upcoming considerations we will ignore that this search point does belong to $X_{\geq i}$. If $x^{t}$ is not in $X_{\geq i}$ while $y^{t}$ is, we add $y^{t}$ as a new root $r_{t}$ to $F^{\prime}$, otherwise we add a new dummy root to $F^{\prime}$. For every node $x\in F^{\prime}$ that is a dummy node (that has no corresponding node in $F$) or whose copy in $F$ has been removed from the population, we add another dummy node as its child with probability $1/\mu$. In this way, for each vertex in $F^{\prime}$ we create a new child with probability $1/\mu$ and a root is added in each round. On the other hand, by construction, $F_{i}$ is a subgraph of $F^{\prime}$ at all times. ∎

Note that the search points associated with the vertices in $F^{\prime}$ are obtained from the root by mutation only, without any interfering selection step. This makes the process easy to analyze. Such a selection-free mutation process has been analyzed before, e.g. [24]. In Lemma 12 we show several useful properties of $F^{\prime}$. Due to the coupling from $F_{i}$ to $F^{\prime}$, the properties will also hold for $F_{i}$ as well.

Lemma 12**.**

$F^{\prime}$* satisfies the following properties:*

Let $s_{t}$ denote the number of vertices in $F^{\prime}$ in round $t$, then $\Pr[s_{t}\geq S]\leq te^{t/\mu}/S$ for all $S>0$. 2. 2.

Let $x$ be a search point that corresponds to a vertex in $F^{\prime}$ of depth at most $d$ with root $y$. Then for $k\geq 2edc$,

[TABLE] 3. 3.

Let $x$ be a search point that corresponds to a vertex in $F^{\prime}$ of depth larger than $d$ with root $y$. If $n$ is sufficiently large and $\textsc{Om}(y)\geq(1-8\varepsilon)n$ then

[TABLE]

and

[TABLE]

If $n$ is sufficiently large and $\textsc{Om}(y)\leq(1-8\varepsilon)n$ then $\Pr[\textsc{Om}(x)\geq(1-4\varepsilon)n]\leq 2\cdot 2^{-\varepsilon n}$. 4. 4.

Let $s_{t}^{d}$ denote the number of vertices of depth $d$ in round $t$ for an arbitrary tree from $F^{\prime}$. Then

[TABLE]

In particular, for $t=O(\mu\log\mu)$ the depth of the tree is at most $e\log\mu$ with probability $1-\mu^{-\Omega(1)}$. Moreover, if $t\geq 2d\mu$, $\sum_{i=0}^{d}\operatorname{\mathbb{E}}[s_{t}^{i}]\leq 2t^{d}/(d!\mu^{d})$.

Proof.

We prove the statements one by one.

In $t$ rounds we have added $t$ roots to the forest, and we will give a uniform bound for all of them. So we fix a root and denote by $\sigma_{\tau}$ the number of vertices in this tree in round $\tau$, where $0\leq\tau\leq t$. We assume pessimistically that the root is introduced in round [math]. Then we have $\sigma_{0}=1$ and $\operatorname{\mathbb{E}}\left[\sigma_{\tau+1}\mid\sigma_{\tau}\right]=(1+1/\mu)\sigma_{\tau}$ for $0\leq\tau\leq t-1$. By linearity of expectation, we have $\operatorname{\mathbb{E}}[\sigma_{t}]\leq(1+1/\mu)^{t}$. Since there are $t$ roots, and using that $(1+1/\mu)^{\mu}\leq e$, we obtain

[TABLE]

By Markov’s inequality, it holds that

[TABLE] 2. 2.

Let $y_{i}$ be the $i$-th bit in $y$, the event $y_{i}\neq x_{i}$ implies that the $i$-th bit is flipped at least once. Denote by $d^{\prime}\leq d$ the distance between $x$ and $y$. By a union bound

[TABLE]

Let $D=|\{i\in[n]\mid y_{i}\neq x_{i}\}|$ be the number of bits in which $y$ and $x$ differ. Then its expectation is $\operatorname{\mathbb{E}}\left[D\right]\leq dc$. Since the bits are modified independently, we can apply Chernoff’s inequality (Theorem 2) for $k\geq 2edc\geq 2e\operatorname{\mathbb{E}}[D]$ , and obtain

[TABLE] 3. 3.

Let the depth of $x$ be $d^{\prime}\geq d$. First we argue that we may assume $d^{\prime}\leq n/(16ec)$. If $d^{\prime}\geq n/(16ec)$, then consider just the last $n/(16ec)$ steps. In these, every bit has a constant probability to be touched exactly once, and a constant probability not to be touched at all. If the number of one-bits before the last $n/(16ec)$ steps was at least $n/2$, then with probability $1-e^{-\Omega(n)}$, $x$ has at least $8\varepsilon n$ zero-bits as each one-bit has a constant probability of being flipped exactly once in those steps, and if the number of one-bits was at most $n/2$, $x$ has also at least $8\varepsilon n$ zero-bits as each zero-bit has a constant probability of being untouched. In either case, $x$ has more zero-bits than $y$ with sufficiently large probability. So we may assume $d^{\prime}\leq n/(16ec)$.

We then consider the case $\textsc{Om}(y)\geq(1-8\varepsilon)n$. Let $B_{01}$ be the number of bits flipped from 0 to 1. Then similarly as for Property 2 we bound $\operatorname{\mathbb{E}}[B_{01}]$ by

[TABLE]

where the second inequality follows from a union bound. Similarly, let $B_{10}$ be the number of bits flipped from 1 to 0 in $d^{\prime}$ mutations, its expectation $\operatorname{\mathbb{E}}[B_{10}]$ is

[TABLE]

Since all bits contribute independently, we may apply the Chernoff bound. With probability at least $1-e^{-d^{\prime}c/32}$ each, we have $B_{01}\leq cd^{\prime}/8$ and $B_{10}\geq cd^{\prime}/8$. Both inequalities together imply that $\textsc{Om}(x)\leq\textsc{Om}(y)$ as desired, and the probability that at least one of the inequalities is violated is at most $2e^{-d^{\prime}c/32}\leq 2e^{-dc/32}$.

Similarly, the probability that $B_{01}$ ($B_{10}$) overshoot (undershoot) its expectation by more than $d^{\prime}c/16$ is at most $e^{-d^{\prime}c/128}$. Therefore, the probability that $B_{01}\geq B_{10}-d^{\prime}c/16$ is at most $2e^{-d^{\prime}c/128}\leq 2e^{-dc/128}$.

For the second statement, assume $\textsc{Om}(y)\leq(1-8\varepsilon)n$, and consider the first vertex $x^{\prime}$ on the path from $y$ to $x$ such that $\textsc{Om}(x^{\prime})\geq(1-6\varepsilon)n$. The probability that more than $\varepsilon n$ bits were flipped in the creation of $x^{\prime}$ is at most $2^{-\varepsilon n}$ by the Chernoff bound, since by definition of $x^{\prime}$ the parent of $x^{\prime}$ has an Om-value smaller than $(1-6\varepsilon)n$, we may assume that $\textsc{Om}(x^{\prime})\leq(1-5\varepsilon)n$. Then, starting from $x^{\prime}$ we may use the same calculation as above, only that we need to bound the probability that $\varepsilon n$ more zero-bits than one-bits are flipped. This is bounded by the probability that $B_{01}\geq\varepsilon n$. Since $d^{\prime}\leq n/(16ec)$ we have $\varepsilon n\geq 16\varepsilon cd^{\prime}\geq 2e\operatorname{\mathbb{E}}[B_{01}]$, by the Chernoff bound, this probability is at most $2^{-\varepsilon n}$. 4. 4.

There can only be one root in a tree, so $s_{t}^{0}=1$ for all $t\geq 0$. For $d\geq 1$ and $t\geq 1$, it holds that

[TABLE]

where $Y_{i}$ is an indicator variable that takes value 1 if the $i$-th vertex of depth $d-1$ creates a offspring in round $t$. By Wald’s equation, we obtain

[TABLE]

Plugging in $\operatorname{\mathbb{E}}[s_{t}^{d}]=0$ for all $t<d$, we can derive that

[TABLE]

for all $t\geq d\geq 1$.

We show the result by induction. For $d=1$, by equation (7) we have $\operatorname{\mathbb{E}}[s_{t}^{1}]=t/\mu$ for all $t\geq 1$. Now assume that $\operatorname{\mathbb{E}}[s_{t}^{d}]\leq t^{d}/(d!\mu^{d})$ for all $t\geq d$ where $d\geq 1$, again by equation (7) it holds that

[TABLE]

for all $t\geq d+1$.

Now consider $t=O(\mu\log\mu)$ and $d=k\log\mu$ for some constant $k>e$. With Stirling’s approximation $d!=(1+O(1/d))\sqrt{2\pi d}(d/e)^{d}$ and equation (4), we have

[TABLE]

By Markov’s inequality, $\Pr[s_{t}^{d}\geq 1]=O(\mu^{k(1-\log k)}\sqrt{\log\mu})=\mu^{-\Omega(1)}$ as $k(1-\log k)<0$. Therefore, with probability $1-\mu^{-\Omega(1)}$, $s_{t}^{d}=0$ for any $d>e\log\mu$, which implies that the depth of the tree is at most $e\log\mu$.

For the last statement, let $a_{t}^{d}:=t^{d}/(d!\mu^{d})$. If $t\geq 2d\mu$, $a_{t}^{d-1}/a_{t}^{d}=d\mu/t\leq 1/2$ for $d\geq 1$. Therefore,

[TABLE]

5.2.3 Tail Bound on Steps of $Z_{i}$

The first consequence of the coupling is an exponential tail bound on the difference $Z_{i}-Z_{i-1}$. Note that the tail bound only holds in one direction. There is no comparable tail bound for $Z_{i-1}-Z_{i}$, at least not without further knowledge on $X_{i-1}$: if there is a single search point $x\in X_{i-1}$ that has $k$ more one-bits than all other search points in $X_{i-1}$, then $x$ might not spawn an offspring and $Z_{i}$ could drop by $k$ or more, and $k$ could be as large as $\Omega(n)$ without assumptions on $X_{i-1}$.

Lemma 13**.**

For all $0<\alpha,\eta<1$, $c>0$ there is a constant $\mu_{0}\in\mathbb{N}$ such that the following holds for all $\mu_{0}\leq\mu\leq n$, where $n$ is sufficiently large. Let $i\in[(1-2\varepsilon)\alpha n,(1-\varepsilon/2)\alpha n]$, where $\varepsilon=\mu^{-1+\eta}$. Assume that the $(\mu+1)$-EA with mutation rate $c/n$ on the linear function $f_{\ell}$ satisfies $Z_{i-1}\geq(1-4\varepsilon)n$. Then for all $1\leq\beta\leq\varepsilon n/\log^{2}\mu$ and $C_{2}=6400e^{\alpha c+1}$,

[TABLE]

If on the other hand $Z_{i-1}<(1-4\varepsilon)n$, then $Z_{i}<(1-2\varepsilon)n$ with probability $1-e^{-\Omega(\varepsilon n/\log^{2}\mu)}$.

Proof.

By Lemma 10, there is $C=16e^{\alpha c}$ such that for all $\beta\geq 1$,

[TABLE]

By Lemma 12.1, at round $t=(\beta+2)\cdot C\mu\log\mu$ we have

[TABLE]

where the last step holds for all $\mu\geq\mu_{0}$ if $\mu_{0}$ is sufficiently large. That is, the probability that the algorithm visits at least $\mu^{2(\beta+2)C}$ vertices in $X_{\geq i}$ is at most $\tfrac{1}{4}2^{-\beta}$.

From now on, we consider $F^{\prime}$ at a time when it has at most $\mu^{2(\beta+2)C}$ vertices. Let $x$ be a search point that corresponds to a vertex in $F^{\prime}$ of depth at most $d=\beta C_{2}^{\prime}\log\mu$ with root $r$, where $C_{2}^{\prime}=200C/c$. By Lemma 12.2, for $C_{2}=400eC$ it holds for large enough $\mu_{0}$ that

[TABLE]

By a union bound over all vertices in $F^{\prime}$, the probability that there exists such a vertex $x$ among them is at most $\tfrac{1}{4}2^{-\beta}$.

Now let $x$ be a search point that corresponds to a vertex in $F^{\prime}$ of depth larger than $d$ with root $r$. For large enough $\mu_{0}$ by Lemma 12.3, if $\textsc{Om}(r)\geq(1-8\varepsilon)n$ then

[TABLE]

The probability that there exists such a vertex $x$ in $F^{\prime}$ is at most $\tfrac{1}{8}2^{-\beta}$ by a union bound. On the other hand, if $n$ is sufficiently large and $\textsc{Om}(r)\leq(1-8\varepsilon)n$ then for $\beta\leq\varepsilon n/\log^{2}\mu$,

[TABLE]

Similarly, the probability that such a vertex $x$ exists in $F^{\prime}$ is at most $\tfrac{1}{8}2^{-\beta}$.

To summarize, we have shown that each of the following four events happens with probability at least $1-1/4\cdot 2^{-\beta}$.

$\mathcal{E}_{1}$: $T_{i}-t_{i}<(\beta+2)C\mu\log\mu$.

• 2.

$\mathcal{E}_{2}$: $s_{t}<\mu^{2(\beta+2)C}$ at time $t=(\beta+2)C\mu\log\mu$.

• 3.

$\mathcal{E}_{3}$: Among the first $\mu^{2(\beta+2)C}$ vertices in $F^{\prime}$, there is no search point $x$ with a distance at most $\beta C_{2}^{\prime}\log\mu$ to its root $r$ such that $\left|\{i\in[n]\mid r_{i}\neq x_{i}\}\right|>\beta C_{2}\log\mu$.

• 4.

$\mathcal{E}_{4}$: Among the first $\mu^{2(\beta+2)C}$ vertices in $F^{\prime}$, there is no search point $x$ with a distance larger than $\beta C_{2}^{\prime}\log\mu$ to its root $r$ such that either $\textsc{Om}(r)\geq(1-8\varepsilon)n$ and $\textsc{Om}(x)>\textsc{Om}(r)$ or $\textsc{Om}(r)\leq(1-8\varepsilon)n$ and $\textsc{Om}(x)\geq(1-4\varepsilon)n$.

Now we argue how the bounds for these events imply the lemma. By $\mathcal{E}_{1}$ and $\mathcal{E}_{2}$, we may restrict ourselves to the first $\mu^{2(\beta+2)C}$ vertices in $F^{\prime}$. We claim that there are no offspring $x$ in distance at most $\beta C_{2}^{\prime}\log\mu-1$ from their root $r$ that have Om-value larger than $Z_{i-1}+\beta C_{2}\log\mu$. To see this, we add the parent of $r$, $r^{\prime}\in X_{<i}$, and the edge between $r^{\prime}$ and $r$ to $F^{\prime}$. Now $r^{\prime}$ is the root of $x$ and it can act as a reference point: by the definition of $Z_{i-1}$ we have $Z_{i-1}\geq\textsc{Om}(r^{\prime})$. If the distance from $r^{\prime}$ to $x$ is at most $\beta C_{2}^{\prime}\log\mu$, by $\mathcal{E}_{3}$ we have $\textsc{Om}(x)\leq\textsc{Om}(r^{\prime})+\beta C_{2}\log\mu$. If $x$ is of larger distance from the added root $r^{\prime}$, we need to discriminate two cases. Either $r^{\prime}$ has Om-value at least $(1-8\varepsilon)n$ in which case $\textsc{Om}(x)$ do not exceed $\textsc{Om}(r^{\prime})$ by the first part of $\mathcal{E}_{4}$. Or $r^{\prime}$ has Om-value at most $(1-8\varepsilon)n$, in which case $x$ do not exceed a Om-value of $(1-4\varepsilon)n$ by the second part of $\mathcal{E}_{4}$. Therefore, if $Z_{i-1}\geq(1-4\varepsilon)n$, we can conclude that Om-value of $x$ do not exceed $Z_{i-1}$ in both cases. Hence, we have shown that $\textsc{Om}(x)-Z_{i-1}>\beta\cdot C_{2}\log\mu$ is only possible if at least one of the events $\mathcal{E}_{1}$ - $\mathcal{E}_{4}$ does not occur, and thus

[TABLE]

If $Z_{i-1}<(1-4\varepsilon)n$, with the same arguments and letting $\beta\geq\varepsilon n/(\log^{2}\mu)$ we have $Z_{i}<(1-2\varepsilon)n$ with probability $1-2^{-\Omega(\varepsilon n/\log^{2}\mu)}$. ∎

5.3 Typical Situations

As outlined in the overview, our analysis of the drift will be based on studying what happens in ’typical’ situations. To characterize these, we use the following definition of ’good’ events. Again we consider the $(\mu+1)$-EA on the linear function $f_{\ell}$. For parameters $\phi,c_{d},c_{e}>0$ we define the event $\mathcal{E}_{\textup{good}}(i):=\mathcal{E}_{a}\cap\mathcal{E}_{b}\cap\ldots\cap\mathcal{E}_{e}$, where $\mathcal{E}_{a}$ etc. are the following events about the family forest $F_{i}$ of rank $i$. Recall the family forest consists of all $x\in X_{\geq i}$, and a vertex $u$ is a child of $v$ if $u$ was created as an offspring of $v$. We will be concerned about those vertices in the family forest in $X_{i}$, i.e., vertices of rank exactly $i$.

$\mathcal{E}_{a}$: No vertex in $X_{\leq i-1}$ creates offspring in $X_{\geq i+1}$.

• 2.

$\mathcal{E}_{b}$: There are at most $\varepsilon\mu\log^{3}\mu$ roots in $F_{i}$.

• 3.

$\mathcal{E}_{c}$: No vertex in $X_{i}$ of depth at most $\phi\log\mu$ in $F_{i}$ creates offspring in $X_{>i}$.

• 4.

$\mathcal{E}_{d}$: For every vertex $x\in X_{i}$ that creates an offspring in $X_{\geq i+1}$, if the root $r$ of $x$ has $\textsc{Om}(r)\geq(1-8\varepsilon)n$ then $\textsc{Om}(x)\leq\textsc{Om}(r)-c_{d}\log\mu$, and if $\textsc{Om}(r)\leq(1-8\varepsilon)n$ then $\textsc{Om}(x)\leq(1-4\varepsilon)n$. Moreover, the mutation changes at most $c_{d}/2\cdot\log\mu$ bits.

• 5.

$\mathcal{E}_{e}$: No vertex in $X_{i}$ has an Om-value which exceeds the Om-value of its root in $F_{i}$ by more than $c_{e}\log\mu$.

Lemma 14**.**

For every $0<\alpha<1$, $c>0$ there are $c_{d},c_{e}>0$ such that the following holds. For any constant parameters $0<\phi<1$ and $\eta>0$ that satisfy the following conditions, where $g(\phi)=\phi(\log(8e^{\alpha c+1})-\log\phi)$,

[TABLE]

there exists $\mu_{0}$ such that for all $\mu_{0}\leq\mu\leq n$ and all $i\geq(1-8\varepsilon)\alpha n$, the $(\mu+1)$-EA on $f_{\ell}$ satisfies

[TABLE]

We remark that $g(\phi)>0$ for $0<\phi<1$ and $g(\phi)<1/2$ for small enough $\phi$, so there exists $\eta>0$ that satisfies (9).

Proof.

We need to show that $\Pr[\mathcal{E}]=1-O(\log^{-2}\mu)$ holds for $\mathcal{E}=\mathcal{E}_{a},\ldots,\mathcal{E}_{e}$. Thus we split the proof into five parts. Note that we actually show the stronger statement $\Pr[\mathcal{E}]=1-\mu^{-\Omega(1)}$ for $\mathcal{E}=\mathcal{E}_{a},\mathcal{E}_{c},\mathcal{E}_{d},\mathcal{E}_{e}$.

$\mathcal{E}_{a}$: No vertex in $X_{\leq i-1}$ creates offspring in $X_{\geq i+1}$ for $\eta<1/2$.

We consider the number of offspring that are created from points in $X_{\leq i-1}$ and are members of $X_{\geq i+1}$ after the first point $x$ in $X_{\geq i}$ is created.

We first argue that the probability that $x\in X_{i}$ is $1-O(\varepsilon)$. Since we assume $i\geq(1-8\varepsilon)\alpha n$ and the rank of $x$ is at least $i$, the density of zero-bits is $d(A_{\ell+1},x)\leq 8\varepsilon$. By Lemma 4,

[TABLE]

which implies $\Pr[x\in X_{i}\mid x\in X_{\geq i}]=1-O(\varepsilon)=1-O(\mu^{\eta-1})$.

By Lemma 10, after the first search point in $X_{i}$ is created, with probability $1-O(\mu^{-\Omega(1)})$ it takes at most $T=8e^{\alpha c}\mu\log\mu$ rounds until the set $X_{\leq i}$ is completely deleted. If a search point in $X_{\leq i-1}$ creates an offspring in $X_{\geq i+1}$, at least 2 zero-bits need to be flipped. This probability is $O(\varepsilon^{2})$ by a union bound, and hence the expected number of offspring in $X_{\geq i+1}$ created from $X_{\leq i-1}$ is at most $O(\varepsilon^{2}T)=O(\mu^{-1+2\eta}\log\mu)$. Since $\eta<1/2$, by Markov’s inequality, the probability that the number of such offspring is at least 1 can be bounded by $O(\mu^{-1+2\eta}\log\mu)=O(\log^{-2}\mu)$, as required.

$\mathcal{E}_{b}$: There are at most $\varepsilon\mu\log^{3}\mu$ roots in $F_{i}$.

We know from $\mathcal{E}_{a}(i-1)$ that we may assume that no points in $X_{i}$ are created from $X_{\leq i-2}$. Hence, it suffices to count the number of roots in $X_{i}$ that are created from $X_{i-1}$. As in the proof for $\mathcal{E}_{a}$, by Lemma 10, after the first search point in $X_{i-1}$ is created, with probability $1-\mu^{-\Omega(1)}$ it takes at most $T=8e^{\alpha c}\mu\log\mu$ rounds until the set $X_{\leq{i-1}}$ is completely deleted. In each round we have a probability of at most $p_{U}=O(\varepsilon)$ to create a new root in $X_{i}$ ($p_{U}$ defined in Lemma 4), so the expected number of roots in $X_{i}$ is $O(\varepsilon T)$. By Markov’s inequality, the number of roots is at most $\varepsilon\mu\log^{3}\mu$ with probability $O(\varepsilon T/(\varepsilon\mu\log^{3}\mu))=O(\log^{-2}\mu)$.

$\mathcal{E}_{c}$: No vertex in $X_{i}$ of depth at most $\phi\log\mu$ in $F_{i}$ creates offspring in $X_{>i}$.

As a sketch for the proof, we first show that the number of vertices of depth at most $\phi\log\mu$ in $F_{i}$ is at most $\mu^{2g(\phi)}$ with high probability. Then by a simple estimation, the expected number of offspring in $X_{>i}$ created by those vertices is $O(\mu^{{-1+2\eta+2g(\phi)}}\log^{4}\mu)$. Since $g(\phi)<1/2$ for small enough $\phi$, for $\eta<1/2-g(\phi)$ with probability $1-O(\mu^{-\Omega(1)})$ no such offspring is created.

By Lemma 11, we couple $F_{i}$ with $F^{\prime}$. Since by $\mathcal{E}_{b}$ there are at most $\varepsilon\mu\log^{3}\mu$ roots in $F_{i}$, we only need to consider $\varepsilon\mu\log^{3}\mu$ trees in $F^{\prime}$.

Recall that by Lemma 10, the lifetime of $X_{i}$ is at most $T:=8e^{\alpha c}\mu\log\mu$ with probability at least $1-\mu^{-\Omega(1)}$, if $\mu\geq\mu_{0}$ for a sufficiently large $\mu_{0}$. Hence, it suffices to study $F^{\prime}$ after $T$ rounds. We want to bound the number of vertices with depth at most $\phi\log\mu$. We fix a root, and consider the tree attached to this root. By Property Lemma 12.4 and by the Stirling formula $k!=\Theta(\sqrt{k}(k/e)^{k})$ in the second step, the expected number of vertices with depth at most $\phi\log\mu$ at round $T$ is

[TABLE]

Note that for $0<\phi<1$ we have $g(\phi)>0$. By Markov’s inequality,

[TABLE]

Since we consider $\varepsilon\mu\log^{3}\mu=\mu^{\eta}\log^{3}\mu$ trees in $F^{\prime}$, by a union bound over all trees, with probability at least $1-o(\mu^{\eta-g(\phi)}\log^{3}\mu)$ the number of vertices with depth at most $\phi\log\mu$ is at most $\mu^{\eta+2g(\phi)}\log^{3}\mu$. Note that the error probability is $o(1)$ since we assumed that $\eta<g(\phi)$.

In each round, every such vertex has a probability of at most $O(\varepsilon/\mu)$ to create an offspring of strictly larger rank: it must be selected as parent and its offspring must have strictly larger rank. Since the vertices in $X_{i}$ are present for at most $T=8e^{\alpha c}\mu\log\mu$ rounds, the expected number of offspring in $X_{\geq i+1}$ created by vertices in $X_{i}$ of depth at most $\phi\log\mu$ is $O(T\cdot\varepsilon/\mu\cdot\mu^{\eta+2g(\phi)}\log^{3}\mu)=O(\mu^{-1+2\eta+2g(\phi)}\log^{4}\mu)$. By Markov’s inequality, the probability that the number of such offspring is at least 1 is $O(\mu^{-1+2\eta+2g(\phi)}\log^{4}\mu)$. Since $g(x)$ is monotonically increasing in $(0,1)$ and $g(0)=0$, $\eta<1/2-g(\phi)$ holds for small enough constant $\phi$, making the error probability $\mu^{-\Omega(1)}$. Hence, we have shown that with sufficiently small probability the vertices in depth at most $\phi\log\mu$ do not create offspring in $X_{>i}$.

$\mathcal{E}_{d}$: For every vertex $x\in X_{i}$ that creates an offspring in $X_{\geq i+1}$, if the root $r$ of $x$ has $\textsc{Om}(r)\geq(1-8\varepsilon)n$ then $\textsc{Om}(x)\leq\textsc{Om}(r)-c_{d}\log\mu$, and if $\textsc{Om}(r)\leq(1-8\varepsilon)n$ then $\textsc{Om}(x)\leq(1-4\varepsilon)n$. Moreover, the mutation changes at most $c_{d}/2\cdot\log\mu$ bits.

If $\mathcal{E}_{c}$ holds, the vertices in $X_{i}$ that create offspring in $X_{\geq i+1}$ must be of distance at least $d=\phi^{\prime}\log\mu$ where $\phi^{\prime}>\phi$ from their roots. Consider a root $r$ with $\textsc{Om}(r)\geq(1-8\varepsilon)n$. By equation (6) in Lemma 12, for $c_{d}=c\phi/16$, $\Pr[\textsc{Om}(x)-\textsc{Om}(r)\geq-c_{d}\log\mu]\leq 2e^{-c_{d}\log\mu/8}=2\mu^{-M}$, with $M:=c\phi/128$. If $\mathcal{E}_{b}(i+1)$ holds, the number of offspring in $X_{\geq i+1}$ created by points in $X_{i}$ is at most $\varepsilon\mu\log^{3}\mu$, which means the number of points in $X_{i}$ that create offspring in $X_{\geq i+1}$ is at most $\varepsilon\mu\log^{3}\mu$. By a union bound, with probability at least $1-O(\varepsilon\mu\log^{3}\mu\cdot 2\mu^{-M})=1-O(\mu^{-M+\eta}\log^{3}\mu)$, a vertex in $X_{i}$ that creates an offspring in $X_{\geq i+1}$ has a Om-value which is at least $c_{d}\log\mu$ smaller than that of its root. Since we assumed $\eta<M$, this probability is $1-\mu^{-\Omega(1)}$, and thus sufficiently large. This concludes the case that the root has Om-value at least $(1-8\varepsilon)n$.

If a vertex $x$ has a root which has at most Om-value $(1-8\varepsilon)n$, we consider the first vertex $x^{\prime}$ of Om-value at least $(1-6\varepsilon)n$ on the path from the root to $x$. Then we know that $x^{\prime}$ has Om-value at most $(1-5\varepsilon)n$, since its direct parent has Om-value less than $(1-6\varepsilon)n$ and the probability to flip at least $\varepsilon n$ bits in one mutation is $2^{-\varepsilon n}$. Then by similar arguments as above, the probability that a descendant of $x^{\prime}$ has Om-value which is $\varepsilon n$ larger than $x^{\prime}$ is also $2^{-\varepsilon n}=\mu^{-\omega(1)}$, and thus we can easily apply a union bound over $\varepsilon\mu\log^{3}\mu$ vertices in $X_{i}$ that create offspring in $X_{\geq i+1}$.

Finally, we come to the number of bit flips in the improving mutation. In one mutation the expected number of changed bits is $c$. Let $c_{d}/2\cdot\log\mu=(1+\delta^{\prime})c$ for some $\delta^{\prime}>1$, by Chernoff bound, the probability that the number of changed bits is larger than $c_{d}/2\cdot\log\mu$ can be bounded by $e^{-\delta^{\prime}c/3}=\Theta(\mu^{-c_{d}/6})$. Similarly, by a union bound, the error probability is at most $O(\varepsilon\mu\log^{3}\mu\cdot\mu^{-c_{d}/6})=O(\mu^{\eta-c_{d}/6}\log^{3}\mu)$, which is $\mu^{-\Omega(1)}$ since $\eta<c_{d}/6$.

$\mathcal{E}_{e}$: No vertex in $X_{i}$ has an Om-value which exceeds the Om-value of its root in $F_{i}$ by more than $c_{e}\log\mu$.

We set $c_{e}:=2e^{2}ck$ where $k>1$ is a positive constant to be chosen later and assume the distance between a vertex $x$ and its root $r$ is $d$. By Lemma 12.4, with probability $1-\mu^{-\Omega(1)}$, $d\leq e\log\mu$, and thus $c_{e}\log\mu\geq 2edc$. By Lemma 12.2, the probability that $x$ and $r$ differ in more than $c_{e}\log\mu$ is at most $2^{-c_{e}\log\mu}=\mu^{-2e^{2}ck\log 2}$. Therefore, the probability that $\textsc{Om}(x)$ exceeds $\textsc{Om}(y)$ by more than $c_{e}\log\mu$ is at most $\mu^{-2e^{2}ck\log 2}$. Moreover, The lifetime of $X_{i}$ is $8e^{\alpha c}\mu\log\mu$ with probability $1-\mu^{-\Omega(1)}$ by Lemma 10. By Lemma 12.1, with probability $1-O(\mu^{1-e^{\alpha c}}\log\mu)$ there are at most $\mu^{9e^{\alpha c}}$ vertices in $F_{i}$. By a union bound over all these vertices, the error probability is at most $O(\mu^{9e^{\alpha c}-2e^{2}ck\log 2})$. By choosing $k>9e^{\alpha c}/(2e^{2}c\log 2)$, this probability is $\mu^{-\Omega(1)}$. ∎

5.4 Estimating the Drift

We are now ready to collect the information to prove negative drift of the $Z_{i}$. We first give a lemma that shows that $Z_{i+K}-Z_{i}$ is negative in case of good events. As outlined in the introduction, good events don’t imply that $Z_{i+1}-Z_{i}$ is negative, we need to make $K$ steps for some constant $K\in\mathbb{N}$.

Lemma 15**.**

Let $\ell\in[L]$ and $i\in\mathbb{N}$. Consider the $(\mu+1)$-EA on the linear auxiliary function $f_{\ell}(x):=n\sum_{j\in A_{\ell}}x_{j}+\sum_{j\in R_{\ell}}x_{j}$. Assume that in some step $t\geq 0$ the highest rank in the population is $i$, that $\mathcal{E}_{\textup{good}}(i),\ldots,\mathcal{E}_{\textup{good}}(i+K)$ hold, where $K:=\lceil 2(c_{e}+1)/c_{d}\rceil$, and that $\textsc{Om}(r)\geq(1-8\varepsilon)n$ holds for all roots $r$ in $F_{i},\ldots,F_{i+K-1}$. Then $Z_{i+K}\leq Z_{i}-\log\mu$.

Proof.

Let $j\in\{i+1,\ldots,i+K\}$, and let $r\in X_{j}$ be any root in $F_{j}$. By $\mathcal{E}_{a}(j-1)$, the parent individual $x$ of $r$ is in $X_{j-1}$. By $\mathcal{E}_{d}(j-1)$, the root $r^{\prime}$ of $x$ in $F_{j-1}$ satisfies $\textsc{Om}(r^{\prime})\geq\textsc{Om}(x)+c_{d}\log\mu\geq\textsc{Om}(r)+c_{d}/2\cdot\log\mu$. By induction, we obtain that for every root $r\in X_{j}$ there exists a root $\tilde{r}\in X_{i}$ such that $\textsc{Om}(r)\leq\textsc{Om}(\tilde{r})-(j-i)c_{d}/2\cdot\log\mu\leq Z_{i}-(j-i)c_{d}/2\cdot\log\mu$, where the second step holds since $\textsc{Om}(\tilde{r})\leq Z_{i}$ by definition of $Z_{i}$. Now consider any individual $\tilde{x}\in X_{i+K}$, and let $r\in X_{j}$ be its root. By $\mathcal{E}_{e}(i+K)$, we have

[TABLE]

where the latter inequality follows from the definition of $K$. Since (5.4) holds for all $\tilde{x}\in X_{i+K}$, we obtain $Z_{i+K}\leq Z_{i}-\log\mu$, as required. ∎

We are now ready to prove the main theorem on the drift of $Z_{i}$. Recall that we have upper, but no lower tail bounds on $Z_{i}-Z_{i-1}$, cf.​ the comment before Lemma 13. In order to still be able to apply the negative drift theorem later, we show that the drift is even negative if we truncate the difference $Z_{i+K}-Z_{i}$ at $-\log\mu$.

Theorem 16**.**

For every $c>0$ there is a $\mu_{0}\in\mathbb{N}$ and a $K\in\mathbb{N}$ such that for all $\mu_{0}\leq\mu\leq n$ where $n$ is sufficiently large the following holds for the $(\mu+1)$-EA with mutation parameter $c$ on the auxiliary function $f_{\ell}$. Assume that in some generation the fittest search point satisfies (3). Then

[TABLE]

Proof.

Let $K$ be the constant from Lemma 15. Recall from Lemma 14 that the event $\mathcal{E}_{\textup{good}}$ has probability $1-O(\log^{-2}\mu)$, which is at least $1/2$ if $\mu$ is sufficiently large. By Lemma 15, the event $\mathcal{E}_{\textup{good}}$ implies $Z_{i+K}-Z_{i}\leq-\log\mu$, so in this case the term $\max\{Z_{i+K}-Z_{i},-\log\mu\}$ evaluates to $-\log\mu$. Hence, let $E_{\text{good}}:=\operatorname{\mathbb{E}}[\max\{Z_{i+K}-Z_{i},-\log\mu\}\mid\mathcal{E}_{\textup{good}}]\cdot\Pr[\mathcal{E}_{\textup{good}}]$, it holds that

[TABLE]

where the second equality holds because $\Pr[Z_{i+K}-Z_{i}=j\land\mathcal{E}_{\textup{good}}]=0$ for $j\geq-\lfloor\log\mu\rfloor$ and the last step follows from $\Pr[\mathcal{E}_{\textup{good}}]\geq 1/2$ if $\mu$ is sufficiently large.

In the remainder, we will show that the term $E_{\text{good}}$ is very close to $\operatorname{\mathbb{E}}[\max\{Z_{i+1}-Z_{i},-\log\mu\}]$. In fact, the difference is

[TABLE]

For an arbitrary constant $C>0$ we may define $j_{0}:=\lceil C\log\mu\log\log\mu\rceil$. Then we bound $j$ by $j_{0}$ in the range $j\leq j_{0}$, and we bound $\Pr[Z_{i+K}-Z_{i}=j\land\neg\mathcal{E}_{\textup{good}}]$ by $\Pr[Z_{i+K}-Z_{i}=j]$ for $j>j_{0}$. Since for $j>j_{0}$,

[TABLE]

We obtain

[TABLE]

where the factor $\log\mu$ in the second term appears because $\sum_{s=s_{0}}^{\infty}s2^{-s/x}=O(xs_{0}2^{-s_{0}/x})$ for $x\geq 1$, which can be seen by grouping the sum into batches of $x$ summands. The second term is $O(\log^{-1}\mu)$ if we choose the constant $C>0$ in the definition of $j_{0}=\lceil C\log\mu\log\log\mu\rceil$ appropriately. The first term is $O(\log\log\mu\cdot\log^{-1}\mu)$. Hence, by choosing $\mu$ sufficiently large, we can make both terms smaller than $1/2$, and obtain that $\operatorname{\mathbb{E}}[\max\{Z_{i+K}-Z_{i},-\log\mu\}]\leq E_{\text{good}}+1\leq-1$, as desired. ∎

6 Proof of Theorem 7

In the previous section we have analyzed the random variable $Z_{i}$, and in particular we have shown that it has negative drift. In this section we will show how our main result, the lower bound on the runtime for the $(\mu+1)$-EA, follows from the negative drift of $Z_{i}$. The proof follows from similar ideas as in [5] and [6]. We start with a lemma that describes the behavior of the $(\mu+1)$-EA on $f_{\ell}$.

Lemma 17**.**

For every constant $0<\delta<2/7$ the following holds. Let $\ell\in[L]$ and consider the $(\mu+1)$-EA on $f_{\ell}$ under the assumption that $d([n],x)\geq\varepsilon(1+2\delta)$ and $d(A_{\ell+1},x)\geq\varepsilon(1+\delta)$ hold for all $x$ in the initial population. For $t\geq 0$, let $x^{t}$ be the offspring in round $t$. Then with probability $1-\exp\{-\Omega(\varepsilon n/\log^{2}\mu)\}$, the following holds for all $t\leq L$.

$d([n],x^{t})\geq\varepsilon(1+\delta)$.

• 2.

$d([n],x^{t})\geq\varepsilon(1+2\delta)$* or $d(A_{\ell+1},x^{t})\geq\varepsilon(1+\delta/4)$.*

Before we come to the proof, let us briefly explain why the lemma is useful. It is tailored to support an inductive proof for Theorem 7 for HotTopic. In this induction, we will show that $d([n],x^{t})\geq\varepsilon(1+\delta)$ for exponential time. In fact, when the algorithm enters a new level then the density is at least $\varepsilon(1+2\delta)$. Moreover, one can show that with high probability the new hot topic did not influence the algorithm up to this point, so it behaves just as a random subset of positions of size $\alpha n$. In particular, with high probability its density is at least $\varepsilon(1+\delta)$, so the assumptions of the lemma are satisfied. As long as the level does not change, the HotTopic function is identical to $f_{\ell}$, so we may apply Lemma 17. The first item implies what we actually want to prove, at least as long as we stay on the same level. For the second item, by the construction of the HotTopic function the level increases when $d(A_{\ell+1},x^{t})\approx\varepsilon<\varepsilon(1+\delta/4)$. So the second item implies that at this point in time we have $d([n],x^{t})\geq\varepsilon(1+2\delta)$, which is the requirement for the next step in the induction. Note that we can’t just merge the items into one. For example, if we would weaken the second item to assert $d([n],x^{t})\geq\varepsilon(1+\delta)$ at the beginning of a level, then we could not conclude that the next offspring satisfies the same bound with exponentially small error probability.

Proof of Lemma 17.

Let $i_{0}$ be the largest rank in the initial population, i.e., the largest number of one-bits in $A_{\ell+1}$ in the initial population. We fix an offset $a\in\{0,\ldots,K-1\}$ and consider the sequence of random variables $Y_{i,a}:=Z_{i_{0}+a+iK}/\log\mu$, where $i$ is a non-negative integer. In the initial population, each individual has at most $n\left(1-\varepsilon(1+2\delta)\right)$ one-bits by assumption. Hence, we also have $Z_{i_{0}+a}\leq n(1-\varepsilon(1+3\delta/2))$ with probability $1-\exp\{-\Omega(\varepsilon n)\}$ for all offsets $a\in\{0,\ldots,K-1\}$, since otherwise at least one of the $K$ mutations would need to flip $\Omega(\varepsilon n)$ bits, which happens only with probability $\exp\{-\Omega(\varepsilon n)\}$ by the Chernoff bound. Thus for the first statement it suffices to show that $Y_{i,a}\leq Y_{0,a}+\varepsilon\delta n/(2\log\mu)$ for all $i\geq 0$. Since that is equivalent to $Z_{i_{0}+a+iK}\leq Z_{i_{0}+a}+\varepsilon\delta n/2$ for all $i\geq 0$, and we already have $Z_{i_{0}+a}\leq n(1-\varepsilon(1+3\delta/2))$ for all $a$ with high probability, altogether it implies $Z_{i^{\prime}}\leq n(1-\varepsilon(1+\delta))$ for all $i^{\prime}\geq i_{0}$. As $Z_{i^{\prime}}$ denotes the maximum number of one-bits in rank $i^{\prime}$, we conclude that $d([n],x^{t})\geq\varepsilon(1+\delta)$ holds for any individual $x^{t}$ of rank $i^{\prime}\geq i_{0}$. For the second statement, we distinguish between two cases. Note that the index $i$ counts, up to the factor $K$, the increase in one-bits in $A_{\ell+1}$. If $i\leq\alpha n\varepsilon\delta/(4K)-1$, then for any $x^{t}$ of rank $i_{0}+a+iK$, $d(A_{\ell+1},x^{t})\geq(\alpha n\varepsilon(1+\delta/2)-a-iK)/(\alpha n)>\varepsilon(1+\delta/2)-(i+1)K/(\alpha n)\geq\varepsilon(1+\delta/4)$. For $i>\alpha n\varepsilon\delta/(4K)-1$, we aim to show that $Y_{i,a}\leq Y_{0,a}$.

We would like to apply the negative drift theorem to $Y_{i,a}$ for the range $[(1-\varepsilon(1+3\delta/2))n/\log\mu,(1-\varepsilon(1+\delta))n/\log\mu]$. First note that we study a linear function, and that the bits in $A_{\ell}$ have larger weights than the remaining bits. Thus, it can be shown by a coupling argument (Lemma 4.2 in [5]) that if $d(A_{\ell+1},x)\leq d([n],x)+\delta\varepsilon$ holds initially, then the slightly weaker condition $d(A_{\ell+1},x)\leq d([n],x)+2\delta\varepsilon$ remains true for all individuals in the population for the next $L$ rounds, with probability at least $1-Le^{-\Omega(\varepsilon n)}$. By choosing the constant parameter $\rho$ in the definition of $L=\exp\{\rho\varepsilon n/\log^{2}\mu\}$ small enough, the factor $L$ can be swallowed by the term $e^{-\Omega(\varepsilon n)}$. Thus we may assume that whenever $Y_{i,a}$ is in the range $[(1-\varepsilon(1+3\delta/2))n/\log\mu,(1-\varepsilon(1+\delta))n/\log\mu]$ then $d(A_{\ell+1},x)\leq d([n],x)+2\delta\varepsilon\leq\varepsilon(1+3\delta/2+2\delta)\leq 2\varepsilon$ as $\delta<2/7$. In addition, we have $d(A_{\ell+1},x)\geq\varepsilon/2$ before the level changes, since otherwise with probability $1-e^{-\Omega(\varepsilon n)}$ it holds that $d(B_{\ell+1},x)<\varepsilon$, which implies an increase of level. Thus the conditions in (3) are satisfied, and thus Lemma 6 is applicable.

So let us study the drift of $Y_{i,a}$ in the range $[(1-\varepsilon(1+3\delta/2))n/\log\mu,\\ (1-\varepsilon(1+\delta))n/\log\mu]$. First note that the probability to jump over more than half of this interval is $\exp\{-\Omega(\varepsilon n/\log^{2}\mu)\}$: for $Y_{i,a}\geq(1-4\varepsilon)n/\log\mu$ this follows from the first statement in Lemma 13, for $Y_{i,a}<(1-4\varepsilon)n/\log\mu$ it follows from the second statement in Lemma 13. So we may assume that $Y_{i,a}$ is contained in the first half of the interval for some $i=i^{*}$. To ease notation, we will assume $i^{*}=0$. Inside of the interval, by Theorem 16, it holds that

[TABLE]

Moreover, by Lemma 13 the sequence of random variables $(Y_{i,a})_{i\geq 0}$ has an upper exponential tail bound, i.e., $\Pr[Y_{i+1,a}-Y_{i,a}\geq K\cdot\beta C_{2}]\leq K\cdot 2^{-\beta}$ for all $1\leq\beta\leq\varepsilon n/\log^{2}\mu$. (In particular, the probability that there is ever a jump larger than $KC_{2}\varepsilon n/\log^{2}\mu$ within $L$ steps is at most $O(L\cdot 2^{-\varepsilon n/\log^{2}\mu})=o(1)$, so we may assume that such jumps never occur.) To show sub-Gaussianity, we should extend the inequality also for $\beta<1$. Since any probability is bounded by 1, the bound $\Pr[Y_{i+1,a}-Y_{i,a}+1/{\log\mu}\geq K\beta C_{2}+1/{\log\mu}]\leq 2K\cdot 2^{-\beta}$ is not just true for $\beta\geq 1$, but also trivially satisfied for any $\beta\in[0,1]$. Therefore, for any $y\geq 0$, it holds that

[TABLE]

However, we need exponential tail bounds in both directions, so we need to truncate the downwards steps of $Y_{i,a}$ as follows. We set $\tilde{Y}_{0,a}:=Y_{0,a}$, and we define $\tilde{Y}_{i,a}$ recursively by $\tilde{Y}_{i,a}-\tilde{Y}_{i+1,a}:=\min\{Y_{i,a}-Y_{i+1,a},1/{\log\mu}\}$. Then clearly we have $\tilde{Y}_{i,a}\geq Y_{i,a}$ for all $i\geq 0$, and $\tilde{Y}_{i,a}$ satisfies the tail bound condition that

[TABLE]

Therefore, by Theorem 5, $(\tilde{Y}_{i,a}+i/{\log\mu})_{i\geq 0}$ is $(128c^{\prime}\delta^{\prime-3},\delta^{\prime}/4)$-sub-Gaussian, where $c^{\prime}=2^{1+1/(KC_{2}\log 2)}K$ and $\delta^{\prime}=2^{1/(KC_{2})}-1$. And by Theorem 6,

[TABLE]

Now for any $i\geq 0$ and $y=i/{\log\mu}+\varepsilon\delta n/(4\log\mu)$, with probability $1-\exp\{-\Omega(\varepsilon n/\log^{2}\mu)\}$ we have $Y_{i,a}\leq Y_{0,a}+\varepsilon\delta n/(4\log\mu)$. Note that $\varepsilon\delta n/(4\log\mu)$ is half of the length of the interval of interest, which implies that $Y_{i,a}$ does not go beyond the interval with high probability. Similarly, for every fixed $i\geq\alpha n\varepsilon\delta/(4K)$ and $y=i/{\log\mu}$ we have $Y_{i,a}\leq Y_{0,a}$ with probability $1-\exp\{-\Omega(\varepsilon n/\log^{2}\mu)\}$. The proof is concluded by a union bound over all possible $i$. Since there are at most $n$ possible values, this increases the error probability by a factor of $n$, which we can swallow in the expression $\exp\{-\Omega(\varepsilon n/\log^{2}\mu)\}$. ∎

Finally, we have collected all ingredients to prove our main result.

Proof of Theorem 7.

Let $L:=\exp\{\rho\varepsilon n/\log^{2}\mu\}$ be the number of levels. For the proof, we will consider an auxiliary run of the $(\mu+1)$-EA with a dynamic fitness function $\tilde{f}$ in which we only allow the levels to increase by one. In particular, the function $\tilde{f}$ does not only depend on the current state of the algorithm, but also on the algorithm’s history. More precisely, we define an auxiliary level $\tilde{\ell}(x,t)$ of a search point $x$, which we only allow to increase by at most one per round. Recall that $\ell(x)$ was defined in (1) as $\ell(x)=\max\{\ell^{\prime}\in[L]:d(B_{\ell^{\prime}},x)\leq\varepsilon\}$. For $\tilde{\ell}(t)$, we use the same definition except that we let the maximum go over only $\ell^{\prime}\leq\min\{\tilde{\ell}(t-1)+1,L\}$. I.e., we set $\tilde{\ell}(0):=0$, and if an offspring $y^{t}$ of $x^{t}$ enters the population in round $t$, then we set $\tilde{\ell}(y^{t},t):=\max\{\ell^{\prime}\in[\min\{\tilde{\ell}(x^{t},t-1)+1,L\}]:d(B_{\ell^{\prime}},y^{t})\leq\varepsilon\}$. (If the population stays the same in round $t$, then we leave $\tilde{\ell}$ unchanged.) Then we define the auxiliary fitness of $y^{t}$ as

[TABLE]

i.e., we use the same definition as for the HotTopic function except that we replace $\ell(y^{t})$ by $\tilde{\ell}(t)$. Then we proceed as the $(\mu+1)$-EA, i.e., in each round we compute and store the auxiliary fitness of the new offspring (which may depend on the whole history of the algorithm), and we remove the search point for which we have stored the lowest auxiliary fitness. This definition does not make much sense from an algorithmic perspective, but we will see in hindsight that the auxiliary process behaves identical to the actual $(\mu+1)$-EA. We will next argue why this is the case.

For the auxiliary process, it is obvious that we only need to uncover the set $A_{i+1}$ and $B_{i+1}$ when we reach level $\tilde{\ell}(t)=i$. As we will show later for the auxiliary process, with high probability the density $d([n],x^{t})$ stays strictly above $\varepsilon\cdot(1+\delta)$ for a suitable constant $\delta>0$. Now fix any round $t$ with auxiliary level $\tilde{\ell}(t)$. Since we do need to uncover $B_{\tilde{\ell}(t)+2}$ at some point after time $t$, its choice does not influence the behavior of the auxiliary process until time $t$. Hence, we can first let the auxiliary process run until time $t$, and afterwards uncover the set $B_{\tilde{\ell}(t)+2}$. Since $B_{\tilde{\ell}(t)+2}\subset[n]$ is a uniformly random subset of size $\beta n$, it contains at least $\beta\varepsilon(1+\delta)n$ zero-bits in expectation, and the probability that $B_{\tilde{\ell}(t)+2}$ contains at most $\beta\varepsilon n$ zero-bits is $\exp\{-\Omega(\beta\varepsilon n)\}$. The same argument also holds for $B_{\tilde{\ell}(t)+3},\ldots,B_{L}$. Since $L=\exp\{\rho\varepsilon n/\log^{2}\mu\}$ with desirably small $\rho>0$, we can afford a union bound over all such sets and all times $t\leq L$, which is a union bound over less than $L^{2}=\exp\{2\rho\varepsilon n/\log^{2}\mu\}$ terms. Hence, with high probability we have $d(B_{i},x^{t})>\varepsilon$ for all $1\leq t\leq L$ and all $\tilde{\ell}(t)+2\leq i\leq L$. A straightforward induction shows that this implies $\ell(t)=\tilde{\ell}(t)$ for all $t\leq L$, and thus the $(\mu+1)$-EA behaves identical to the auxiliary process. Note that this already implies that the $(\mu+1)$-EA visits each of the $L$ levels, which implies the desired runtime bound. It only remains to show that there is a constant $\delta>0$ such that the auxiliary process satisfies $d([n],x^{t})>\varepsilon\cdot(1+\delta)$ for all $t\leq L$.

The advantage of the auxiliary process is that we may postpone drawing $A_{\ell+1}$ until we reach level $\tilde{\ell}=\ell$. In particular, since $A_{\ell+1}\subseteq[n]$ is a uniformly random subset, we may use the same argument as before and conclude that $|d(A_{\ell+1},x)-d([n],x)|<\delta\varepsilon$ holds with probability $1-\exp\{-\Omega(\varepsilon n)\}$ for any constant $\delta>0$ that we desire, and for all members $x$ of the population when we reach level $\ell$. In fact, we have exponentially small error probability, so we may afford a union bound and conclude that with high probability the same holds for all $\ell$. We want to show that the auxiliary process, if running on level $\ell$ and starting with a population that initially satisfies $|d(A_{\ell+1},x)-d([n],x)|<\delta\varepsilon$ for $\delta<2/7$, maintains $d([n],x^{t})\geq\varepsilon(1+\delta)$ for all new search points $x^{t}$ until $t>L$.

By the first conclusion from Lemma 17, $d([n],x^{t})\geq\varepsilon(1+\delta)$ holds as long as the level remains to be $\ell$ and $t\leq L$. When a point $x$ reaches level $\ell+1$, by definition we have $d(B_{\ell+1},x)<\varepsilon$. Since $B_{\ell+1}$ is a uniformly random subset of $A_{\ell+1}$, by the Chernoff bound $d(A_{\ell+1},x)<\varepsilon(1+\delta/4)$ holds with probability $1-\exp\{-\Omega(\varepsilon n)\}$. So we apply the second conclusion of Lemma 17 to $x$ and conclude that $d([n],x)\geq\varepsilon(1+2\delta)$. With high probability, it holds that $d(A_{\ell+2},x)\geq\varepsilon(1+2\delta)-\varepsilon\delta$ and the conditions in Lemma 17 are satisfied again for level $\ell+1$. By induction we obtain $d([n],x^{t})\geq\varepsilon(1+\delta)$ for all $t\leq L$. As the choice of $\ell$ is arbitrary, we start with $\ell=0$ and $d([n],x^{t})\geq\varepsilon(1+\delta)$ holds for all $t\leq L$. This concludes the proof. ∎

7 Simulation

In this section we will illustrate the detrimental effect of large populations on $(\mu+1)$-EAs by numerical simulations. Unless otherwise stated, the parameters that we used to generate the HotTopic functions are $n=10000$, $L=100$, $\alpha=0.25$, $\beta=0.05$ and $\varepsilon=0.05$. Each data point is obtained by 10 independent runs on the same HotTopic function with different random seeds. Our implementation is available at https://github.com/zuxu/MuOneEA-HotTopic.

7.1 Population Size

First of all, we plot typical behaviours of evolutionary algorithms with small, medium and large population sizes. Figure 2 shows the distance between the optimum and the fittest point $p^{*}$ in the population with respect to time. We have two metrics for the distance: the density of 0-bits in $p^{*}$ and the remaining levels of $p^{*}$ divided by $L$. As indicated by the sudden drops in level, for a small population size ($\mu=30$), the algorithm skips many levels and reaches the optimum quickly. In contrast, an algorithm with large $\mu=70$ visits the levels one by one, without improvement on the fitness. This happens where the density of 1-bits is relatively high, such that even though it gradually improves on the current hot topic, it often accepts offspring that flip 1-bits to 0-bits outside of the hot topic. With such offspring accumulating in a large population, the average density of 0-bits remains significantly above $\varepsilon$ before reaching the last level. Therefore, with high probability the algorithm does not skip any level. Once the highest level is reached, the remaining bits can be optimized easily as in the coupon collector. For $\mu=50$, the density gets close, but slightly above $\varepsilon$, so that it depends on chance whether levels are skipped or not. This leads to a high variance in the running time.

In Figure 3, we show the running time and the number of visited levels for a wide range of $\mu$. The running time is highly concentrated when $\mu$ is very small or very large. The reason is that the algorithm keeps skipping levels with small $\mu$ and visits all levels with large $\mu$. For a medium sized $\mu$ like 50, level skipping only happens a few times. Since each time when the algorithm skips a level, it lands at some higher level uniformly at random due to the definition of the HotTopic function, which results in a larger variance in the running time.

7.2 Mutation Rate

The mutation rate $c$ is the other factor that affects the magnitude of the negative drift, so we also plot the running time for various values of $c$, see Figure 4. For a small $c$, detrimental mutations do not occur frequently and thus the average density of 1-bits in the population keeps increasing. Conversely, with a large $c$, the algorithm tends to visit all the levels. To demonstrate the resulting effect on the running time, we compare the cases where $L=100$ and $L=200$. If $c$ is small ($c\leq 0.9$), the algorithm skips levels quickly, and the running time is almost independent of the number of levels. On the other hand, if $c$ is large ($c\geq 1.2$) then the algorithm visits every level. In this case, the running time is essentially proportional to the number of levels, plus some initial phase. Note that in this range the running time can get almost arbitrarily bad, since doubling the number of levels $L$ will essentially lead to a doubling of the running time. As our theoretical analysis shows, this holds even when $L$ becomes exponential in $n$, but for so many levels the running time becomes too large to run experiments. Finally, for a medium sized $c$ like 1.1, level skipping only happens a few times. Each time when the algorithm skips a level, it lands at some higher level uniformly at random, which results in a larger variance in the running time, similar to Figure 3.

Finally, we investigate how values of $\mu$ and $c$ jointly influence the behaviour of a $(\mu+1)$-EA. For a fixed $c$, we search for the minimum value of $\mu$ such that the algorithm visits all levels in at least half of the 10 simulations. That is, we seek for a minimum $\mu$ that induces long running times with constant probability. As we can see from Figure 5, large values of $c$ ($\geq 3.2$) are extremely harmful even when there is only one individual in the population. An algorithm with a large population can benefit greatly from having a small mutation rate. We did not observe a stable slowdown for $c=0.7$ until we raise the value of $\mu$ to more than $1000$.

8 Conclusion

We have shown that the $(\mu+1)$-EA with arbitrary mutation parameter $c>0$ needs exponential time on some monotone functions if $\mu$ is too large. This is one of the very few known situations in which even a slightly larger population size $\mu$ can lead to a drastic decrease in performance. The main reason is that, if progress is steady enough that the population does not degenerate, the search points that produce offspring are typically not the fittest ones. We believe that this is an interesting phenomenon which deserves further investigations, also in less artificial contexts.

For example, consider the $(\mu+1)$-EA on weighted linear functions with a skewed distribution (e.g., on BinVal), and with a fixed time budget (so that the action happens away from the optimum). It is quite conceivable that the same effect hurts performance, i.e., if the algorithm flips a high-weight bit, it will allow (almost) any offspring of this individual into the population, even though this offspring has probably fewer correct bits than other search points in the population. Does that mean that the fixed-budget performance of the $(\mu+1)$-EA on BinVal deteriorates with increasing $\mu$? Are the resulting individuals further away from the optimum?

An even more pressing question is about crossover. We have studied the $(\mu+1)$-EA, but do the same results also apply for the $(\mu+1)$-GA? In [6] it was shown that close to the optimum (for small values of the HotTopic parameter $\varepsilon$) crossover helps dramatically, and that a large population size can even counterbalance large mutation parameters $c$. So, close to the optimum, for the $(\mu+1)$-GA the effect of large population size was beneficial, while for the $(\mu+1)$-EA it was neutral and did not affect the threshold $c_{0}$. Thus if we study the $(\mu+1)$-GA on HotTopic functions with large $\varepsilon$, then a beneficial effect of large populations is competing with a detrimental effect. Understanding this interplay would be a major step towards a better understanding of crossover in general.

Similarly, since the problems originate in non-trivial populations, what happens if we equip the $(\mu+1)$-EA with a diversity mechanism (duplication avoidance, genotypical or phenotypical niching), and study it close to the optimum? Does it fall for the same traps? This question was already asked in [6], but our results shed additional light on the question.

Finally, it is open whether the $(\mu+1)$-EA is fast on any monotone function if it starts close enough to the optimum. i.e., for every $\mu\in\mathbb{N}$, does there exist an $\varepsilon=\varepsilon(\mu)$ such that the $(\mu+1)$-EA, initialized with a random search point with $\varepsilon n$ zero-bits, has runtime $O(n\log n)$ for every monotone function? Of course, the same question also applies to other algorithms like the $(\mu+1)$-GA and the ‘fast’ counterparts of the $(\mu+1)$-EA and the $(\mu+1)$-GA. Interestingly, the result in [6] that the ‘fast $(1+\lambda)$-EA’ with good parameters is efficient for every monotone function was only proven under this assumption, that the algorithm starts close to the optimum. So this also raises the question whether there are traps for the ‘fast $(1+\lambda)$-EA’ that only take effect far away from the optimum.