Transitions in optimal adaptive strategies for populations in fluctuating environments

TL;DR

This paper investigates how biological populations adapt to fluctuating environments by analyzing optimal strategies like specialization, generalism, and bet-hedging using a mathematical model, revealing new insights especially when environmental fluctuations are temporally correlated.

Contribution

The study extends existing models by providing analytical results for optimal adaptive strategies in correlated environmental fluctuations, incorporating memory effects and trade-offs.

Findings

Optimal strategies depend on environmental fluctuation statistics.

Temporal correlations enable memory-based adaptive strategies.

Results apply to antibiotic resistance and immune response phenomena.

Abstract

Biological populations are subject to fluctuating environmental conditions. Different adaptive strategies can allow them to cope with these fluctuations: specialization to one particular environmental condition, adoption of a generalist phenotype that compromise between conditions, or population-wise diversification (bet-hedging). Which strategy provides the largest selective advantage in the long run depends on the range of accessible phenotypes and the statistics of the environmental fluctuations. Here, we analyze this problem in a simple mathematical model of population growth. First, we review and extend a graphical method to identify the nature of the optimal strategy when the environmental fluctuations are uncorrelated. Temporal correlations in environmental fluctuations open up new strategies that rely on memory but are mathematically challenging to study: we present here new analytical results to address this challenge. We illustrate our general approach by analyzing optimal adaptive strategies in the presence of trade-offs that constrain the range of accessible phenotypes. Our results extend several previous studies and have applications to a variety of biological phenomena, from antibiotic resistance in bacteria to immune responses in vertebrates.

Equations

$$N_{t+1}(\sigma )=f(\sigma ,x_t)\sum _{{\sigma }^{\prime }}\pi (\sigma |{\sigma }^{\prime })N_t({\sigma }^{\prime }).$$

$${\mathbf{N}}_{t+1}={\mathbf{A}}^{(x_t)}{\mathbf{N}}_t,\text{with}{A}_{\sigma ,{\sigma }^{\prime }}^{(x_t)}=f(\sigma ,x_t)\pi (\sigma |{\sigma }^{\prime }).$$

$$\Lambda =\underset{T\to \infty }{\lim }\frac{1}{T}\ln N_T/N_0,$$

$${\pi }^{\ast }(\sigma |{\sigma }^{\prime })=\underset{\pi (\sigma |{\sigma }^{\prime })}{\mathrm{argmax}}\Lambda ,$$

$$N_{t+1}=N_{t}f(x_t),$$

$$f(x)=\sum _{\sigma }f(\sigma ,x)\pi (\sigma )$$

$$\Lambda =\sum _{x}p(x)\ln f(x),$$

$$f(2)=\frac{e^{K/p(2)}}{f(1{)}^{p(1)/p(2)}}.$$

$$p(1)m(1)+p(2)m(2)=K.$$

$$f(\sigma ,x)=\{\begin{array}{ll}1& \sigma =x,\\ w_x& \sigma \ne x.\end{array}$$

$$\begin{array}{rl}\Lambda =& p_2\log [(1-{\pi }_2)w_2+{\pi }_2]\\ & +(1-p_2)\log [1-{\pi }_2+{\pi }_2{w}_1)].\end{array}$$

$$\pi_{2}^{\star}=\left\{\begin{array}[]{cl}0&\textrm{if}\quad p_{2}\leq p_{2}^{\rm lb},\\ \frac{p_{2}-p_{2}^{\rm lb}}{p_{2}^{\rm ub}-p_{2}^{\rm lb}}&\textrm{if}\quad p_{2}^{\rm lb}<p_{2}<p_{2}^{\rm ub},\\ 1&\textrm{if}\quad p_{2}\geq p_{2}^{\rm ub}\\ \end{array}\right.$$

$$p_2^{\mathrm{lb}}=\frac{w_2}{1+(1-w_2)w_1/(1-w_1)}$$

$$p_2^{\mathrm{ub}}=\frac{1}{1+(1-w_2)w_1/(1-w_1)}$$

$${\pi }_2^{\star }=p_2.$$

$$\begin{array}{rl}\Lambda =& p\ln [\pi f_{\mathrm{def}}+(1-\pi )f_{\inf }]\\ & +(1-p)\ln [\pi f_{\mathrm{con}}+(1-\pi )f_{\mathrm{base}}],\end{array}$$

$$\begin{array}{rl}\Lambda =& \sum _{x}p(x)\ln f(x,x)\\ & +\sum _{x,x^{\prime }}p(x|x^{\prime })p(x^{\prime })\ln [\pi (x|x^{\prime })\Gamma (x,x^{\prime })],\end{array}$$

$$\Gamma (x,x^{\prime })=\frac{f(x,x^{\prime })}{f(x^{\prime },x^{\prime })-f(x,x^{\prime })}+\frac{f(x,x)}{f(x,x)-f(x^{\prime },x)},$$

$$\Gamma (x,x^{\prime })=\frac{w(x,x^{\prime })}{1-w(x,x^{\prime })}+\frac{1}{1-w(x^{\prime },x)},$$

$$n_{t+1}(\mathbf{\sigma })=\frac{1}{Z_t}f(\mathbf{\sigma },{\mathbf{x}}_t)\sum _{{\mathbf{\sigma }}^{\prime }}\pi (\mathbf{\sigma }|{\mathbf{\sigma }}^{\prime })n_t({\mathbf{\sigma }}^{\prime }),$$

$$n_{t+1}=\frac{1}{Z_t}{w}_1^{1-x_t}\left(n_t(1-{\pi }_{21})+(1-n_t){\pi }_{12}\right)$$

$$Z_t=n_t{w}_1^{1-x_t}+(1-n_t)w_2^{x_t}.$$

$$n_{t+1}={\pi }_{12}+(1-{\pi }_{21})w_1^{1-x_t}{w}_2^{x_t}{n}_t.$$

$$\begin{array}{rl}\ln Z_t& =x_t\ln w_2-n_t+n_t{w}_1^{1-x_t}{w}_2^{x_t}\\ & =x_t\ln w_2+n_t(w_1-1)+x_t{r}_t(w_2^{-1}-w_1).\end{array}$$

$$\Delta \Lambda =⟨n⟩(w_2^{-1}-w_1)\left(\frac{⟨xn⟩}{⟨n⟩}-\frac{1-w_1}{w_2^{-1}-w_1}\right),$$

$$\frac{⟨xn⟩}{⟨n⟩}>\frac{1-w_1}{w_2^{-1}-w_1}.$$

$${\rho }_{1,t+1}=$$

$$+$$

$${\rho }_{2,t+1}=$$

$$+$$

$$\begin{array}{rl}\frac{⟨xn⟩}{⟨n⟩}& =\frac{{\rho }_2}{{\rho }_2+{\rho }_1}\\ & =\frac{p_2\left[1-(1-{\pi }_{21})e^{-1/t_c}{w}_1\right]}{1-(1-{\pi }_{21})e^{-1/t_c}{w}_1\left[(1-p_2)w_1{w}_2+p_2\right]},\end{array}$$

$$p_2>\frac{(1-w_1)(1-e^{-1/t_c}{w}_2^{-1})}{(w_2^{-1}-w_1)(1-e^{-1/t_c})}.$$

Full text

I Introduction

Nothing is as constant as change. This age-old adage applies to biological populations, which may respond by evolving mechanisms to mitigate the consequences of environmental fluctuations Levins (1968); Seger and Brockmann (1987); Kussell and Leibler (2005); Chevin et al. (2010); Simons (2011). This adaptation can be implemented at different levels. At an individual level, the simplest strategy consists in adopting a generalist phenotype that does reasonably well across environments. At a population-level, another strategy is to constantly generate a phenotypically diverse mixture of individuals, each specialized to a different environmental condition. Which strategy provides the largest selective advantage in the long run depends on the nature of environmental fluctuations and on the fitness costs and trade-offs limiting the range of accessible phenotypes. For instance, although tracking the environment to adopt a phenotype specialized to each current condition may seem optimal, this strategy is often precluded by the costs of constantly monitoring environmental changes and of frequently switching between phenotypes.

Which strategies to deal with environmental fluctuations may be selected is a long-standing question in evolutionary biology. Interest in this question has recently been rekindled by novel laboratory experiments with populations growing in controlled fluctuating environments Acar et al. (2008); Beaumont et al. (2009); Lambert and Kussell (2015), new theoretical developments providing links to ideas from information theory and stochastic thermodynamics Vinkler et al. (2016); Kobayashi and Sughiyama (2015); Rivoire (2016), and its relevance to understanding non-genetic modes of inheritance Lachmann and Jablonka (1996); Rivoire and Leibler (2014) and how biological populations might respond to climate change Chevin et al. (2010); Botero et al. (2015).

Here, we study this question in a model of population growth in a randomly fluctuating environment. The model considers a large population of organisms characterized by their phenotype and replicating at discrete generations. An optimal adaptive strategy is defined by the choice of phenotypes and switching rates between them that ensures the largest long-term population growth rate. We analyze how this optimal strategy depends on the environmental statistics and the replication rates. The analysis reveals transitions between qualitatively different strategies: non-switching or single-phenotype strategies, where all of the population is of the same phenotype; and switching or bet-hedging strategies, where the population diversifies. Further transitions arise between strategies where the population adopts a phenotype specialized in a single environment, and strategies relying on a generalist phenotype.

Our work extends the growing literature investigating transitions between optimal adaptive strategies Donaldson-Matasci et al. (2008); Salathé et al. (2009); Rivoire and Leibler (2014); Patra and Klumpp (2015); Skanata and Kussell (2016) and generalizes some of our previous results on the adaptation of immune strategies to pathogen statistics Mayer et al. (2016). In particular, we derive exact expressions for the transitions between different modes of immunity in memoryless environments when the strategy includes an adjustable investment into immunity. We also calculate analytically the transitions between switching and non-switching strategies between two phenotypes in temporally correlated environments. After briefly introducing the mathematical framework (Sec. II), we present a graphical method for studying transitions in optimal adaptive strategies in temporally uncorrelated environments (Sec. III), and apply it to the case of an immune system with adjustable investment (Sec. IV). We then turn to the case of temporally correlated environments and provide new analytical and numerical results on transitions in this more general setting (Sec. V).

II Population growth in fluctuating environments

We are interested in describing the evolution of a possibly phenotypically heterogeneous biological population (of cells, organisms, etc.) in a fluctuating environment. We describe the population at generation $t$ by the number $N_{t}(\sigma)$ of individuals with a given phenotype $\sigma$. Phenotypes differ by their replication rate $f(\sigma,x)$, which give the mean number of offspring produced by an individual of phenotype $\sigma$ in environmental condition $x$ (see Fig. 1). The environment is described as a discrete Markov chain with a transition matrix $p(x|x^{\prime})$, which we assume to be stationary and ergodic. The population changes under the influence of the selective pressures generated by the differences in replication rates between phenotypes, and through phenotype switches described by a transition matrix $\pi(\sigma|\sigma^{\prime})$. In the limit of infinitely large population size, the population composition follows the recursion Rivoire and Leibler (2011)

[TABLE]

This equation can also be written in a compact matrix notation as

[TABLE]

Here and in the following, we write vectors and matrices in bold notation.

The different modalities by which populations might cope with fluctuating environmental conditions correspond to different properties of the switching matrix $\pi(\sigma|\sigma^{\prime})$. For non-switching strategies, the whole population has the same phenotype $\tilde{\sigma}$ and the switching matrix consists in a row of ones, $\pi(\sigma|\sigma^{\prime})=1$ if $\sigma=\tilde{\sigma}$ and 0 otherwise. If the chosen phenotype is a better all-rounder doing intermediately well across environments, this corresponds to an individual-level generalist strategy. For switching strategies, we may distinguish those with and without memory. In a switching strategy without memory, the probability of switching to a phenotype does not depend on the parental phenotype, $\pi(\sigma|\sigma^{\prime})=\pi(\sigma)$. Such strategies implement population-level bet-hedging, i.e., diversification of the population into phenotypes that may each be specialized to one of the environmental conditions to come. Switching with memory, where $\pi(\sigma|\sigma^{\prime})$ does depend on $\sigma^{\prime}$, provides the basic ingredients, variation and heritability, to enable adaptive tracking of the environment through Darwinian evolution. In the limit where switching is very rare, $\pi(\sigma|\sigma^{\prime})\ll\pi(\sigma^{\prime}|\sigma^{\prime})$ for $\sigma\neq\sigma^{\prime}$, the phenotypic dynamics is equivalent to the strong-selection weak-mutation limit of population genetics Rivoire and Leibler (2014). The model thus integrates in a common mathematical framework a range of different modes of response to environmental variations.

Over long evolutionary time scales, selection might act on the adaptive mechanisms to adjust them to the statistics of environmental fluctuations. Explicit models of the evolution of the switching rates $\pi(\sigma|\sigma^{\prime})$ show that variation in switching rates can indeed be selected upon Salathé et al. (2009); Carja et al. (2014); Rivoire and Leibler (2014). Transgenerational feedback reinforcing the production of successfull phenotypes provides an alternative mechanism to learn a good strategy Xue and Leibler (2016). Which adaptive strategy do we expect to evolve in the long run? Here, we focus on the optimal strategy representing the optimal possible end-product of this evolution. In our model, the optimal switching rates maximize long-term growth rate, defined as

[TABLE]

where $N_{T}=\sum_{\sigma}N_{T}(\sigma)$ is the total population size. To understand why this is the relevant measure of evolutionary success in the long run, consider a population with two subpopulations following different strategies. Then in the long run the population following the strategy with highest long-term growth rate almost surely outnumbers the one following the other strategy for almost every sequence of environments Cover and Thomas (2005). The question of which adaptive strategy $\pi^{*}(\sigma|\sigma^{\prime})$ has the largest selective advantage is thus recast as the problem of maximizing the long-term growth rate over possible strategies:

[TABLE]

for given replication rates $f(\sigma,x)$ and given environmental dynamics $p(x|x^{\prime})$. This is the problem that we address in this paper.

III When and how to be a generalist in uncorrelated environments

III.1 Extended fitness set and Pareto optimality

The simplest environmental fluctuations to consider are memoryless fluctuations, where the state of the environment is independent of its state in the previous generation, $p(x|x^{\prime})=p(x)$. In this case, no gain can be expected from keeping a memory of past phenotypic states, and the optimal adaptive strategy is also memoryless, $\pi(\sigma|\sigma^{\prime})=\pi(\sigma)$. Since the population composition is constant over generations, the number of offspring depends only on the state of the environment and (1) reduces to a recursion for the total population size $N_{t}=\sum_{\sigma}N_{t}(\sigma)$:

[TABLE]

where

[TABLE]

is the average population fitness. Graphically, it is convenient to represent each possible phenotype $\sigma$ as a point in the space of environmental conditions $x$ (where each environment $x$ defines a dimension), with coordinates given by the replication rates $f(\sigma,x)$ (orange dots in Fig. 2A). The set $D_{f}=\{\sum_{\sigma}f(\sigma,x)\pi(\sigma)|\sum_{\sigma}\pi(\sigma)=1,\pi(\sigma)\geq 0\}$ of achievable $f(x)$ when switching rates $\pi(\sigma)$ are varied then corresponds to the convex hull of these points (orange area in Fig. 2B). In the ecological literature, this set of achievable strategies is known as the extended fitness set and was introduced by Levins Levins (1968).

The recursion for the total population size (5) is solved by $N_{T}=N_{0}\prod_{t}f(x_{t})$. Taking logarithms, we have $\ln N_{T}/N_{0}=\sum_{t=1}^{T}\ln f(x_{t})$ and we can apply the law of large numbers to write the long-term growth rate (3) as a weighted average of log-fitnesses:

[TABLE]

with weights given by the frequency of each environment.

Finding the optimal strategy $\pi^{*}(\sigma)$ that maximizes $\Lambda=\sum_{x}p(x)\ln\sum_{\sigma}f(\sigma,x)\pi(\sigma)$ over the domain allowed by the rules of probabilities is a convex optimization problem whose solution is well known Cohen (1966); Levins (1968); Iwasa and Haccou (1995); Cover and Thomas (2005); Rivoire and Leibler (2011); Donaldson-Matasci et al. (2008). It is useful to rephrase the problem as the optimization of $\Lambda=\sum_{x}p(x)\ln f(x)$ over the fitnesses $\boldsymbol{f}$ constrained to belong to the extended fitness set $D_{f}$ introduced above. One can go further and equivalently optimize $\Lambda=\sum_{x}p(x)m(x)$ over the log-fitnesses $m(x)=\ln f(x)$ contrained to belong to $\ln(D_{f})$. Going from $\boldsymbol{\pi}$ to $\boldsymbol{f}$ to $\boldsymbol{m}$ simplifies the expression of the objective function $\Lambda$ but makes the domain of optimization more complex.

Eq. (7) shows that the long-term growth rate is an increasing function of each environment fitness $f(x)$. Increasing fitness in one environment is always desirable if this can be done without impairing fitness in any other environment. Thus, any optimal solution must lie on the set of fitnesses $\boldsymbol{f}$ for which no improvement can be made in one environment without impairing performance in another, called the Pareto frontier. Usually, no phenotype provides the best fitness for all environments due to trade-offs between performance under different conditions. Thus the Pareto frontier is generally not a single point but a line when the environment alternates between two conditions (blue line in Fig. 2B), and a hyper-surface of dimension $n-1$ when the environment alternates between $n$ conditions. To find the overall optimum along the Pareto front requires to consider the explicit way in which performances for different objectives combine into a scalar measure, which is determined in our case by the frequency of the different environments (7).

III.2 Graphical method for finding the optimal strategy

The various views of the optimization problem discussed in the previous subsection imply a graphical method to determine the optimal strategy. For simplicity, we illustrate it by considering switching between only two environments (Fig. 2). Starting from the graphical representation of the Pareto front for the set of achievable fitnesses (Fig. 2B), we need to find the point of this frontier with the highest growth rate: this is done graphically by representing the growth rate isolines $\Lambda[f(1),f(2)]=K$ (red and green lines in Fig. 2C where the two colors corresponds to different environmental statistics) given by (7):

[TABLE]

By plotting the isolines for different $K$ we can find the isoline for the largest $K$ that still intersects with the Pareto frontier, called supporting line. The intersection point defines the optimal adaptive strategy the population should adopt (red and green stars in Fig. 2C). This construction was first proposed by Levins Levins (1968).

Here, we propose to go one step further and work in log-fitness space to circumvent the difficulty of handling curved isolines. In log-fitness space, the isolines are linear and normal to the vector $\boldsymbol{p}$:

[TABLE]

If the Pareto front has a tangent of slope $-p(1)/p(2)$, the tangent point thus defines the optimal strategy for the environment $\boldsymbol{p}$ (Fig. 2D). More generally, the supporting isoline corresponding to the optimal growth rate shares at least one point with the Pareto frontier but is otherwise entirely above that frontier.

The graphical method generalizes to $d$ environments by studying the extended fitness set in a space of $d$ dimension, according to the following procedure. First, represent the phenotypes’ fitnesses as points in the space of different environments, each environment defining a dimension (orange dots in Fig. 2A). Second, construct the convex hull of these points to find the fitnesses achievable by switching strategies $D_{f}$ (orange area in Fig. 2B), and find the Pareto-optimal frontier of that set (blue line in Fig. 2B). Third, plot this Pareto surface in log-fitness space (blue line in Fig. 2D). Finally, find the hyperplane normal to $\boldsymbol{p}$ that is a supporting hyperplane of the Pareto frontier (red and green lines in Fig. 2D), and read off the optimal strategy as the intersection point between that hyperplane and the Pareto frontier (red and green stars in Fig. 2D).

When the Pareto frontier is contained in a hyperplane, fitnesses can be rescaled onto the unit simplex, $\sum_{x}f(x)=1$, with no loss of generality Donaldson-Matasci et al. (2008). In this case the optimal strategy is given in terms of the rescaled fitnesses as $\boldsymbol{f}^{*}=\boldsymbol{p}$, making the graphical construction even simpler (Fig. S1 and App. A).

III.3 Transitions between switching, non-switching, and generalist strategies

The graphical method provides a visual approach to classify the different possible adaptive strategies. For the sake of simplicity, we start again with the case of a two-state environment and first assume that only two phenotypes are accessible: a blue phenotype ($\sigma=1$) best suited to environment 1 and an orange phenotype ($\sigma=2$) best suited to environment 2 (Fig. 3A-C). In this case, the Pareto front is a segment joining the two phenotypes. In log-fitness space, this segment is curved and concave, implying that $\partial m(2)/\partial m(1)$ is a decreasing function of $m(1)$. Different environmental statistics are characterized by the frequencies $p(1)$ and $p(2)=1-p(1)$ of the two environmental states. The value of $p(1)$ sets the slope $-p(1)/p(2)$ of the isolines of growth rate that we should consider (9).

Depending on the value of $p(1)$, different cases arise. First, if $p(1)$ is too high or too low, there is no tangent to the Pareto front of slope $-p(1)/p(2)$ and the support point lies at one of the two extremities of the Pareto front. In these cases, the optimal strategy (crosses in Fig. 3B) is to adopt a constant phenotype – the phenotype optimal for the most frequent environmental state. When $p(1)$ takes an intermediate value, the isoline is tangent to the Pareto frontier at an intermediate support point, indicating an optimal strategy involving switching between the two possible phenotypes. As a function of the frequency of encountering different environments, there are thus two transitions, from non-switching to switching and to non-switching again. This succession of optimal strategies is read off as a function of the environmental frequency from the Pareto line (Fig. 3C).

One can make the problem more interesting by adding a third “generalist” phenotype, which does relatively well across both environments (Fig. 3D-F, green dot). This generalist creates a kink in the Pareto frontier, meaning that it will be optimal as a constant phenotype for a certain range of environmental conditions. Thus, depending on the frequencies of the two environmental states, the optimal strategy consists either of having a constant specialized phenotypes (blue or orange) when one environment is much more frequent than the other, a constant generalist phenotype (green) when the two environments have similar frequencies, or switching between a specialized phenotype and the generalist phenotype in intermediate situations (Fig. 3F). The transition from specialist to generalist was studied in a similar model in Donaldson-Matasci et al. (2008), but in the slightly different context of a continuous choice of strategies.

These conclusions generalize to an arbitrary number $d$ of environmental states. It follows from the graphical construction that for a given statistics of the environment, the number of discrete phenotypes between which the population may switch in optimal strategies is at most equal to the number of different environmental conditions, $d$: the subset of the extended fitness set corresponding to this switching is the polytope of dimension $d-1$ whose vertices are these $d$ phenotypes (a segment for $d=2$, a triangle for $d=3$). This observation may be viewed as extending to changing environments the principle of competitive exclusion stating that a single niche cannot support more than one species.

We complement the graphical analysis by analytical results in the simplest case of two environments and two phenotypes illustrated by Fig. 4A-C. Since only the relative fitnesses in each environment is relevant for the dynamics, we set without restriction of generality the replication rate of each phenotype in its preferred environment to 1. The other phenotype has a selective disadvantage, with replication rate $w_{x}<1$:

[TABLE]

The parameter $w_{x}$ can be interpreted as the degree of specialization: $w_{x}=1$ means no specialization, while $w_{x}=0$ means extreme specialization. Since $p(1)=1-p(2)$ and $\pi(1)=1-\pi(2)$, there are just two free parameters $p_{2}\equiv p(2)$ and $\pi_{2}\equiv\pi(2)$. In these variables the long-term growth rate (7) is written as

[TABLE]

To find the optimal fraction of the population with phenotype 2, $\pi_{2}^{\star}$, Eq. (11) is to be maximized over $\pi_{2}\in[0,1]$. The optimization yields

[TABLE]

with lower and upper bounds

[TABLE]

on the environmental frequencies for which diversification is optimal. The calculation recapitulates the conclusions from the graphical method (Fig. 4C). In the limit were selection is very stringent, $w_{1}\rightarrow 0$ and $w_{2}\rightarrow 0$, the transitions disappear, $p_{2}^{\rm lb}\to 0$ and $p_{2}^{\rm ub}\to 1$ and the optimal strategy reduces to proportional betting,

[TABLE]

In the context of biological bet-hedging, this result was already noted by Cohen Cohen (1966); it was also derived earlier in the context of gambling by Kelly Cover and Thomas (2005).

The range of environmental frequencies for which bet-hedging is favored over the non-switching strategies depends strongly on the selectivity of the environments (Fig. 4A). Consider for simplicity the symmetric case $w_{1}=w_{2}=w$, then non-switching strategies are favored for $\left|p_{2}-1/2\right|\geq(1-w)/[2(1+w)]$. In the limit, $w\rightarrow 0$, the strategy tends to proportional bet-hedging as discussed earlier. The larger $w$, the smaller the region of environmental frequencies for which switching strategies are optimal. As there is smaller variability in fitness across generations for the same phenotype, switching is less needed to hedge against environmental fluctuations.

Instead of considering transitions in optimal strategies as environmental frequencies are varied, we can also consider transitions as selection pressures are varied at fixed environmental frequencies (Fig. 4B). As selection pressures are decreased there are transitions to a non-switching strategy (white areas in Fig. 4B). The optimality of bet-hedging (shaded areas in Fig. 4B) for weak selection pressures depends on a precise matching between the asymmetry in selection pressures and environmental frequencies. This conclusion generalizes the results of Salathé et al. (2009), which considered numerically asymmetric fitness landscapes, $w_{1}\neq w_{2}$, but only with a symmetric environment, $p_{1}=p_{2}=1/2$.

IV Transitions between optimal immune strategies

Fitnesses achievable by single phenotypes (orange dots in Fig. 2) can fill a set delimited by a continuous line, called trade-off function, which is the Pareto frontier of non-switching strategies. It is common to consider such a continuous set of phenotypes with all possible switching strategies between them Levins (1968); Donaldson-Matasci et al. (2008). The Pareto frontier of switching strategies defined in the previous section then delimits the convex hull of that continuous set. The two Pareto frontiers (of switching and non-switching strategies) coincide if the trade-off function is concave, i.e. if the set of achievable phenotypes is convex; in that case non-switching strategies are optimal everywhere. Otherwise, similar transitions as in the previous section will arise Donaldson-Matasci et al. (2008). In some biological situations however, only some combinations of phenotypes along a trade-off function may be accessible at the same time, meaning that one cannot switch between all phenotypes on the trade-off line. Such a constraint on switching rates can induce discontinuous transitions, or cause the co-existence of multiple locally optimal solutions, as we now illustrate in a simple model of evolution of immunity.

Our illustrative example is a model that we proposed to explain the diversity of immune strategies observed across the tree of life Mayer et al. (2016). The purpose is to show how different strategies are associated with different statistics of pathogen dynamics. In its simplest form, the model has two environmental states, presence ($x=1$) or absence ($x=0$) of a pathogen. In a given strategy, it has two accessible phenotypes, protected ($\sigma=1$) or unprotected ($\sigma=0$). Strategies are represented by $\boldsymbol{f}=(f(x=0),f(x=1))$ as before.

The unprotected phenotype is fixed in fitness space: $\boldsymbol{f}=(f_{\rm base},f_{\rm inf})$ (grey dot in Fig. 5), where $f_{\rm inf}<f_{\rm base}$ is the reduced fitness in infected unprotected individuals. By constrast, the protected phenotype lies on a trade-off function: $\boldsymbol{f}=(f_{\rm con},f_{\rm def}(f_{\rm con}))$, with $f_{\rm con}\in[f_{\rm con}^{\rm min},f_{\rm con}^{\rm max}]$ (dashed purple line delimited by red and blue dots in Fig. 5). $f_{\rm con}<f_{\rm base}$ represents the reduced fitness due to the investment into the protection, while $f_{\rm def}>f_{\rm inf}$ is the fitness of protected individuals in presence of the pathogen.

The choice of $f_{\rm con}$ along the trade-off function sets the investment into the protection, and is part of the strategy: once this strategy is fixed, it is possible to switch between protected and unprotected phenotypes, but not between different points of the trade-off function. This constraint can be justified biologically by the high cost of plasticity that such switches would incur.

The function $f_{\rm def}(f_{\rm con})$ encodes the trade-off between the efficiency of the protection and its cost. By analogy with immune mechanisms in vertebrates, we interpret it in terms of adaptivity of the response within the lifetime of the organism, with higher adaptivity enabling lower cost at the expense of lower protective efficiency Mayer et al. (2016). We therefore refer to the maximally protective and costly strategy with $f_{\rm con}=f_{\rm con}^{\rm max}$ as innate immunity and to the minimally protective and costly strategy with $f_{\rm con}=f_{\rm con}^{\rm min}$ as adaptive immunity. Intermediate strategies with $f_{\rm con}^{\rm min}<f_{\rm con}<f_{\rm con}^{\rm max}$ are referred to as protoadaptive.

Within this model, the equation for long-term growth rate in an uncorrelated environment (7) becomes

[TABLE]

where $p\equiv p(x=1)$ is the probability of the presence of the pathogen and $\pi\equiv\pi(x=1)$ the probability of being protected. Here, the problem is not only to find the optimal switching probability $\pi^{\star}$, but also to find the optimal protection adaptability, $f_{\rm con}^{\star}$. To summarize, the problem is as follows: for a given $p$, $f_{\rm inf},f_{\rm base}$ and $f_{\rm def}(f_{\rm con})$, find $f_{\rm con}^{\star}$ and $\pi^{\star}$ that maximize long-term growth rate in Eq. 16.

We are particularly interested in transitions between $f_{\rm con}^{\star},\pi^{\star}$ taking intermediate or extremal values within their respective ranges. Given that each of these two variables can either reach its lower or upper bound or take an intermediate value, nine different cases may arise. However, since the level of adaptability of the response is inconsequential if none of the population is protected ($\pi^{\star}=0$), only seven qualitatively different immune defense strategies are relevant: tolerance ($\pi^{\star}=0$, grey dot in Fig. 5), innate ($\pi^{\star}=1$, $f_{\rm con}=f_{\rm con}^{\rm max}$, blue dot in Fig. 5), adaptive ($\pi^{\star}=1$, $f_{\rm con}=f_{\rm con}^{\rm min}$, red crossed dot in Fig. 5), protoadaptive ($\pi^{\star}=1$, $f_{\rm con}^{\rm min}<f_{\rm con}<f_{\rm con}^{\rm max}$, light blue line with purple dashes in Fig. 5), innate switching ($0<\pi^{\star}<1$, $f_{\rm con}=f_{\rm con}^{\rm max}$, blue line in Fig. 5), adaptive switching ($0<\pi^{\star}<1$, $f_{\rm con}=f_{\rm con}^{\rm min}$, red line in Fig. 5), and protoadaptive switching ($0<\pi^{\star}<1$, $f_{\rm con}^{\rm min}<f_{\rm con}<f_{\rm con}^{\rm max}$, light-blue line in Fig. 5).

Which of these strategies is optimal in a given environment? And what is the nature of the transitions between strategies as the frequency of encountering the pathogen is varied? Here, we apply the graphical method to answer these questions and show how the answers depend critically on the shape of the trade-off function. Our conclusions, summarized in Fig. 5, are supported by analytical results derived in Appendix B.

The simplest case is when adaptability comes at an excessive cost, as depicted in Fig. 5A: an innate switching strategy is then always preferable to an adaptive strategy. In this case, as the probability of encountering the pathogen increases, the optimal strategy transitions from tolerance (grey dot in Fig. 5A) to an innate defense strategy (blue dot in Fig. 5A) via an innate switching (blue line in Fig. 5A). When adaptability of the defense does not impair its effectiveness as severely, as in Fig. 5B, two new transitions occur. As the probability of encountering the pathogen increases, the optimal strategy now transitions from tolerance to, successively, adaptive switching, adaptive, protoadaptive and finally innate defense strategy. In other cases, a switching protoadaptive defense strategy may also be optimal, as in the case of the trade-off function of Fig. 5C. In this case, as the probability of encountering the pathogen increases, the optimal strategy transitions from tolerance to, successively, protoadaptive switching, protoadaptive and finally innate defense strategy. Finally, we may consider a case where the trade-off line is not convex as in Fig. 5D. The Pareto frontier is then not necessarily concave, and we might have first order transitions between strategies. For the trade-off shape shown in Fig. 5D, there is a transition from protoadaptive switching (blue line with purple dashes) directly to innate switching (blue line), with a discontinuity in the level of adaptability of the response.

V When and how to use memory in temporally correlated environments

In temporally correlated environments, the past phenotypes of an individual carry information about the next environmental state. The optimal switching strategy may thus involve memory, i.e. it may be advantageous for $\pi(\sigma|\sigma^{\prime})$ to depend on $\sigma^{\prime}$. Stochastic switching with memory serves an additional purpose relative to the memoryless switching strategies considered so far: in addition to providing a bet-hedging mechanism against the uncertainty of the environment, it provides the variation and heritability needed for tracking the environmental state. Here, we extend the previous analysis to characterize the conditions under which temporal correlations in environmental fluctuations favor switching strategies with memory over non-switching strategies. The graphical method does not extend to correlated environments but we show that the transitions between switching and non-switching strategies can be characterized analytically.

V.1 Insights from the adiabatic limit

It is instructive to start with long correlation times, when the duration of each environmental state is much longer than the time that it takes for the population to reach its steady state composition. In this adiabatic limit, the model is analytically solvable Kussell and Leibler (2005); Rivoire and Leibler (2011). We present the solution for the case where switching takes place between a number of different phenotypes, with each phenotype $\sigma$ being best in one environment $x$, which we denote by the same symbol $\sigma=x$ (other cases can in fact always be reduced to this one Rivoire and Leibler (2011)). A calculation based on a series of eigendecompositions of the growth matrix in different environments (see Appendix C for derivation) leads to an expression of the long-term growth rate as Rivoire and Leibler (2011)

[TABLE]

which involves the overlap $\Gamma(x,x^{\prime})$ between steady-state population compositions in environments $x,x^{\prime}$, given by

[TABLE]

if the environment changes, $x\neq x^{\prime}$, and $1$ otherwise.

Optimizing (17) over $\pi(x|x^{\prime})$ subject to the normalization constraint leads to $\pi^{\star}(x|x^{\prime})=p(x|x^{\prime})$. Within the adiabatic limit, the optimal strategy is therefore always to diversify, with switching rates equal to the environmental switching rates. This generalizes the result that proportional betting is optimal in the limit of strong selection, (15), to the case where reaching steady state takes longer but environmental switches are rarer. In contrast to the results in the previous section, switching is always favored in the adiabatic limit, even when selection is weak.

We can use the expression of (17) to ask how much each phenotype $\sigma$ should be specialized to its environement $x=\sigma$. Being more specialized means higher fitnesses of the adapted phenotypes, $f(x,x)$, at the expense of lower fitnesses for the maladapted phenotypes, $f(x,x^{\prime}\neq x)$, assuming a trade-off between the two. More specialized phenotypes have lower relative replication rate $w(x,x^{\prime})=f(x,x^{\prime})/f(x^{\prime},x^{\prime})$ [$w(x,x^{\prime})$ reduces to $w_{x^{\prime}}$ of (10) in the case of two environmental states]. Rewriting

[TABLE]

we see that specialization also implies lower overlaps $\Gamma(x,x^{\prime})$, and thus lower values for the second term in the long-term growth rate (17). On the other hand, the first term in (17) grows with $f(x,x)$, i.e. with higher specialization. Because of these contradictory terms, the optimal strategy along the trade-off between $f(x,x)$ and $f(x,x^{\prime})$ will depend on the details of trade-off function and of the environmental statistics. However, as environment fluctuations become slower, $p(x|x^{\prime}\neq x)\to 0$, the second term in (17) vanishes for $x\neq x^{\prime}$, letting the first term dominate. In that limit, highly specialized phenotypes become more and more advantageous. This observation is again in contrast with the results of the preceding section (Fig. 3D-F), which have shown that generalists are optimal under certain environmental conditions.

V.2 Connecting the limit of uncorrelated and adiabatically switching environments numerically

So far we have considered two opposite limits: temporally uncorrelated environments in Sec. III and IV, and temporally correlated environment with long correlation times in Sec. V.1. These two limits give very different answers to the questions of whether bet-hedging is desirable, or whether generalist phenotypes can be optimal. To study the intermediate regime between these two extremes, we first start by presenting the results of a numerical study, based on the recursion equation (1). We apply the numerical approach described in Ref. Mayer et al. (2016). In short, we approximate the long-term growth rate numerically by simulating for a large number of generations, and then use a derivative-free global optimization algorithm to roughly find the global optimum. In practice, we focus on two-state environments, which we characterize by their characteristic time scale, $t_{c}$, defined by $e^{-1/t_{c}}=1-p(1|2)-p(2|1)$ and the probability of being in state 2, $p(x=2)$. The numerical results show how the two limits are connected for the case without (Fig. 6) and with a generalist phenotype (Fig. 7). In temporally correlated strategies, phenotype frequencies vary with the environmental history. To represent strategies in a simple way that generalizes the case of memoryless strategies, we define $\pi(\sigma)$ as the steady-state frequency of phenotype $\sigma$ in a lineage, $\sum_{\sigma^{\prime}}\pi(\sigma|\sigma^{\prime})\pi(\sigma^{\prime})=\pi(\sigma)$. Consistent with results in the adiabatic limit, for large $t_{c}$ switching strategies dominate across the range of environmental frequencies (Figs. 6 and 7): $\forall\sigma,\pi^{\star}(\sigma)<1$. In the case where there is an intermediate, generalist phenotype ($\sigma=3$) the switching takes place primarily between the specialist types: $\pi^{\star}(\sigma=3)\ll 1$ for large $t_{c}$ (Fig. 7), consistent with the argument that specialized phenotypes are optimal in the adiabatic limit (Sec. V.1). The transition to a regime where non-switching strategies are optimal happens when the temporal correlations of the environment are of the order of the generation time, $t_{c}\sim 1$. In this regime, all three phenotypes (two specialists and one generalist) may co-exist in the optimal strategy, $\forall\sigma,\pi^{*}(\sigma)>0$. Recall that such mixtures involving more phenotypes than distinct environments are suboptimal in memoryless environments, $t_{c}=0$, as deduced from the graphical construction (see Sec. III.3).

V.3 An analytical result for intermediate timescales

We present here an approach to derive analytically the boundaries between optimal switching and non-switching strategies in correlated environments. The approach is based on an expansion at small switching rates of the Master equation of the joint environmental and population switching process near the transition boundary.

We first rewrite (1) as a recursion equation for the fraction of the population in each state $n_{t}(\boldsymbol{\sigma})=N_{t}(\boldsymbol{\sigma})/N_{t}$ with $N_{t}=\sum_{\boldsymbol{\sigma}}N_{t}(\boldsymbol{\sigma})$,

[TABLE]

where $Z_{t}$ is a normalization constant enforcing $\sum_{\boldsymbol{\sigma}}n_{t}(\boldsymbol{\sigma})=1$. Since $N_{T}=N_{0}\prod_{t=0}^{T}Z_{t}$ the long-term growth rate given by (3) becomes $\Lambda=\lim_{T\rightarrow\infty}\frac{1}{T}\sum_{t=0}^{T}\ln Z_{t}$.

For simplicity, we consider a two-state model again. We introduce the simplified notations $\pi(1|2)=\pi_{12}$, $\pi(2|1)=\pi_{21}$ for the type switching rates, $p(1|2)=p_{12}$, $p(2|1)=p_{21}$ for the environment switching rates, and denote $n_{t}(2)=n_{t}$, and $x_{t}(2)=x_{t}$. We use the same convention as in (10), $f(1,1)=f(2,2)=1$, $w(2,1)=w_{1}$ and $w(1,2)=w_{2}$. This allows us to rewrite the recursion equation as

[TABLE]

with

[TABLE]

To analyze the transition from an optimal strategy where all individuals have phenotype 1 to a strategy with some switching to the other phenotype, we need to know whether a small $\pi_{12}$ is better than $\pi_{12}=0$ – if that is the case, then switching is advantageous. We thus consider $\pi_{12}\ll 1$ and $n_{t}\ll 1$. The recursion (21) becomes at leading order in $n_{t}$ and $\pi_{12}$:

[TABLE]

$\ln Z_{t}$ can also be expanded:

[TABLE]

Over long times the joint environmental-population process is ergodic. The long-term growth rate is thus given as $\langle\ln Z\rangle$, where $\langle.\rangle$ indicates an average over the steady state distribution of $x,n$. No switching ($n=0$) gives a long-term growth rate of $\langle x\rangle\ln w_{2}$ Thus the difference in long-term growth rate between stochastic switching and the single-phenotype strategy is

[TABLE]

which shows that stochastic switching is advantageous if

[TABLE]

We can identify the right-hand side of this equation with the lower bound environmental frequency $p_{2}^{\rm lb}$ in uncorrelated environments defined in (13). When there is no memory, $n$ and $x$ are uncorrelated, the left-hand side reduces to $\langle x\rangle=p(x=2)$ and we recover the result of (12). If there is memory, then $n$ and $x$ are positively correlated through the effects of selection on the population composition, which increases the fraction on the left-hand side. This leads us to a first important conclusion: switching is favored over non-switching strategies under a wider range of environmental parameters in the presence of temporal autocorrelation.

We go further and calculate analytically the left-hand side of (26) at the transition. Some algebra shows that $\rho_{1,t}=\langle(1-x_{t})n_{t}\rangle$ and $\rho_{2,t}=\langle x_{t}n_{t}\rangle$ satisfy the recursion

[TABLE]

where we use the short-hand notation $p_{2}=p_{21}/(p_{12}+p_{21})=\langle x\rangle$ for the average fraction of generations the environment is in state $2$. Therefore at steady state, $\rho_{\sigma,t}=\rho_{\sigma,t+1}=\rho_{\sigma}$, we have

[TABLE]

where we recall that $e^{-1/t_{c}}=1-p(1|2)-p(2|1)$ quantifies memory in the environment. The expression in (29) is a decreasing function of $\pi_{21}$ so its maximum is achieved in the limit of $\pi_{21}$ going to zero. Setting $\pi_{21}=0$ in (29) and plugging the result into (26), we obtain the condition needed for optimal switching to outperform always being in state $\sigma=1$:

[TABLE]

The second transition, between optimal switching and always being in state $\sigma=2$, is given by the replacements $w_{1}\to w_{2}$, $w_{2}\to w_{1}$, $p_{2}\to 1-p_{2}$, yielding the condition:

[TABLE]

These transitions reduce to (12) in the limit of no environment memory, $t_{c}=0$. The transition curves reach $p_{2}=0$ and $p_{2}=1$ at $t_{c}=-1/\ln(w_{2})$ and $t_{c}=-1/\ln(w_{1})$, respectively. The resulting phase diagram is shown in Fig. 6 along with a numerical optimization, which confirms the results.

The analytical results show that temporal correlations in the environment favor the evolution of stochastic switching. We can compare to the case of uncorrelated environments considered in Fig. 4. While switching is only optimal in uncorrelated environments if selection is strong in both environments (blue line in Fig. 8), temporally-correlated environments make it optimal for smaller or asymmetric selection (e.g. red line in Fig. 8). We may interpret this broadening of the range where switching is optimal by noting that, in correlated environments, switching does not just contribute to bed-hedging but also to adaptively tracking the state of the environment.

V.4 Continuous time limit

Lastly, we discuss the continuous time limit of (1) where our results take a simple form. The limit is obtained by rescaling the switching rates, growth rates, and times by $\delta t$, $p(x|x^{\prime})\to p(x|x^{\prime})\delta t$ for $x\neq x^{\prime}$, $\pi(\sigma|\sigma^{\prime})\to\pi(\sigma|\sigma^{\prime})\delta t$ for $\sigma\neq\sigma^{\prime}$, and $\ln[f(\sigma,x)]\to m(\sigma,x)\delta t$, $t\to t/\delta t$, $t_{c}\to t_{c}/\delta t$, $\ln w_{x}\to\ln w_{x}/\delta_{t}$, $\Lambda\to\Lambda/\delta t$ and sending $\delta t\to 0$, which yields

[TABLE]

where $A^{x(t)}_{\sigma,\sigma^{\prime}}=m(\sigma,x(t))\delta_{\sigma,\sigma^{\prime}}+\pi(\sigma|\sigma^{\prime})$.

We can take the limit of the results obtained in Sec. V.3 to see how temporal autocorrelation influences the results in this case. From (30) we obtain

[TABLE]

and from (31)

[TABLE]

The formulas are simpler and notably linear in the correlation time $t_{c}$. The range of environmental frequencies for which stochastic switching is optimal thus grows linearly with the environmental correlation time scale $t_{c}$, as $-2t_{c}\ln w_{1}\ln w_{2}/\ln(w_{1}w_{2})$.

The point $p_{2}=0$ is reached by the first transition (33) from the non-switching to switching regime for $t_{c}=-1/\ln w_{2}$, and the point $p_{2}=1$ reached by the second transition for $t_{c}=-1/\ln w_{1}$ (34), as in the discrete time case. In the limit of no environmental memory, $t_{c}=0$, the two transitions are at the same point $\ln w_{1}/\ln(w_{1}w_{2})$: this means that bet-hedging is never advantageous and the transition is from one single-phenotype strategy to the other. This is in contrast with the solution in discrete time, where there always is a window in which bet-hedging is favored, regardless of the environmental memory. Since in any finite time interval, the environment cycles through all its states, the population effectively only sees the mean environment. The long-term growth rate in continuous time is thus given by

[TABLE]

which is a linear function in $\pi(\sigma)$. $\Lambda$ is optimized by putting all weight on the phenotype $\sigma$ with largest average fitness $\sum_{x}p(x)f(\sigma,x)$. Thus no switching strategies can be optimal and the optimal strategy always consists of a single phenotype.

VI Discussion

Our results provide a unified view of transitions between optimal adaptive strategies in randomly fluctuating environments. By revisiting the fitness set representation of Levins Levins (1968), valid for temporally uncorrelated environments, we presented a graphical method, supplemented by analytical calculations, to determine the transitions between bet-hedging and single-phenotype strategies, as well as between specialist and generalist phenotypes (Fig. 3), generalizing previous results Cohen (1966); Levins (1968); Iwasa and Haccou (1995); Cover and Thomas (2005); Rivoire and Leibler (2011); Donaldson-Matasci et al. (2008). Extending the method to phenotypes constrained by a trade-off function, we constructed graphically and calculated analytically the transitions between optimal strategies of diversification and adaptability in a simple model of evolution of immunity Mayer et al. (2016) (Fig. 5).

As noticed in previous studies, temporal correlations in the environmental conditions influences the choice of optimal adaptation strategies Ratcliff et al. (2015); Botero et al. (2015); Skanata and Kussell (2016). The intermediate timescale regime, where the environmental correlation time is of the same order as the generation time, has been notoriously difficult to handle analytically. Here, we presented an analytical approach to show how temporal correlations in environments can be exploited by switching strategies that keep some memory of previous phenotypes. Our results show that temporal correlations broaden the range of selective pressures for which a switching strategy is better than a single-phenotype one. Everything else being equal, switching strategies are thus more favorable in correlated environments than in uncorrelated environments. To our knowledge, only one other analytical approach is available to analyze optimal strategies in correlated environments Skanata and Kussell (2016).

The results are independent of mechanisms, which may take different forms. For instance, one mechanism to achieve a generalist phenotype is through plasticity, i.e., a generalist phenotype may partly or totally be induced by the environmental condition. In our approach, however, only the value of the replication rate $f(\sigma,x)$ in environmental condition $x$ given the inherited type $\sigma$ matters, not the process by which it is achieved. Only when the induced phenotype may be transmitted to the next generation, as for instance with the Lamarckian CRISPR-like strategy of Mayer et al. (2016), does the distinction between inherited and induced phenotype, and therefore the concept of plasticity, become relevant.

Possible extensions of our results include the influence of non-random environmental changes, such as periodic environments Gaál et al. (2010); Carja et al. (2014); Botero et al. (2015); Skanata and Kussell (2016), constraints on relative switching rates Salathé et al. (2009); Gaál et al. (2010); Patra and Klumpp (2015), active sensing mechanisms Kussell and Leibler (2005) and heritable plasticity Rivoire and Leibler (2014); Mayer et al. (2016), or finite population size effects Xue and Leibler (2017). Some of these factors are known to lead to transitions between adaptive strategies, e.g. the variability of environmental durations Skanata and Kussell (2016), or cause the transitions to become discontinuous, e.g. when switching rates are constrained to be independent of phenotype Salathé et al. (2009); Gaál et al. (2010); Patra and Klumpp (2015).

Acknowledgements. We thank B. Xue and O. Carja for helpful discussions. The work was supported by grant ERCStG n. 306312.

Appendix A Optimal strategies by mapping to unit simplex

If the sum of fitnesses of a phenotype over environments $f(\sigma)=\sum_{x}f(\sigma,x)$ is constant for all phenotypes, then any mixture will also have a constant sum of fitnesses. The normalization constraint on $\boldsymbol{\pi}$ then translates into an equivalent constraint on $\boldsymbol{f}$. The solution of the optimization problem in its fitness form is then particularly simple Iwasa and Haccou (1995); Donaldson-Matasci et al. (2008). Therefore, where possible, it is worthwhile to map the optimization problem to this simpler case by a rescaling of fitnesses in different environments. Here we show how to perform the rescaling and the conditions under which it is possible. Fig. S1 illustrates such a mapping in a simple case with two environmental states.

The optimization problem is invariant with respect to additions of terms that are constant with respect to the variables over which one optimizes. Specifically, we can add the term $\sum_{x}p(x)\log c(x)$ to Eq. (7) with all positive $c(x)$, which is constant with respect to $\boldsymbol{\pi}$. This gives us a new optimization problem with the objective function $\tilde{\Lambda}=\sum_{x}p(x)\log[f(x)c(x)]=\sum_{x}p(x)\log\tilde{f}(x),$ in terms of the rescaled fitnesses $\tilde{f}(x)=\sum_{\sigma}\pi(\sigma)\tilde{f}(\sigma,x)$ and $\tilde{f}(\sigma,x)=f(\sigma,x)c(x)$. The equivalence of these problems shows that a rescaling of the axes of fitness space does not change the optimal adaptation strategy.

We can now try and use this rescaling to make the sum of scaled fitnesses a constant, which we chose to be $1$ without restriction of generality. This means we aim to chose $c(x)$, such that $\sum_{x}\tilde{f}(\sigma,x)=\sum_{x}f(\sigma,x)c(x)=1$ holds for all $\sigma$. In matrix-vector notation we can represent these conditions as the systems of equation

[TABLE]

where $\boldsymbol{1}=(1,1,...,1)^{T}$ is the vector of all ones and $\boldsymbol{F}$ the matrix of phenotype fitness profiles with entries $F_{\sigma,x}=f(\sigma,x)$. Eq. (36) requires that the scalar products of $\boldsymbol{c}$ with the row vectors of $\boldsymbol{f}$ (the phenotypes fitness profiles) are equal to $1$ for all rows. The vector $\boldsymbol{c}$ thus is a normal vector to the hyperplane spanned by the fitness profiles. The mapping is therefore only possible if a hyperplane passing through all fitness profiles exists. The intercept $d_{x}$ of the hyperplane with the $x$ axis is given by $\boldsymbol{c}(d_{x}\boldsymbol{e}_{x})=1\Leftrightarrow d_{x}=1/c_{x}$, where $\boldsymbol{e}_{x}$ is the $x$-th unit vector. Eq. (36) thus specificies that we should rescale fitnesses by dividing through these intercepts to achieve our goal of mapping the problem to the unit simplex. The positivity of the scaling constants $c(x)$ puts further requirements on $\boldsymbol{F}$ for the mapping to work: geometrically, all intercepts need to be positive, or algebraically, the inverse of the fitness matrix needs to have all positive row sums. In the case where $\boldsymbol{F}$ is an invertible matrix fitnesses should be rescaled using $\boldsymbol{c}=\boldsymbol{F}^{-1}\boldsymbol{1}.$ If the scaling is possible then in the scaled variables the normalization constraint on $\boldsymbol{\pi}$ leads to a normalization constraint on $\boldsymbol{\tilde{f}}$.

We can derive the optimal fitness profile using the Lagrange formalism. The Lagrangian of the optimization problem is

[TABLE]

where we have assumed that the optimum is in the interior of $D_{\tilde{f}}$, i.e. none of the non-negativity constraints on elements $\boldsymbol{\pi}$ are active. Taking the derivative with respect to $\tilde{f}(x)$ and setting it to zero yields $\tilde{f}^{\star}(x)=p(x)/\lambda$. As we have rescaled fitnesses such that the sum of fitnesses scale to $1$ the Lagrange multiplier is $\lambda=1$. In the rescaled variables the optimal strategy thus allocates fitness to each environment proportional to its frequency:

[TABLE]

From the optimum in the rescaled variables the optimum in the original variables can be obtained by reversing the scaling $f^{\star}_{i}=\tilde{f}^{\star}_{i}/c_{i}$.

Due to the non-negativity constraints on $\pi(\sigma)$, which we have neglected so far in the discussion, only a subset of the unit simplex is accessible if there are no phenotypes that are not completely specialized to the different environments. Where the unconstrained solution lies outside the feasible region a value on the boundary of the fitness set is constrained optimum instead. The fitness allocation among the remaining unconstrained directions still is proportional to the frequency of the respective environments.

Appendix B Analytical results on optimal immune strategies in uncorrelated environments

B.1 Optimization problem

The cost function of the optimization problem is the long-term population growth rate, which depends on the environmental statistics and the chosen strategy. The long-term growth rate in uncorrelated environments for a given $p,f_{\rm base},f_{\rm inf},f_{\rm def}(f_{\rm con})$ is given by (16), which we recall here

[TABLE]

To find the optimal strategy we need to solve the following optimization problem

[TABLE]

The optimization consists in finding the (global) maximum of a two-variable objective function subject to bound constraints on both variables. In the following derivations we make use of the ordering of the costs in the non-trivial case $f_{\rm base}\geq f_{\rm con},f_{\rm def}>f_{\rm inf}$ and of the Pareto condition on the trade-off line $f_{\rm def}^{\prime}(f_{\rm con})<0$.

This problem can be solved numerically, but as is shown in the following a lot of information is available from a purely analytical treatment of the optimization problem. The Karush-Kuhn-Tucker conditions give necessary conditions for local optimality of a point $\pi^{\star},f_{\rm con}^{\star}$. For bound constrained problems these conditions boil down to the statement that the partial derivative of the objective function with respect to either variable needs to be Boyd and Vandenberghe (2004): zero if the variable is in the interior of its feasible interval, negative if the variable is at the lower end of its feasible domain, and positive if the variable is at the upper end of its feasible domain. Expressed in equations the necessary conditions for $\pi^{\star},f_{\rm con}^{\star}$ to be locally optimal is that

[TABLE]

and that

[TABLE]

The conditions provide only necessary but not sufficient conditions for local optimality. A condition ensuring sufficiency is that the Hessian at the optimum constricted to the feasible directions is negative definite.

B.2 Derivatives of the cost function

The optimality conditions derived in the previous subsection involve the derivatives of the cost function, which can be obtained using simple algebra and which we give below. The derivative of the cost function with respect to $\pi$ is given by

[TABLE]

For sufficiency we also need to consider the second derivatives of the cost function:

[TABLE]

The second derivative with respect to $\pi$ is always negative which shows that the long-term growth rate is a concave function of $\pi$. For a fixed value of $f_{\rm con}$ the optimization thus corresponds to a maximization of a concave function and always yields a unique optimum. The second derivative of the long-term growth rate with respect to $f_{\rm con}$ is also always negative, if $f_{\rm def}^{\prime\prime}\leq 0$. This condition on the trade-off function is fulfilled if individuals might bet hedge in their degree of specialization in environment $1$. Otherwise the second derivative might be positive for some $p$ and there can thus exist more than one local optimal in the full optimization problem.

A sufficient condition for having a local maximum is the negative definiteness of the Hessian. As one of its diagonal elements is always negative this is equivalent to showing that the determinant of the Hessian is positive. The determinant of the Hessian at an interior stationary point can be calculated to be

[TABLE]

It follows that for $f_{\rm con}^{\star}$ to be optimal for an intermediate $\pi^{\star}$ the trade-off curve needs to be locally concave $f_{\rm def}^{\prime\prime}(f_{\rm con}^{\star})<0$.

B.3 Regions of local optimality for different phases

The optimality conditions Eqs. (41) and (42) allow for three different cases for $\pi^{\star}$ and $f_{\rm con}^{\star}$ each. This makes for a total of $3\times 3=9$ different combinations. For the case $\pi^{\star}=0$ the growth rate does not depend on $f_{\rm con}^{\star}$, so there exists up to seven distinct phases. Under which conditions are these strategies locally optimal? In the following we analytically derive the interval of $p$ for which these strategies are optimal.

B.3.1 Tolerance ($\pi^{\star}=0$, arbitrary $f_{\rm con}^{\star}$)

The condition of local optimality is $\partial_{\pi}\Lambda(0,f_{\rm con}^{\star})\leq 0$ (see Eq. (41)), which needs to hold for all feasible $f_{\rm con}^{\star}$. This translates to the condition $p\leq\frac{f_{\rm inf}(f_{\rm base}-f_{\rm con}^{\star})}{f_{\rm base}f_{\rm def}^{\star}-f_{\rm inf}f_{\rm con}^{\star}}$. The condition needs to hold for the $f_{\rm con}^{\star}$ giving the strictest bound. Tolerance thus is optimal for

[TABLE]

i.e. for the rarest pathogens. If the adaptive strategy comes without constitutive cost ($f_{\rm con}^{min}=0$), then the tolerance phase disappears ($p^{(0)}=0$). Where the phase exists it is followed by one of the bet hedging strategies.

B.3.2 Innate ($\pi^{\star}=1,f_{\rm con}^{\star}=f_{\rm con}^{min}$)

From (41) the condition of local optimality is $\partial_{\pi}\Lambda(1,f_{\rm con}^{min})\geq 0$. This translates to the condition

[TABLE]

The second optimality condition (42) is $\partial_{f_{\rm con}}\Lambda(1,f_{\rm con}^{min})\geq 0$, leading to

[TABLE]

Both conditions need to hold at the same time for local optimality so an innate strategy is optimal for

[TABLE]

i.e. for the most frequent pathogens. Depending on which of the two conditions is more stringent it is followed either by a innate bet hedging strategy or a protoadaptive phase.

B.3.3 Adaptive ($\pi^{\star}=1,f_{\rm con}^{\star}=f_{\rm con}^{max}$)

Eq. (41) leads to

[TABLE]

and Eq. (42) to

[TABLE]

Taken together an adaptive strategy is optimal for

[TABLE]

B.3.4 Protoadaptive ($\pi^{\star}=1$, intermediate $f_{\rm con}^{\star}$)

Eq. (41) leads to $p\geq\frac{(f_{\rm base}-f_{\rm con}^{\star})f_{\rm def}^{\star}}{f_{\rm base}f_{\rm def}^{\star}-f_{\rm inf}f_{\rm con}^{\star}}$ and Eq. (42) to $p=\frac{f_{\rm def}^{\star}}{f_{\rm def}^{\star}-f_{\rm con}^{\star}f_{\rm def}^{\prime}(f_{\rm con}^{\star})}$. The two conditions together lead to

[TABLE]

i.e. the derivative of the trade-off function needs to be more shallow then the derivative of costs of a mixture with the current type. As we have an intermediate level of regulation we need to check the second derivative to assure the extremum is a maximum. As shown in the main text this leads to the condition $\frac{\mathrm{d}^{2}\ln f_{\rm def}}{\mathrm{d}(\ln f_{\rm con})^{2}}<0$. If the trade-off function is assumed to be fulfill both conditions everywhere and to be smooth then by the intermediate value theorem there is a $f_{\rm con}^{\star}$, which is optimal for a $p$ in the region

[TABLE]

B.3.5 innate switching (intermediate $\pi^{\star}$, $f_{\rm con}^{\star}=f_{\rm con}^{min}$)

Eq. (41) leads to

[TABLE]

with

[TABLE]

and Eq. (42) to

[TABLE]

i.e. the derivative of the trade-off shape needs to be steeper then the line joining the unprotected state.

B.3.6 adaptive switching (intermediate $\pi^{\star}$, $f_{\rm con}^{\star}=f_{\rm con}^{max}$)

Eq. (41) leads to

[TABLE]

with

[TABLE]

and Eq. (42) to

[TABLE]

B.3.7 protoadaptive switching (intermediate $\pi^{\star},f_{\rm con}^{\star}$)

Eq. (41) leads to

[TABLE]

with

[TABLE]

and Eq. (42) to

[TABLE]

The derivative needs to be equal to the slope of the line connecting the fitness profile to the non-protected type. The sufficiency condition $\det H(\pi^{\star},f_{\rm con}^{\star})>0$ leads to

[TABLE]

Appendix C Derivation of long-term growth rate in the adiabatic limit

The study of the adiabatic limit in which the durations of environmental periods are large relative to the time scales of population composition change goes back to Kussell and Leibler (2005). Mathematically the long-term growth rate can be approximated by an eigenvalue perturbation approach. In the following we give a derivation following the notations of Rivoire and Leibler (2011).

The transfer matrix connecting the population composition at successive time points is $\langle\sigma^{\prime}\rvert A^{(x)}\lvert\sigma\rangle=f(\sigma^{\prime},x)\pi(\sigma^{\prime}|\sigma)$ (in bra-ket notation), which one can decompose as $A^{(x)}=A_{0}^{(x)}+A_{1}^{(x)}$ with

[TABLE]

and

[TABLE]

Using this decomposition we treat $A_{1}^{(x)}$ as a perturbation to $A_{0}^{(x)}$ to approximately solve the eigenvalue problem of $A^{(x)}$. As $A_{0}$ is diagonal its eigenvalues are $\lambda_{0,\sigma}=f(\sigma,x)$ with corresponding eigenvectors $\lvert\sigma\rangle$, which have all but the $\sigma$-th element set to zero. Applying the formulas for the eigenvalues and eigenvectors of the perturbed problem we obtain

[TABLE]

and the corresponding right eigenvectors

[TABLE]

In order to calculate overlaps we also need to calculate left eigenvectors. The left eigenvectors of $A_{0}^{(x)}$ are equal to its right eigenvectors as its a diagonal matrix. The left eigenvectors of the perturbed problem are

[TABLE]

Let us assume that for every environment $x$ there is a type $\sigma=x$, which provides optimal growth. The overlap between the largest eigenvectors in environments $x$ and $x^{\prime}$ is given by

[TABLE]

with

[TABLE]

In the adiabatic limit the long-term growth rate is given by

[TABLE]

where we have defined $\Gamma(x,x)=1$.

We can write out the sums in the case of an environment switching between two states as

[TABLE]

To compare the best switching strategies using phenotypes of fitness $\boldsymbol{f}$ or $\boldsymbol{\tilde{f}}$ we calculate the long-term growth rate difference

[TABLE]

where we have used short-hand notations for the environmental switching frequencies as introduced in the text. Setting $\Delta\Lambda=0$ we can solve for the transition line between the two sets of phenotypes,

[TABLE]

Such an analysis can be applied to the case where a generalist phenotype is on the Pareto frontier to find when switching only uses specialists (Fig. 7). To do so we compare the growth rate of switching between the specialist phenotypes $\sigma=1$ and $\sigma=2$ to the growth rates of switching between one of the specialists and the generalist $\sigma=3$. (81) then gives an approximate result for the time scale of environmental correlations above which switching only involves specialists.