Restart could optimize the probability of success in a Bernoulli trial

TL;DR

This paper explores how restart strategies can optimize the likelihood of achieving a desired outcome in Bernoulli-like stochastic processes, revealing universal principles for maximizing success probability.

Contribution

It demonstrates that restart can significantly influence splitting probabilities in Bernoulli processes and identifies conditions for optimal restart to maximize success likelihood.

Findings

Restart can alter the probability of process outcomes.

Optimal restart strategies can maximize the chance of desired completion.

Universal aspects of optimal restart behavior are identified.

Abstract

Recently noticed ability of restart to reduce the expected completion time of first-passage processes allows appealing opportunities for performance improvement in a variety of settings. However, complex stochastic processes often exhibit several possible scenarios of completion which are not equally desirable in terms of efficiency. Here we show that restart may have profound consequences on the splitting probabilities of a Bernoulli-like first-passage process, i.e. of a process which can end with one of two outcomes. Particularly intriguing in this respect is the class of problems where a carefully adjusted restart mechanism maximizes probability that the process will complete in a desired way. We reveal the universal aspects of this kind of optimal behaviour by applying the general approach recently proposed for the problem of first-passage under restart.

Equations

$$p_r=\frac{r+\alpha }{\alpha e^{\sqrt{(r+\alpha )/D}{x}_0}+r}.$$

$$x=I(T<R)y_T+I(T\ge R)x^{\prime },$$

$$p_r=⟨x⟩=\frac{⟨I(T<R)y_T⟩}{⟨I(T<R)⟩}.$$

$$p_r=\frac{{\tilde{P}}^s(r)}{\tilde{P}(r)},$$

$$T_r=I(T\ge R)(R+T_r^{\prime })+I(T<R)T,$$

$$⟨T_r^s⟩=\frac{⟨I(T>R)R⟩}{⟨I(T<R)⟩}+\frac{⟨I(T<R)y_{T}T⟩}{⟨I(T<R)y_T⟩}.$$

$$⟨T_r^s⟩=⟨T_r⟩-\frac{d\ln p_r}{dr},$$

$$⟨T_{r^{\ast }}^s⟩=⟨T_{r^{\ast }}⟩.$$

$$⟨T_{t^{\ast }}^s⟩\ge ⟨T_{t^{\ast }}⟩.$$

$$P(T)=\frac{x_0}{\sqrt{4\pi D{T}^3}}{e}^{-\alpha T-x_0^2/4DT}+\alpha e^{-\alpha T}\text{mbox}erf\left(\frac{x_0}{\sqrt{4DT}}\right),$$

$${\partial }_t\rho =D{\partial }_x^2\rho -(\alpha +r)\rho +r\delta (x-x_0){\int }_0^{\infty }dy\rho (y,t),$$

$${\tilde{\rho }}_s(x)={\int }_0^{\infty }{e}^{-st}\rho (x,t)dt,$$

$$s{\tilde{\rho }}_s-\delta (x-x_0)=D{\partial }_x^2{\tilde{\rho }}_s-(r+\alpha ){\tilde{\rho }}_s+r\delta (x-x_0){\int }_0^{\infty }dy{\tilde{\rho }}_s(y).$$

$$\tilde{\rho}_{s}(x)=\left\{\begin{array}[]{ll}A\sinh\gamma_{s}x,\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ x\leq x_{0},\\ Ae^{\gamma_{s}(x_{0}-x)}\sinh\gamma_{s}x_{0},\ \ \ x>x_{0},\end{array}\right.$$

$$A=\frac{{\gamma }_s}{(s+\alpha )e^{{\gamma }_s{x}_0}+r}.$$

$$\tilde{j}(s)=D[{\partial }_x\tilde{\rho }(x){]}_{x=0}=\frac{s+r+\alpha }{(s+\alpha )e^{{\gamma }_s{x}_0}+r}.$$

$$p_r={\int }_0^{\infty }j(t)dt=\tilde{j}(0)=\frac{r+\alpha }{\alpha e^{\sqrt{(r+\alpha )/D}{x}_0}+r},$$

$${\tilde{P}}^s(s)=\sqrt{\frac{x_0^2}{4\pi D}}{\int }_0^{\infty }{T}^{-3/2}\exp \left(-(s+\alpha )T-\frac{x_0^2}{4DT}\right)dT=e^{-\sqrt{(\alpha +s)/D}{x}_0},$$

$${\tilde{P}}^f(s)=\alpha {\int }_0^{\infty }{e}^{-(\alpha +s)T}\text{mbox}erf\left(\sqrt{\frac{x_0^2}{4DT}}\right)dT=\frac{\alpha }{\alpha +s}(1-e^{-\sqrt{(\alpha +s)/D}{x}_0}).$$

$$T_r=T_{\text{on}}+I(T\ge R)(R+T_r^{\prime })+I(T<R)T,$$

$$⟨T_r⟩=\frac{⟨T_{\text{on}}⟩+⟨\min (T,R)⟩}{⟨I(T<R)⟩}.$$

$$⟨T_r⟩=\frac{r⟨T_{\text{on}}⟩+1-\tilde{P}(r)}{r\tilde{P}(r)}.$$

$$⟨T_r^s⟩=\frac{⟨T_{\text{on}}⟩+⟨I(T>R)R⟩}{⟨I(T<R)⟩}+\frac{⟨I(T<R)y_{T}T⟩}{⟨I(T<R)y_T⟩}.$$

$$⟨I(T<R)⟩=r{\int }_0^{\infty }{\int }_0^R{e}^{-rR}P(T)dRdT={\int }_0^{\infty }{e}^{-rR}P(R)dR=\tilde{P}(r),$$

$$⟨I(T<R)y_T⟩=r{\int }_0^{\infty }{\int }_0^R{e}^{-rR}{P}^s(T)dRdT={\int }_0^{\infty }{e}^{-rR}{P}^s(R)dR={\tilde{P}}^s(r),$$

$$⟨I(T>R)R⟩=r{\int }_0^{\infty }{\int }_R^{\infty }R{e}^{-rR}P(T)dRdT={\int }_0^{\infty }{\int }_R^{\infty }{e}^{-rR}P(T)dRdT-{\int }_0^{\infty }R{e}^{-rR}P(R)dR=$$

$$={\int }_0^{\infty }{\int }_R^{\infty }{e}^{-rR}P(T)dRdT-{\int }_0^{\infty }R{e}^{-rR}P(R)dR=\frac{1}{r}-\frac{1}{r}{\int }_0^{\infty }{e}^{-rR}P(R)dR+\frac{d}{dr}{\int }_0^{\infty }{e}^{-rR}P(R)dR=$$

$$=\frac{1}{r}-\frac{1}{r}\tilde{P}(r)+\frac{d\tilde{P}(r)}{dr},$$

$$⟨I(T<R)y_{T}T⟩=r{\int }_0^{\infty }{\int }_0^{R}T{e}^{-rR}{P}^s(T)dRdT={\int }_0^{\infty }R{e}^{-rR}{P}^s(R)dR=-\frac{d}{dr}{\int }_0^{\infty }{e}^{-rR}{P}^s(R)dR=$$

$$=-\frac{d{\tilde{P}}^s(r)}{dr},$$

$$⟨T_r^s⟩=\frac{⟨T_{\text{on}}⟩}{\tilde{P}(r)}+\frac{1-\tilde{P}(r)}{r\tilde{P}(r)}+\frac{1}{\tilde{P}(r)}\frac{d\tilde{P}(r)}{dr}-\frac{1}{{\tilde{P}}^s(r)}\frac{d{\tilde{P}}^s(r)}{dr}=\frac{r⟨T_{\text{on}}⟩+1-\tilde{P}(r)}{r\tilde{P}(r)}-\frac{d}{dr}\ln \frac{{\tilde{P}}^s(r)}{\tilde{P}(r)}.$$

$$p^{\text{exp}}=\underset{t\to \infty }{\lim }\frac{1}{t}{\int }_0^{t}X(t)dt=\underset{t\to \infty }{\lim }\frac{1}{t}\left(\sum _{i=1}^{N(t)}{x}_i{T}_i+x_{N(t)+1}(t-t_{N(t)})\right)=\underset{t\to \infty }{\lim }\frac{1}{t}\sum _{i=1}^{N(t)}{x}_i{T}_i.$$

Full text

Stochastic processes subject to restart appear in many disciplines including physics, chemistry, biology and computer science. Restart means sudden interruption of a process followed by its start anew. In some contexts restart is an integral part of a phenomenon under study (e.g. substrate unbinding in enzymatic reactions Reuveni_2014 and recovery of RNA polymerase from the backtracked state Roldan_PRE_2017 ), while in other it plays a role of an external control tool (e.g. reinitialization of a randomized computer algorithm Luby_1993 ; Montanari_2002 and reduction of growing tumor to its initial size by chemical treatment Gupta_2014 ).

Significant amount of research effort has been dedicated towards study of the effect of restart on the first-passage properties. The growth of interest in this problem was triggered by the surprising observation that restart may significantly reduce the mean first-passage time (MFPT). Over recent years it has been demonstrated in a range of diverse examples that a carefully chosen restart rate can bring the MFPT to a minimum EM_2011 ; Evans_2011 ; Whitehouse_2013 ; Evans_2014 ; Kusmierz_2014 ; Kusmierz_2015 ; Pal_2016 ; Eule_2016 ; Nagar_2016 . Along with the investigation of particular cases, we witness ongoing attempts to reveal the general principles allowing to navigate in a vast space of first-passage problems under restart. Remarkable result of those attempts is the discovery of universality displayed by all optimally restarted processes Reuveni_PRL_2016 ; Reuveni_PRL_2017 .

To the best of our knowledge, first-passage processes under restart considered so far had only one way of completion. Say, diffusion mediated search with stochastic resetting to initial position EM_2011 – a classical example of a first-passage problem under restart – ends if and only if a searcher finds a target. However, real-life settings often offer a variety of possible ways in which stochastic process can complete. Plurality of the process outcomes may arise from the competition among several different first-passage phenomena or due to multiple thresholds for one and the same first-passage mechanism. Assume, for instance, that gambler stops playing after winning a certain amount of money or getting ruined, whichever happens first Edwards_1983 ; Feller_1968 . In many-target search problems and diffusion-limited reactions, different completion scenarios may correspond to finding of different targets Condamin_2007 ; Condamin_2007_1 ; Condamin_2008 ; Meyer_2012 ; Calandre_2012 ; Benichou_2015 ; Calandre_2014 . In search problems with time constraints, a search process can finish either by target detection or by searcher/target death Abad_2012 ; Yuste_2013 ; Abad_2013 ; Abad_2013_1 ; Abad_2015 ; Campos_2015 ; Meerson_2015 ; Grebenkov_2016_1 . When there are several competitive paths of chemical reaction, an individual molecule may be converted into one product or other depending on which path has been realized Rehbein_2011 ; Rehbein_2015 ; Martin-Soomer_2016 . Similarly, a protein may fold along one of many possible pathways to one of multiple native states Solomatin_2010 ; Marek_2011 ; Hyeon_2012 ; Paudel_2014 ; Hinczewski_2016 ; Pierse_2017 . In evolutionary biology and ecology, one could ask if a population goes extinct before its size attains some threshold level Lande_2003 ; Drake_2009 . Clearly, the immense set of possibilities is not limited to these few examples.

What happens when a first-passage process with several possible outcomes becomes subject to restart? The main goal of this Letter is to draw attention to previously unknown type of optimal behaviour in first-passage phenomena: a carefully chosen rate of Poisson restart brings the probability of observing a particular completion scenario to a maximum (or minimum). In other words, we argue that stochastic restart could optimize the so-called splitting probabilities Kampen_1992 ; Redner_2001 . The effect is first illustrated on a particular example and after that we apply a general framework recently proposed in Ref. Reuveni_PRL_2017 by Pal and Reuveni to gain a more deep insight. For the sake of simplicity we focus on the case where the process has exactly two possible outcomes, but the analysis can directly be extended to a more general situation. We show that optimality of the splitting probabilities always entails an exact match between the unconditional and conditional mean completion times of the process. Looking for further generalization, we go beyond the assumption of Poisson restart and demonstrate advantage of the deterministic restart strategy in terms of attaining the most pronounced extrema of splitting probabilities.

The key properties of first passage under restart have been originally learned from the one dimensional diffusion process EM_2011 . We will use the same ”Drosophila” to demonstrate the ability of stochastic restart to optimize the splitting probabilities. Specifically, let us consider a mortal Brownian searcher with the diffusion constant $D$ and the mortality rate $\alpha$ which starts from the initial position $x_{0}\geq 0$. The search process ends when either searcher dies or when it finds a target located at $x=0$. It is shown in Ref. Abad_2013_1 (see also Meerson_2015 ) that target detection occurs with the probability $p=e^{-\sqrt{\alpha x_{0}^{2}/D}}$. Assume now that the process is stochastically restarted, i.e. the searcher is returned to its initial position $x_{0}$ at some constant rate $r$ note_1 . What is the detection probability $p_{r}$ in the presence of restart? Exact solution of the initial-boundary value problem for the probability density of the searcher’ position yields (see Supplemental Material)

[TABLE]

Analysing Eq. (1), one can readily see that if $\alpha\geq\alpha_{0}=(z^{*})^{2}D/x_{0}^{2}$, where $z^{*}\approx 1.59362...$ is the solution to $z/2=1-e^{-z}$, then $p_{r}$ monotonically decreases as $r$ increases from zero to infinity. Otherwise, when $\alpha<\alpha_{0}$, the probability $p_{r}$ takes its maximum at the non-vanishing restart rate $r_{0}=\alpha_{0}-\alpha$. In Figure 1 we plot $p_{r}$ as a function of $r/\alpha_{0}$ for different $\alpha/\alpha_{0}$.

Let us give a qualitative explanation for the observed behaviour of $p_{r}$. If $\alpha$ is large compared to $D/x_{0}^{2}$, then the typical size of the region explored by the searcher during its lifespan is less than the initial distance to the target and, thus, a non-vanishing restart inevitably leads to reduction of the search efficiency. Otherwise, when $\alpha$ is small in comparison with $D/x_{0}^{2}$, the searcher leave long enough to be able to reach target via typical diffusive path, but it is also able to execute distant excursion in empty areas of the search space. These excursions prolong the search process and typically end with searcher death. Then, the non-vanishing restart rate censors the fatal paths and increases chances to find the target. On the other hand, too large restart rate hinders target detection since the searcher has less time between restarts to reach the origin under the same mortality rate. This is why there exists a non-vanishing optimal restart rate $r^{*}$ which brings the probability that searcher will find the target before dying to a maximum.

Having examined the exemplary case, we now turn to more general setting. Let us consider a generic stochastic process which can end in two incompatible ways and is subject to a generic restart mechanism. For the sake of convenience we will call one of two possible outcomes as success and the other one as failure. Thus, the problem can be viewed as a kind of Bernoulli experiment, see Fig. 2a. In the example discussed above, detection of target naturally corresponds to success, while searcher’ death is interpreted as failure. Obviously, in other contexts these conventional terms may not have any real meaning.

The original process is characterized by a random completion time $T$ having the probability distribution $P(T)$. The later can be decomposed into a sum $P(T)=P^{s}(T)+P^{f}(T)$, where $P^{s}(T)$ and $P^{f}(T)$ are the probability densities of successful and failed trials, respectively. Normalization of the probability density $P^{s}(T)$ defines the ”unperturbed” probability $p$ of success: $p=\int_{0}^{\infty}P^{s}(T)dT$. Conservation of total probability implies that $\int_{0}^{\infty}P^{f}(T)dT=1-p$. We will also utilize the trivial fact that the ratio $P^{s}(T)/P(T)$ gives the probability of success in a trial with the completion time $T$.

Being subject to restart, the process can be interrupted at a random time $R$, characterized by a proper probability distribution $P^{r}(R)$, and started again. The probability $p_{r}$ of success for the restarted process can be computed as expectation of a binary random variable $x$ which takes the value $1$ if the process is completed in success and is equal to [math] in the case of failure. This variable obeys the following renewal equation:

[TABLE]

where $I(T<R)=1-I(T\geq R)$ is an indicator random variable which is equal to unity when $T<R$ and is zero otherwise; $x^{\prime}$ is an independent and identically distributed copy of $x$; $y_{T}$ is an auxiliary binary variable which takes the value one with probability $P^{s}(T)/P(T)$.

The intuition behind Eq. (2) is very simple. Imagine that we run a computer simulation designed to reproduce behaviour of the random variable $x$. At the first step, we should choose two random times from the distributions $P(T)$ and $P^{r}(R)$ and decide which of the two, restart or completion, happened first. If $T<R$, then the process is completed prior to restart. To determine whether the process end in success or in failure we toss a coin with probability of success $P^{s}(T)/P(T)$ and assign the outcome to the variable $x$. Otherwise, if $T<R$, the process begins completely anew and we should repeat the procedure until the process reaches completion. This scheme is best illustrated in the form of pseudocode, see Fig. 2b.

After averaging over the statistics of the underlying process and random restart events, Eq. (2) yields

[TABLE]

Once the probability density functions $P^{s}(T)$ and $P^{r}(R)$ are known, one can readily compute $\langle I(T<R)y_{T}\rangle=\int_{0}^{\infty}\int_{0}^{R}P^{r}(R)P^{s}(T)dRdT$ and $\langle I(T<R)\rangle=\int_{0}^{\infty}\int_{0}^{R}P^{r}(R)P(T)dRdT$. When restart events posses Poisson statistics with constant rate parameter $r$, the restart time $R$ has exponential distribution $P^{r}(R)=re^{-rR}$ and Eq. (3) reduces to

[TABLE]

where $\tilde{P}^{s}(r)$ and $\tilde{P}(r)$ denote the Laplace transforms of, respectively, $P^{s}(T)$ and $P(T)$ evaluated at $r$. Note that for the above problem of diffusion mediated search $P^{s}(T)=\sqrt{x_{0}^{2}/4\pi DT^{3}}e^{-\alpha T-x_{0}^{2}/4DT}$ and $P^{f}(T)=\alpha e^{-\alpha T}\mbox{erf}(\sqrt{x_{0}^{2}/4DT})$ Roldan_PRE_2017 . It is straight forward to show then that Eq. (4) reproduces Eq. (1) previously obtained through the less generic method (see Supplementary Material).

We are mostly interested in the class of problems where probability $p_{r}$ is maximised at a nonvanishing optimal rate $r^{*}$ of Poisson restart. In principle, one can construct an infinite number of examples belonging to this class. What do they all have in common? To address this question let us take a look at the first-passage-time properties of the process illustrated in Fig. 2a. As it is shown in Reuveni_PRL_2017 , the completion time $T_{r}$ of a generic first-passage process under a generic restart mechanism obeys the following identity

[TABLE]

in which $T_{r}^{\prime}$ is an independent and identically distributed copy of $T_{r}$. Equation (5) allows one to express the MFPT as $\langle T_{r}\rangle=\langle\min(T,R)\rangle/\langle I(T<R)\rangle$, where $\min(T,R)$ is the minimum of $T$ and $R$. Next, one could ask also how to compute the MFPT $\langle T_{r}^{s}\rangle$ conditional to success, which is simply the average completion time of successful trials. By virtue of its definition, this quantity can be written as $\langle T_{r}^{s}\rangle={\langle xT_{r}\rangle}/{\langle x\rangle}$. Substituting Eqs. 2 and (5) into this relation results in

[TABLE]

For exponentially distributed restart, Eq. (6) takes a particularly simple form (see Supplementary Material)

[TABLE]

where $p_{r}$ is given by Eq. (4) and $\langle T_{r}\rangle=r^{-1}(1-\tilde{P}(r))/\tilde{P}(r)$ Reuveni_PRL_2016 . If the success probability $p_{r}$ of the restarted process attains a maximum at some $r^{*}$, the second term in the right hand side of Eq. (7) vanishes and we get

[TABLE]

Also, since $p_{r}\langle T_{r}^{s}\rangle+(1-p_{r})\langle T_{r}^{f}\rangle=\langle T_{r}\rangle$, similar identity holds true for the mean completion time of failed trials: $\langle T_{r^{*}}^{f}\rangle=\langle T_{r^{*}}\rangle$. We thus conclude that when the rate of Poisson restart is optimal, in the sense that it maximizes or minimizes the probability to observe specific outcome, the unconditional MFPT is equal to the MFPT conditional to this outcome. This universal feature is shared by all optimally restarted processes irrespective on their fine details (see the left panel of Fig. 3 for illustration).

Surprising simplicity of Eq. (8) calls for its intuitive explanation. To provide such an explanation let us assume that one starts to observe a first-passage process, which is allowed to repeat itself over and over, at a random moment of time. What is then the expected probability $p^{\text{exp}}$ of getting success in the next outcome? It can be shown that this probability is given by $p^{\text{exp}}=p\langle T^{s}\rangle/\langle T\rangle$ (see Supplementary Material). Obviously, applying Poisson restart with infinitesimally small rate $\delta r$ will increase the chances of success whenever $p^{\text{exp}}<p$, while at $p^{\text{exp}}>p$ the effect will be opposite. At the same time, if the process is already restarted at the optimal rate $r^{*}$, then $[dp_{r}/dr]_{r^{*}}=0$ and small additional correction $\delta r$ to $r^{*}$ does not change the probability of success $p_{r^{*}}$ in the leading order approximation. Therefore, for the optimally restarted process, $p_{r^{*}}$ must be equal to $p_{r^{*}}^{\text{exp}}=p_{r^{*}}\langle T_{r^{*}}^{s}\rangle/\langle T_{r^{*}}\rangle$ that immediately leads to Eq. (8). Interestingly, the match of unconditional and conditional MFPTs is an inherent property of some two-thresholds first-passage processes relevant to kinetics of enzyme reactions Qian_2006 ; Ge_2008 , motor proteins dynamics Kolomeisky_2005 , entropy-production fluctuations Neri_2016 and decision making Roldan_2015 . From the foregoing considerations it follows that for all these processes the splitting probabilities coincide with the corresponding expected splitting probabilities.

Anticipating that optimization is not an exclusive prerogative of Poisson restart, it is natural to ask how to choose a restart time distribution $P^{r}(R)$ which provides the maximum probability of success $p_{r}$ for a given first-passage process. Recently it was proven that deterministic restart (i.e. $P^{r}(R)=\delta(R-t)$) always outperforms stochastic restart strategies in terms of attaining the lowest MFPT Reuveni_PRL_2017 . Arguments similar to those used in Reuveni_PRL_2017 allow us to conclude that deterministic restart is also universally preferable when one needs to optimize the splitting probabilities. It can be shown that if there exists such $t^{*}$ that deterministic restart with restart time distribution $P^{r}(R)=\delta(R-t^{*})$ brings the probability of success to a maximum $p_{t^{*}}$, then the value $p_{t^{*}}$ cannot be exceeded by stochastic restart strategies (see Supplementary Material).

Equation (8) is not longer valid when restart events have non-Poisson statistics. Instead, the conditional and unconditional mean first-passage times of a process undergoing optimally tuned deterministic restart obey the universal inequality constraint

[TABLE]

To prove Eq. (9), let us assume that the process, which is being restarted deterministically in an optimal way, becomes subject to additional Poisson restart with an infinitesimally small rate $\delta r$. That produces a deferential correction $\delta p$ to the probability of success $p_{t^{*}}$ attained by deterministic restart. Equation (7) allows us to write $\langle T_{t^{*}}\rangle-\langle T_{t^{*}}^{s}\rangle=\delta p/(p_{t^{*}}\delta r)$. Due to dominance of deterministic restart over other restart strategies, one can be sure that $\delta p\leq 0$ and, therefore, $\langle T_{t^{*}}^{s}\rangle\geq\langle T_{t^{*}}\rangle$. Taking into account the identity $p_{r}\langle T_{r}^{s}\rangle+(1-p_{r})\langle T_{r}^{f}\rangle=\langle T_{r}\rangle$, one arrives at the opposite inequality for the MFPT of failed trials: $\langle T_{t^{*}}^{f}\rangle\leq\langle T_{t^{*}}\rangle$. These features of deterministic restart are clearly seen in the right panel of Fig 3.

Conclusion.— First-passage processes exhibiting stochasticity not only in the timing of their evolution but also in the very result of the evolution are ubiquitous in science. When the possible outcomes of a process are not equally valuable, the splitting probabilities may come to the fore as a crucial measure of efficiency and reliability Campos_2015 ; Roldan_2015 ; Meerson_2015 ; Rehbein_2011 ; Rehbein_2015 ; Martin-Soomer_2016 ; Grebenkov_2016_1 . In this Letter we applied a general theoretical approach to describe the effect of restart on the splitting probabilities of a process with exactly two possible completion scenarios. It is shown that a carefully chosen rate of Poisson restart could maximize (minimize) the probability that the process will complete in the desirable (undesirable) way. Whenever it is the case, the conditional and unconditional mean completion times are equal to each other. We also established the global dominance of deterministic restart in the entire space of restart strategies - further evidence of the great optimization potential of deterministic restart in first-passage problems Pal_2016 ; Reuveni_PRL_2017 . Note that these conclusions are robust to appearance of a generally distributed random time penalty for restart (see Supplementary Material). Thus, our work adds to the collection of universal results in the field of first-passage phenomena Condamin_2007_2 ; Benichou_2010 ; Chupeau_2015 ; Grebenkov_2016 ; Reuveni_PRL_2016 ; Reuveni_PRL_2017 .

Of many implications of above results, let us emphasize the issue relevant to chemical kinetics. The two fundamental problems of chemistry are control over reaction rate Hanggi_1990 and product selectivity (Rehbein_2011, ; Rehbein_2015, ; Martin-Soomer_2016, ). As we know thanks to the recent study of enzymatic reactions Reuveni_2014 , restart of catalytic step can potentially accelerate the rate of product formation. The results of the present work lead to the complementary conclusion that when competing pathways of a chemical reaction end up with different products the introduction of restart mechanism may allow to control over product selectivity. This is also relevant to the protein folding reactions in which a single protein molecule can fold in one of distinct native conformations Solomatin_2010 ; Marek_2011 ; Hyeon_2012 ; Paudel_2014 ; Hinczewski_2016 ; Pierse_2017 . One could potentially optimize the probabilities of getting different conformational states by initiating the protein refolding that follows the denaturation events Solomatin_2010 ; Liu_2013 ; Paudel_2014_1 with carefully adjusted frequency.

This paper covers only some of many interesting questions relating to the effect of restart on the first-passage processes with plural outcomes. Particularly, we have only considered the mean conditional first-passage times, thus, leaving aside the issue of fluctuations. Besides, it would be also interesting to consider the problem of optimization over both the splitting probabilities and the mean first-passage time because in some settings the time required for process completion is not less important than the outcome of the process. Finally, note that while looking for the optimal restart strategy we did not take into account possible cost associated with its implementation Husain_2016 . The tradeoffs between performance and cost in optimization problems involving first-passage processes under restart represent an important challenge for future studies.

Acknowledgments. The author acknowledges stimulating discussions with S. Redner and M. Tolpina.

I Supplementary information

I.1 Mortal Brownian searcher in one dimension

Assume that a mortal searcher undergoes diffusion starting from the initial position $x_{0}\geq 0$. The search process ends either with detection of a target at $x=0$ or with searcher’ death. The probability that the searcher will die in the time interval $[t,t+dt]$ is given by $\alpha dt$ where $\alpha$ is the time-independent mortality rate. The probability distribution of the completion time is known to be (see Meerson_2015 )

[TABLE]

where $D$ is the diffusion constant. The first term in the right hand side of Eq. (10) comes from those realizations of the search process which end with target detection (success), while the second terms is due to mortality (failure).

Now assume that the searcher is subject to stochastic reset to the initial position $x_{0}$ at the rate $r$. The evolution of the probability distribution $\rho(x,t)$ of the searcher is governed by the following equation

[TABLE]

supplemented by the initial condition $\rho(x,0)=\delta(x-x_{0})$ and the boundary condition $\rho(0,t)=0$. After the Laplace transform

[TABLE]

we obtain

[TABLE]

Imposing the zero boundary condition at $x=0$ and $x\to+\infty$ together with the continuity condition at $x=x_{0}$ we find

[TABLE]

where $\gamma_{s}=\sqrt{(s+r+\alpha)/D}$. To calculate the unknown coefficient $A$ we should take into account the jump of the derivative $\partial_{x}\rho_{s}$ at $x=x_{0}$. That gives

[TABLE]

Then the Laplace transforms of the flux to the target $j(t)=D\partial_{x}\rho(x,t)$ is given by

[TABLE]

The probability that the target is eventually found is

[TABLE]

which coincides with Eq. (1) in the main text. Alternatively, one can derive this result from Eq. (4). Calculating the Laplace transforms of the probability densities of successful and failed trials, which are $P^{s}(T)=\sqrt{x_{0}^{2}/4\pi DT^{3}}e^{-\alpha T-x_{0}^{2}/4DT}$ and $P^{f}(T)=\alpha e^{-\alpha T}\mbox{erf}(\sqrt{x_{0}^{2}/4DT})$ in accordance with Eq. (10), we find

[TABLE]

[TABLE]

The ratio $\tilde{P}^{s}(r)/\tilde{P}(r)$, where $\tilde{P}(r)=\tilde{P}^{s}(r)+\tilde{P}^{f}(r)$, is equal to the right hand side of Eq. (1).

I.2 Derivation of Eq. (7)

Let us derive Eq. (7) for a more general situation than the one described in the main text. Namely, we assume that each restart event entails a generically distributed random time penalty $T_{\text{on}}$. This complication does not restrict the applicability of our arguments concerning the splitting probabilities so that Eqs. (2), (3) (4) from the main text remain unchanged. However, the time penalty will definitely affect the completion time $T_{r}$ and Eq. (5) now becomes

[TABLE]

We take expectation of Eq. (20) to find

[TABLE]

As it follows from Reuveni_PRL_2016 , when the restart time $R$ has exponential distribution $P^{r}(R)=re^{-rR}$, Eq. (21) reduces to

[TABLE]

Next inserting Eqs. (2) and (20) into the definition of the conditional first-passage time $\langle T_{r}^{s}\rangle=\langle xT_{r}\rangle/\langle x\rangle$ we obtain

[TABLE]

For Poisson restart, the averages in the right hand side of Eq. (23) can be transformed in the following way

[TABLE]

[TABLE]

[TABLE]

[TABLE]

where we have used repeated integration by parts. Substituting these expressions back into Eq. (23) yields

[TABLE]

Recalling Eqs. (4) and (22) we see that Eq. (31) can be rewritten in the form $\langle T_{r}^{s}\rangle=\langle T_{r}\rangle-d\ln p_{r}/dr$ which coincides with Eq. (7) from the main text .

I.3 Expected probability of success $p^{\text{exp}}$

Consider a first-passage process which starts at time zero and repeats itself over and over indefinitely. Each of the independent trials can end either with success or failure. Let $\{t_{i}\}_{i=1}^{\infty}=t_{1},t_{2},t_{3},\dots$ be the ordered sequence of time instants when completion events occur ($\forall$ $i$: $t_{i}>0$ ). Then, the completion time of the $i$th trial is given by $T_{i}=t_{i}-t_{i-1}$. Also, let $\{x_{i}\}_{i=1}^{\infty}=x_{1},x_{2},x_{3},\dots$ be a string of binary variables encoding the results of trials. Namely, $x_{i}=1$ if the $i$th trial is successful, and $x_{i}=0$ in the case of failure.

The question that we want to address is the following. If one starts to observe the process at a random point in time $t$, what is the expected probability $p^{\text{exp}}$ of successful completion of the ongoing trial? Given sequences $\{t_{i}\}_{i=1}^{\infty}$ and $\{x_{i}\}_{i=1}^{\infty}$, we can construct a piece-wise process $X(t)=x_{N(t)+1}$, where $N(t)$ is the number on completion events up to time $t$. It is easy to see that $X(t)$ equals to unity if the next outcome is success and zero otherwise. The expected probability of success $p^{\text{exp}}$ is simply the time average of $X(t)$:

[TABLE]

The law of large numbers tells us that

[TABLE]

[TABLE]

Therefore $p^{\text{exp}}=\langle xT\rangle/\langle T\rangle$. Finally, we can rewrite this as $p^{\text{exp}}=p\langle T^{s}\rangle/\langle T\rangle$, where $p=\langle x\rangle$ is the probability of success in a single trial and $\langle T^{s}\rangle=\langle xT\rangle/\langle x\rangle$ represents the mean completion time of successful trials.

I.4 Deterministic restart

Deterministic restart strategy implies that the process is restarted whenever a time $t$ passes. In this case $P^{r}(R)=\delta(R-t)$. Assume that $t^{*}$ is the optimal period of deterministic restart maximizing the probability of success which is given by Eq. (3) in the main text, i.e.

[TABLE]

for all $0<t<\infty$. Let us multiply both sides of Eq. (35) by $\frac{\langle I(T<R)|R\rangle_{T}}{\langle I(T<R)\rangle_{T,R}}$

[TABLE]

Next we replace $t$ by a generally distributed random time $R$ and average over statistics of $R$

[TABLE]

Applying the law of total expectation we find

[TABLE]

and

[TABLE]

Utilizing Eqs. (38) and (39) we obtains from Eq. (37) the following inequality

[TABLE]

On the left we see the success probability $p_{t^{*}}$ attained by optimal deterministic restart, while the right hand side represents the success probability $p_{r}$ for a process restarted at a generally distributed random time $R$. This proves that deterministic restart is optimal among all possible stochastic restart strategies.