Robust random search with scale-free stochastic resetting

TL;DR

This paper introduces a stochastic resetting search model with a rate inversely proportional to time, leading to a paradoxical diffusion process that enhances search efficiency without needing parameter optimization.

Contribution

It proposes a novel resetting protocol based on scale-free, time-dependent rates, demonstrating its effectiveness across different diffusion regimes.

Findings

Paradoxical diffusion with self-similarity and linear MSD growth.

Resetting protocol improves search efficiency without parameter tuning.

Applicable to subdiffusive and superdiffusive regimes.

Abstract

A new model of search based on stochastic resetting is introduced, wherein rate of resets depends explicitly on time elapsed since the beginning of the process. It is shown that rate inversely proportional to time leads to paradoxical diffusion which mixes self-similarity and linear growth of the mean square displacement with non-locality and non-Gaussian propagator. It is argued that such resetting protocol offers a general and efficient search-boosting method that does not need to be optimized with respect to the scale of the underlying search problem (e.g., distance to the goal) and is not very sensitive to other search parameters. Both subdiffusive and superdiffusive regimes of the mean squared displacement scaling are demonstrated with more general rate functions.

Equations

$${\partial }_t\rho (x,t|x_i,t_i)=D{\partial }_{xx}^2\rho (x,t|x_i,t_i)-r(t)\rho (x,t|x_i,t_i)+r(t)\delta (x-x_0)$$

$$r(t)=\frac{\alpha }{t}.$$

$$N(t_0,t_1)=\alpha \ln \frac{t_1}{t_0}.$$

$$⟨(x(t)-x_0{)}^2⟩=\underset{-\infty }{\overset{\infty }{\int }}dx{x}^2\rho (x,t|x_0,0)=\frac{2Dt}{1+\alpha }.$$

$$\rho (k,t|x_0,0)=\alpha e^{ik{x}_0}{t}^{-\alpha }{e}^{-D{k}^{2}t}\underset{0}{\overset{t}{\int }}d\tau {\tau }^{\alpha -1}{e}^{D{k}^2\tau }.$$

$$\rho (k,t|x_0,0)=e^{ik{x}_0}\frac{1-e^{-D{k}^{2}t}}{D{k}^{2}t}.$$

$$⟨{\left(x(t)-x_0\right)}^2⟩=2D\exp \left(-\frac{\alpha }{1-\mu }{t}^{1-\mu }\right)\underset{0}{\overset{t}{\int }}d\tau \exp \left(\frac{\alpha }{1-\mu }{\tau }^{1-\mu }\right),$$

$${\mathrm{S}}_{\alpha }(t)=S_0(t)\Psi (0,t)+\underset{0}{\overset{t}{\int }}d\tau r(\tau ){\mathrm{S}}_{\alpha }(\tau )S_0(t-\tau )\Psi (\tau ,t),$$

$$\Psi \left(t_0,t_1\right)=\exp \left(-\underset{t_0}{\overset{t_1}{\int }}d\tau r(\tau )\right)$$

$${\Psi }_{\alpha }\left(t_0,t_1\right)={\left(\frac{t_0}{t_1}\right)}^{\alpha }.$$

$${\mathrm{S}}_{\alpha }(t)=\frac{\alpha }{t^{\alpha }}\underset{0}{\overset{t}{\int }}d\tau {\tau }^{\alpha -1}{\mathrm{S}}_{\alpha }(\tau )S_0(t-\tau ).$$

$${\mathrm{g}}_{\alpha }(t)\equiv t^{\alpha -1}{\mathrm{S}}_{\alpha }(t),$$

$${\mathrm{g}}_{\alpha }(t)=\frac{\alpha }{t}\underset{0}{\overset{t}{\int }}d\tau {\mathrm{g}}_{\alpha }(\tau )S_0(t-\tau ).$$

$${\partial }_s{\tilde{\mathrm{g}}}_{\alpha }(s)=-\alpha {\tilde{\mathrm{S}}}_0(s){\tilde{\mathrm{g}}}_{\alpha }(s).$$

$${\tilde{\mathrm{S}}}_0(s)=\frac{1-{\tilde{\rho }}_0(s)}{s},$$

$${\tilde{\mathrm{g}}}_{\alpha }(s)=C_{\alpha }{s}^{-\alpha }\exp \left(-\alpha \underset{s}{\overset{\infty }{\int }}du\frac{{\tilde{\rho }}_0(u)}{u}\right),$$

$$\underset{s}{\overset{\infty }{\int }}du(u-s{)}^{\beta }{e}^{-st}=\Gamma (\beta +1)t^{-1-\beta }{e}^{-st},$$

$$L\{t^{-\beta -1}f(t)\}=\frac{1}{\Gamma (\beta +1)}\underset{s}{\overset{\infty }{\int }}du(u-s{)}^{\beta }\tilde{f}(s),$$

$${\tilde{\mathrm{S}}}_{\alpha }(s)=-\frac{1}{\Gamma (\alpha )}\underset{s}{\overset{\infty }{\int }}du(u-s{)}^{\alpha -1}{\partial }_u{\tilde{\mathrm{g}}}_{\alpha }(u).$$

$$\underset{t\to 0}{\lim }{\mathrm{S}}_{\alpha }(t)=\underset{s\to \infty }{\lim }s{\tilde{\mathrm{S}}}_{\alpha }(s)=1$$

$${\tilde{\mathrm{S}}}_{\alpha }(s)=\alpha \underset{s}{\overset{\infty }{\int }}du{\left(\frac{u-s}{u}\right)}^{\alpha -1}\frac{1-{\tilde{\rho }}_0(u)}{u^2}\exp \left(-\alpha \underset{u}{\overset{\infty }{\int }}\mathrm{d}v\frac{{\tilde{\rho }}_0(v)}{v}\right).$$

$${\overline{\mathrm{T}}}_{\alpha }=\alpha \underset{0}{\overset{\infty }{\int }}\mathrm{d}u\frac{1-{\tilde{\rho }}_0(u)}{u^2}\exp \left(-\alpha \underset{u}{\overset{\infty }{\int }}\mathrm{d}v\frac{{\tilde{\rho }}_0(v)}{v}\right).$$

$${\tilde{\rho }}_0(s)=e^{-\sqrt{s{\tau }_{\mathrm{diff}}}},$$

$${\tilde{\mathrm{g}}}_{\alpha }(s)=\Gamma (\alpha +1)s^{-\alpha }\exp \left(-2\alpha E_1\left(\sqrt{s{\tau }_{\mathrm{diff}}}\right)\right),$$

$$E_1(x)=\underset{x}{\overset{\infty }{\int }}\mathrm{d}t\frac{e^{-t}}{t}$$

$$E_1(x)=-\gamma -\ln x-\sum _{k=1}^{\infty }\frac{(-x{)}^k}{k\cdot k!}$$

$${\tilde{\mathrm{g}}}_{\alpha }(s)=\Gamma (\alpha +1){\tau }_{\mathrm{diff}}^{\alpha }\exp \left(2\alpha \gamma +2\alpha \sum _{k=1}^{\infty }\frac{(-\sqrt{s{\tau }_{\mathrm{diff}}}{)}^k}{k\cdot k!}\right),$$

$$⟨{{\mathrm{T}}_{\alpha }}^{\alpha }⟩=\alpha \underset{s\to 0}{\lim }{\tilde{\mathrm{g}}}_{\alpha }(s)=\Gamma (\alpha +1)e^{2\alpha \gamma }{\tau }_{\mathrm{diff}}^{\alpha }.$$

$${\overline{\mathrm{T}}}_1(x_0)=e^{2\gamma }{\tau }_{\mathrm{diff}}\approx 3.17x_0^2/D.$$

$${\mathrm{S}}_{\alpha }(t)\sim t^{-1/2-\alpha }.$$

Full text

I Introduction

Any search is intrinsically governed by randomness: the need to perform it underscores the ignorance of the searching agent and its inability to predict the outcome. In particular, the agent may not know in advance how much time, on average, it will take to find a target and how much variability in search time is expected due the randomness. How can the efficiency of a search strategy be boosted without acquiring such information beforehand? It may happen that the longer the search lasts, the more unfavorable random instance the agent is experiencing, and thus the longer the expected remaining time to find the target. It is then true that restarting the process anew may be beneficial, as it lowers the expected completion time. It has been shown that this is indeed the case if the coefficient of variation of completion times is larger than $1$ Reuveni2016 . This conditions holds if the completion times of a search process are drawn from the hyperexponential or power-law distributions. Such situation arises, for instance, in the one-dimensional, unbiased Brownian motion with a single target. The possibility of arbitrarily long excursions far from the target manifests itself in large fluctuations and the associated power-law distribution of the first passage times.

In the engineering community such mechanism was studied in the context of computer software systems huang1995software ; dohi2001estimating ; van2006analysis ; okamura2016analysis and non-convex optimization methods fukunaga1998restart ; jansen2002analysis ; luersen2004globalized ; loshchilov2016sgdr . More recently, Evans and Majumdar introduced a model with time-homogeneous stochastic resetting (i.e., with exponentially distributed waiting times between the resets) in tandem with diffusion majumdar2011resetting . They showed that one-dimensional majumdar2011resetting and multidimensional evans2014diffusion diffusion with resets exhibit finite mean first passage times (MFPTs) which can be optimized with respect to resetting rate $r$. Moreover, resets lead to non-equilibrium steady-states with a combination of local and non-local currents majumdar2011resetting ; mendez2016characterization ; eule2016non . The simplicity and nontrivial behavior of these systems has sparked the interest of the physics community, with a considerable amount of research focusing on modifying the model to include nontrivial boundary conditions christou2015diffusion ; chatterjee2018diffusion ; pal2018first , external potentials pal2015diffusion , anomalous transport kusmierz2014first ; kusmierz2015optimal ; mendez2016characterization ; shkilev2017continuous ; Montero2017 ; bodrova2018scaled ; bodrova2018non ; kusmierz2019subdiffusive , drift Montero2017 ; ray2018p , delays following the resets Reuveni2016 ; evans2018effects ; maso2019stochastic ; pal2019home , and non-exponential distributions of waiting times between resets nagar2016diffusion ; eule2016non ; pal2016diffusion . Resets were also studied in the context of stochastic energetics fuchs2016stochastic , enzymatic reactions rotbart2015michaelis , fluctuating interfaces gupta2014fluctuating , and power-law distributions in non-equilibrium systems manrubia1999stochastic .

Non-exponential waiting times between the resets can be generated by time-dependent resetting rates nagar2016diffusion ; pal2016diffusion . In models introduced in these previous works, rate depends on the time elapsed since the last reset. Here we introduce a new model with resetting rate $r(t)$ that depends on the time elapsed since the start of the process. Such non-stationary resetting protocol, while preserving the Markovian character of the dynamics, introduces time-inhomogeneity. As we will show, this dramatically affects the behavior of the process. If $r(t)$ decays with time, the process does not converge to any steady state, with the mean square displacement (MSD) growing indefinitely. Moreover, $r(t)$ that is inversely proportional to time provides an efficient and robust mechanism to boost the time efficiency of a search. It has previously been shown that deterministic resetting is the optimal way of minimizing search time via resets pal2016first . However, similarly to stochastic resetting with a constant rate, such resetting has to be optimized assuming full knowledge of details of a given search problem. In contrast, the resetting mechanism we propose does not have to be adapted to the time scale of the search problem.

II Diffusion with time-dependent resetting

We start from one-dimensional diffusion as the underlying search process, which will help build the intuition. Later we present a more general framework. Since the process of diffusion with memoryless resetting is Markovian, it is fully characterized by the initial position distribution and the propagator $\rho(x,t|x_{i},t_{i})$, which denotes the probability density function of $x$ at time $t$ of a particle that at time $t_{i}<t$ was at $x_{i}$. The particle diffuses till a resetting event that brings it instantaneously back to the resetting position $x_{0}$ (equal to the position of the particle at time $t=0$) 111 We are assuming that the initial position corresponds to the resetting position. However, due to the singular behavior of the function $r(t)$ at $t=0$, the choice of the initial position is inconsequential: If it does not match the resetting position, the probability mass is transferred to the resetting position in an infinitesimal time. . The particle then continues to diffuse until the next reset. The corresponding propagator solves the following partial differential equation

[TABLE]

where $\partial_{z}$ denotes the partial derivative with respect to $z$ and $D$ is the diffusion coefficient. Such process has been analyzed extensively in the case of a constant rate $r(t)=r$, with the corresponding resets described by the homogeneous Poisson point process. It has been shown majumdar2011resetting that this process, in the absence of targets, attains a non-equilibrium steady state. If a single target, represented by an absorbing boundary, is placed at the origin, there exists an optimal rate $r^{*}\propto x_{0}^{-2}$ at which the MFPT is minimized.

II.1 Scale-free resetting

In this work we mainly focus our attention on the special case of

[TABLE]

The only parameter $\alpha$ is dimensionless—this form of resetting does not introduce any time scale. For this reason, we refer to (2) as a scale-free resetting. Resets described by (2) are generated by the inhomogeneous Poisson point process with an average intensity (expected number of events) within a time period $(t_{0},t_{1}]$ given by

[TABLE]

Note that our choice of $r(t)$ features a singularity at $t=0$, which translates into diverging intensity with $t_{0}\to 0$. Any practical applications of scale-free resetting have to introduce a short-term cut-off to avoid resetting with infinite frequency. However, all our theoretical predictions without an explicit account of the cut-off are still valid at times much longer than the cut-off.

As evident from (1), the process (1) with (2) preserves the self-similarity of the pure Brownian motion, i.e., the re-scaling $x\to cx$ and $t\to c^{2}t$ does not change any observables (see Fig. 1).

We can therefore expect that the mean square displacement (MSD) scales linearly with time. Indeed, straightforward calculations (multiply (1) by $(x-x_{0})^{2}$, integrate over $x$ and solve the resulting ordinary differential equation) lead to

[TABLE]

Although the self-similarity and associated linear time-dependence of the MSD bear a strong resemblance to free diffusion, resets vastly modify other statistics. For instance, the Fourier transform of the propagator reads

[TABLE]

This propagator is clearly non-Gaussian and has a cusp at ${x=x_{0}}$. It takes a particularly simple form for $\alpha=1$

[TABLE]

Similar non-Gaussian displacement distributions with a cusp, accompanied by a linear time dependence of the MSD have been observed in several systems of diffusing colloidal particles wang2009anomalous ; wang2012brownian . Such systems have recently attracted considerable attention among theoreticians, who since then have developed multiple models with fluctuating diffusivity hapca2008anomalous ; wang2009anomalous ; chubynsky2014diffusing ; chechkin2017brownian ; sposini2018random . Scale-free resetting provides an alternative mechanism behind such behavior.

II.2 Generalizations of the resetting rate function

In addition to the presented applications to search problems, a generalization of the proposed resetting mechanism can be applied to model anomalous transport phenomena. The MSD of (1) with $r(t)=\alpha/t^{\mu}$ is given by the formula

[TABLE]

which for $\mu<1$ exhibits a smooth transition between diffusive behavior with the MSD ${\approx~{}2Dt}$ for ${t\ll\tau_{\alpha}}$ and subdiffusive behavior with the MSD ${\approx 2Dt^{\mu}/\alpha}$ for $t\gg\tau_{\alpha}$, where ${\tau_{\alpha}=\alpha^{-1/(1-\mu)}}$ is the time scale introduced by the power-law resetting. In the opposite case of $\mu>1$ the long-term behavior is diffusive, whereas at short times an apparent superdiffusivity with the MSD $\approx 2D\tau^{\mu}/\alpha$ is observed. By assuming $r(t)\propto(\log t)^{-1}$ one can also model an ultra-slow diffusion with the MSD $\propto\log t$, a behavior previously uncovered in the strongly non-Markovian random walks with preferential relocations to places visited in the past boyer2014random .

One could involve heavy-tails jump distributions leading to a competition between superdiffusivity of the Lévy flights and subdiffusive tendency from resets, similar to the competition observed in the continuous-time random walk scheme montroll1965random ; metzler2000random ; magdziarz2007competition . Many different variants of a combination of anomalous transport with constant-rate stochastic resetting were previously studied kusmierz2014first ; kusmierz2015optimal ; mendez2016characterization ; shkilev2017continuous ; Montero2017 ; kusmierz2019subdiffusive and were shown to exhibit non-trivial features, including first and second order transitions of the optimal search kusmierz2014first ; kusmierz2015optimal ; campos2015phase ; Montero2017 and a non-monotonic behavior of the MSD kusmierz2019subdiffusive . For this reason, we expect that a combination of scale-free resets with independent constant-rate resets may lead to thought-provoking phenomena.

III Completion times

In the previous section we have shown how time-dependent resetting affects the time evolution and shape of the displacement distribution. Another striking consequence of scale-free resetting can be observed in the statistics of the first passage times. As we show in the following, in contrast to free diffusion on infinite line, diffusion with scale-free resetting can find a target in a finite MFPT. Moreover, due to the self-similarity, we can expect that the MFPT is proportional to $x_{0}^{2}$, where $x_{0}$ denotes the initial position with respect to the target.

This observation prompted us to explore more general scenarios of search under scale-free resetting beyond diffusion on intinite line. Since the following calculations apply to general search times that may not be generated as the first passage times of some stochastic process, we hereafter use the term completion times. We show that scale-free resetting is robust, i.e. the optimal form of the scale-free resetting (as prescribed by the parameter $\alpha$) does not depend on the time scale. Thus, the protocol can be optimized in a parsimonious manner, without knowing how much time on average the underlying search takes.

III.1 General analysis

Instead of assuming that the underlying search process is described by a simple, one-dimensional diffusion, we first consider any random search process. Let a non-negative random variable $\operatorname{T_{0}}$ be the completion times of the original process without resetting. Similarly, let $\operatorname{T_{\alpha}}$ denote the completion times of the same process modified by introducing scale-free resets of the form (2). Our goal is to calculate statistics of $\operatorname{T_{\alpha}}$ given known statistics of $\operatorname{T_{0}}$. In particular, we will be interested in the mean completion time (MCT), denoted as $\operatorname{\overline{T}_{\alpha}}$.

For any absolute-time-dependent rate function $r(t)$, one can link the survival probability of the model with resets $\operatorname{S_{\alpha}}(t)$ with the survival probability of the underlying reset-free process $\operatorname{S}_{0}(t)$ by means of the following renewal equation 222 Note that multiple different renewal approaches to processes with resetting have been proposed gupta2014fluctuating ; Reuveni2016 ; pal2016first ; chechkin2018random . Our approach is closely related to a discrete-time counterpart applied in kusmierz2014first .

[TABLE]

where

[TABLE]

is the probability of no resets in the interval of time $(t_{0},t_{1}]$. Eq. (8) splits the survival probability into two cases: either there are no resets until time $t$, which happens with probability $\Psi(0,t)$, or there are resets in this time period with the last reset at time $\tau$. The integral appears here because $\tau$ can be anywhere between [math] and $t$. In the case of scale-free resetting (2), the no-reset probability is given by

[TABLE]

Combining the general expression (8) with (10) and (2) we arrive at the equation

[TABLE]

In order to solve Eq. (11) we introduce an auxillary function

[TABLE]

which solves a simpler integral equation

[TABLE]

By denoting ${\operatorname{\mathcal{L}}\{f(t)\}\equiv\tilde{f}(s)}$ as the Laplace transform of $f(t)$, the corresponding Laplace-transformed equation reads

[TABLE]

In the following we will assume that the probability of finding the target in no-time is equal to zero. In this case the corresponding survival probability can be written in the form redner2001guide

[TABLE]

where $\rho_{0}(t)$ is the probability density function of the completion time of the process without resetting, and ${\lim_{s\to\infty}\operatorname{\tilde{\rho}_{0}}(s)=0}$. We can easily solve the differential equation (14) leading to

[TABLE]

where $C_{\alpha}$ is a constant yet to be determined.

We now show how to recover $\operatorname{\tilde{S}_{\alpha}}(s)$ from $\operatorname{\tilde{g}_{\alpha}}(s)$. We employ the identity

[TABLE]

which holds for $\beta>-1$333 The integral operator in (17) and (18) corresponds to the Riemann-Liouville fractional integral. Similarly, the integral in (19) can be expressed by means of the Caputo fractional derivative. . From (17) we deduce that

[TABLE]

We can apply (18) in (12) by replacing $\beta$ with $\alpha-1$ and noticing that $\operatorname{\mathcal{L}}\{tf(t)\}=-\partial_{s}\tilde{f}(s)$ 444 We could have also applied (18) directly with $\beta=\alpha-2$. The resulting equation is simpler, but holds only for $\alpha>1$. , which leads to

[TABLE]

In the last step we plug (16) into (19). The initial condition

[TABLE]

allows us to calculate $C_{\alpha}=\Gamma(\alpha)$. The final result reads

[TABLE]

Eq. (21) allows us to calculate any statistics of the completion times, at least in principle. In particular, the MCT is simply given by $\operatorname{\tilde{S}_{\alpha}}(0)$, i.e.

[TABLE]

III.2 Special cases

Here we present two special cases of the search process $\operatorname{T_{0}}$, which illustrates basic features of the scale-free resetting.

III.2.1 1D diffusion with a single trap

In the first example we consider one-dimensional diffusion (1) with a single target (trap). The probability density function of the completion times (here corresponding to the first passage times) for $\alpha=0$ (no resets) is well-known and in the Laplace space reads redner2001guide

[TABLE]

where $\tau_{\mathrm{diff}}\equiv x^{2}_{0}/D$. Importantly, $\rho_{0}(t)\sim t^{-3/2}$ for large $t$ and thus the MCT is infinite. Plugging (23) into the general expression for the MCT (22) leads to a rather complicated integral, which can nevertheless be easily integrated numerically. Numerical simulations confirm our theoretical prediction (cf. Fig. 2) and show that the MCT attains its minimum value of $\operatorname{\overline{T}}_{\alpha^{*}}/\tau_{\mathrm{diff}}\approx 1.97$ at $\alpha^{*}\approx 3.5$. Moreover, as expected from the dimensionality analysis, the MCT scales quadratically with the initial distance to the target.

We can learn a lot about the distribution of $\operatorname{T_{\alpha}}$ by directly analyzing the auxiliary function (16), which in the case of diffusion takes the form

[TABLE]

where

[TABLE]

is a version of the exponential integral. For $x>0$ it can be expressed as

[TABLE]

where $\gamma\approx 0.5772$ stands for the Euler-Mascheroni constant. Thus the solution (24) has the following representation

[TABLE]

By letting $s=0$ in (27) we arrive at a simple formula for the $\alpha$-th (fractional) moment

[TABLE]

For $\alpha=1$ we obtain a simple expression for the MCT

[TABLE]

As expected, the MCT scales quadratically with the distance to the target. Small $s$ expansion of (27) shows that ${\operatorname{g_{\alpha}}(t)\sim t^{-3/2}}$ for large $t$, which together with (12) implies that the tail of the survival probability behaves like

[TABLE]

Therefore, the fractional moments ${\langle\operatorname{T_{\alpha}}^{\nu}\rangle}$ are finite if and only if ${\nu<\alpha+1/2}$. In particular, the MCT is finite for $\alpha>1/2$, see Fig. 2.

III.2.2 Random search with failure

As we show in Section III.3, the optimal choice of $\alpha$ does not depend on the search-problem time scale. However, this choice may be sensitive to other features of $\rho_{0}$. In the second example we explore the resilience of the scale-free resetting to search failures. Let the search problem be described by the density

[TABLE]

where $p\leq 1$ is the probability of a successful search. The trivial case $p=1$ corresponds to deterministic completion time, whereas for $p<1$ the density is unnormalized as there is non-zero probability $1-p$ that the search ends with a failure, i.e. the completion time is infinite. We plug

[TABLE]

into (16) and arrive at

[TABLE]

and thus via the expansion (26)

[TABLE]

for $p<1$ and large $t$. We conclude that the MCT is finite if and only if $p=1$ or $\alpha>1/p$, see Fig. 3. Moreover, the MCT is again proportional to time scale $\tau_{0}$:

[TABLE]

This example illustrates the fact that, while robust to the time scale, scale-free resetting may be sensitive to other features of the search problem. In particular, the higher the probability of the search failure, the larger resetting intensity $\alpha$ is necessary to assure a finite value of the MCT. We discuss this issue in more detail in Section III.4.3.

III.3 Linear scale-dependence

We have seen that in two simple special cases the MCT is proportional to the time scale of the underlying search problem. Here we show that this is always the case and that this observation generalizes to higher moments. Given the completion time of a process without resets $\operatorname{T_{0}}$, we define a rescaled completion time as $\tau_{0}\operatorname{T_{0}}$. As in Section III.2.2, the parameter $\tau_{0}$ denotes the time-scale of the search problem, and may be related to its different features, e.g. in the case of diffusion $\tau_{0}$ is proportional to $x_{0}^{2}/D$. The corresponding probability density functions are related as

[TABLE]

whereas the survival probabilities

[TABLE]

Let $\operatorname{T_{\alpha}}(\tau_{0})$ denote the completion time of a process with scale-free resetting with the underlying reset-free completion times given by $\tau_{0}\operatorname{T_{0}}$. We substitute variables ${v\to v/\tau_{0}}$ and ${u\to u/\tau_{0}}$ in (21) and obtain $\operatorname{\tilde{S}_{\alpha}}(s,\tau_{0})=\tau_{0}\operatorname{\tilde{S}_{\alpha}}(\tau_{0}s,1)$, which leads to the conclusion that

[TABLE]

where $\sim$ denotes the equality of distributions. For all finite fractional moments

[TABLE]

and thus the mean value scales linearly $\operatorname{\overline{T}_{\alpha}}(\tau_{0})\propto\tau_{0}$: scale-free restarts of the form (2) yield the MCT that is proportional to time scale of the underlying search problem, which explains why in the case of diffusion $\operatorname{\overline{T}_{\alpha}}\propto x_{0}^{2}$. As a consequence of (39), the optimal value of $\alpha$, as defined by any moment, is robust to changes of $\tau_{0}$. To see this, we can rewrite (39) in the form $\langle\operatorname{T_{\alpha}}(\tau_{0})^{\nu}\rangle=\tau_{0}^{\nu}h(\alpha)$, where the function $h(\alpha)$ does not depend on $\tau_{0}$. Clearly, the value of $\alpha$ that minimizes $h(\alpha)$ at the same time minimizes the $\langle\operatorname{T_{\alpha}}(\tau_{0})^{\nu}\rangle$ for any non-zero value of $\tau_{0}$.

This result is quite remarkable: the scale-free resetting provides a parsimonious search boosting technique that does not employ any knowledge about the time scale of the underlying search problem.

III.4 Comparison with other resetting protocols

III.4.1 Resetting protocols

In order to understand merits and limitations of the proposed scale-free resetting, it is instructive to compare it to other resetting protocols, i.e. predefined schemes of introducing resets into the system. We focus solely on (possibly non-stationary) feedforward protocols that are independent from the state of the system 555 In general resetting protocols may form a feedback loop that depend on the state of the system (e.g. current position $x$ in the case of diffusion pinsky2018diffusive ).

Due to its simplicity, constant-rate resetting (i.e. $r(t)=r$) is particularly popular in the literature. Diffusion with such resetting has been studied extensively and its optimal MCT is given by $\operatorname{\overline{T}}_{\mathrm{const}}(x_{0},r^{*})\approx 1.54x_{0}^{2}/D$ with the optimal resetting rate $r^{*}\propto x_{0}^{-2}$ evans2013optimal .

Another protocol that is of interest to us is a periodic, deterministic resetting. Following Reuveni2016 ; pal2016first we call this resetting sharp. Sharp resetting is important, as it was shown to form the dominant strategy in the stationary setting pal2016first .

III.4.2 Fluctuating or unknown environment

Although the constant resetting can work better than the scale-free resetting ($\operatorname{\overline{T}}_{\mathrm{const}}(x_{0},r^{*})<\operatorname{\overline{T}}_{\alpha^{*}}(x_{0})$), the optimal scale-free search parameter $\alpha^{*}$ does not involve any knowledge about the distance to the target. In contrast, the optimal constant rate $r^{*}$ depends strongly on $x_{0}$, see Fig. 2. Hence, optimizing $r$ may be hard, or even impossible, if the position of the target is unknown.

Consider a scenario of a single, immobile target placed at a random position and assume the location of the target does not change between the resets, but is drawn independently for separate trials kusmierz2015optimal ; kusmierz2017optimal ; pinsky2018optimizing . This is equivalent to the resetting (and initial) position being drawn from a distribution $\rho_{X}(x_{0})$, i.e., the MCTs of the scale-free and constant resetting processes in this case can be calculated as $\langle\operatorname{\overline{T}_{\alpha}}(x_{0})\rangle_{x_{0}\sim\rho_{X}}$ and $\langle\operatorname{\overline{T}}_{\mathrm{const}}(x_{0})\rangle_{x_{0}\sim\rho_{X}}$, respectively. The distribution $\rho_{X}$ may represent the real variability of the environment or the ignorance of the searching agent.

As an important special case, let us take the Laplace distribution

$\rho_{X}(x_{0})=\exp\left(-|x_{0}|/\lambda\right)/(2\lambda)$,

which maximizes entropy for a given average distance to the target $\lambda=\langle|x_{0}|\rangle$. In this case, the MCT of diffusion with constant resetting rate $r$ reads

[TABLE]

with the minimum value of $4\lambda^{2}/D$ at $1/r^{*}=4\lambda^{2}/D$. In contrast, since the variance of the Laplace distribution is equal to $2\lambda^{2}$, the MCT of diffusion with scale-free resetting is given by

[TABLE]

where $f(\alpha)=\operatorname{\overline{T}}_{\alpha}(x_{0})/\tau_{\mathrm{diff}}$ is the same prefactor as in the case of constant $x_{0}$ (Fig. 2) with the minimum ${f(\alpha^{*})\approx 1.97}$ at ${\alpha^{*}\approx 3.5}$—in this case the scale-free resetting leads to a (slightly) better efficiency than the constant resetting. More importantly, in that case the optimal choice of $\alpha$ does not depend on $\lambda$, to which the optimal constant rate is quite sensitive. Indeed, if the chosen constant rate is larger than $r=D/\lambda^{2}$, the MCT diverges, see (40).

More broadly, the MCT of constant resetting search diverges for any distribution of $x_{0}$ that has tails heavier than exponential, in particular for any power-law distribution. In contrast, the MCT of diffusion with scale-free resetting remains finite for any distribution of $x_{0}$ with finite variance and the choice of the optimal parameter $\alpha$ does not depend on any aspect of the distribution of $x_{0}$,

III.4.3 Trade-off

Our results suggest that resetting protocols entail a natural trade-off between sensitivity of the protocol outcomes to two distinct features of the completion times of the underlying search problem: its time scale and the shape of its tail. It is convenient, in this context, to analyze the survival probability of the completion times. The tail of the survival probability captures both probability of large fluctuations and the failure probability, i.e. in the case of a non-zero failure probability, the survival probability does not decay to zero.

The scale-free resetting is robust to the time scale. However, the time-scale robustness does not come without a price: due to the ever-growing intervals between successing resets, this protocol is rather sensitive to the tail of the distribution. In the case of a one-dimensional diffusive search, the survival probability of the completion times has a power-law tail $\sim t^{-1/2}$. The scale-free resetting to some extent tempers the tail, leading to a finite value of the MCT for $\alpha>\frac{1}{2}$. However, the fluctuations of the completion times are still large, with the survival probability $\sim t^{-1/2-\alpha}$. Thus, the variance diverges for $\alpha\leq\frac{3}{2}$. Similarly, the scale-free resetting is sensitive to the failure probability: as shown above, in order to retain a finite MCT, $\alpha$ has to be larger than $1/p$, where $1-p$ is the failure probability, see Fig. 3.

At the other extreme is the sharp resetting protocol. Indeed, the optimal sharp resetting shows shorter MCTs and lower relative fluctuations than any stochastic resetting in a known search environment pal2016first . Additionally, it is easy to show that the optimal sharp protocol does not depend on the failure probability $p$. However, sharp resets are at the same time extremely sensitive to the time scale of the search process. In the case of diffusion they lead to divergent MCTs for any Laplace distribution of $x_{0}$ kusmierz2017optimal .

Within the framework of the discussed trade-off, constant-rate resetting is placed in between sharp resetting and scale-free resetting. In the case of diffusion, fluctuations of the completion times are strongly reduced by this kind of resetting—all moments are finite for any $r$ Reuveni2016 ; pal2016first . In the simple case of constant time failure model (31) its MCT reads

[TABLE]

The optimal parameters of the constant-rate resetting protocol depend both on scale and failure probability, see Fig. 3. On the one hand, in contrast to the scale-free resetting protocol, the MCT is in this case finite for any $p$. On the other hand, for a given value of the resetting rate, the completion time is very sensitive to $\tau_{0}$.

To sum up, sharp resetting is the most efficient resetting protocol in a well-known, static environment (search problem). However, such a protocol is highly sensitive to the time scale of the search problem and may easily fail in the case of an imperfect knowledge or fluctuating environment. If these fluctuations (or ignorance) are not too large and the average time scale is known, constant-rate stochastic resets are advantageous. Otherwise, scale-free resets may be of great advantage, since they are robust to the time scale of the search problem.

III.4.4 Generalizations

The constant-rate and scale-free resetting protocols are special cases of the general family of resetting protocols of the form $r(t)=\alpha/t^{\mu}$, introduced in Section II.2. As shown therein, supplementing diffusion with such a resetting protocol with $\mu<1$ gives rise to the subdiffusive behavior. Thus, this protocol introduces a time scale and, for a fixed $\alpha$, its MCT scales superlinearly with $\tau_{0}$. It is therefore not robust, as the optimal $\alpha$ depends on $\tau_{0}$. Nonetheless, this protocol may still be efficient in some search problems. Since in the limit of $\mu\to 0$ the standard constant-resetting case is recovered, in search problems with large variability one can expect that the fluctuations of the completion times are smaller for lower values of $\mu$. Thus, the subdiffusive protocol may prove useful in the context of balancing the trade-off discussed in Section III.4.3. The rich family of models with absolute-time-dependent stochastic resetting and the associated trade-off will be the subject of a separate study.

IV Conclusions

We envision multiple applications of scale-free resetting, especially in optimization problems. For instance, it seems natural to ask about its relation to simulated annealing kirkpatrick1983optimization , and its applicability in evolutionary algorithms fukunaga1998restart ; jansen2002analysis , deep learning via gradient methods lecun2015deep ; luersen2004globalized , and biologically plausible learning techniques with three factors fremaux2016neuromodulated ; kusmierz2017learning . The robustness of the scale-free resetting protocol should offer additional benefits in the non-stationary setup of curriculum learning bengio2009curriculum and could potentially offer an explanation as to why aging villain1977spin ; metzler2014anomalous may be useful in learning. Of course the practical optimization problems are high-dimensional and thus the general renewal framework (8) should prove useful in the construction and analysis of the practical algorithms. Another interesting avenue of possible applications are models of evidence accumulation and decision making bogacz2006physics ; tajima2016optimal , as recently it was shown that stationary resets can modify splitting probabilities belan2018restart . Restarts in this context may be interpreted as useful forgetting wixted2004psychology ; hardt2013decay .

A number of open problems are left for future studies. In order to assess the efficiency of the proposed scheme, one could compare it to diffusion in time-dependent, scale-free potentials—such comparison in the case of static potentials and resets seems to favor the latter evans2013optimal ; kusmierz2017optimal . Moreover, it is important to find conditions under which scale-free resets can lower the expected completion time, similar to the simple criterion in the case of constant-rate resets reuveni2014role ; rotbart2015michaelis ; Reuveni2016 . A related question is how the optimal $\alpha$ depends on the details of a search problem at hand. Another important issue is the divergence of (2) at $t=0$ which is infeasible and in practice a short-time cut-off has to be introduced. The optimal cut-off should depend on a cost associated with resets.

Acknowledgements.

We are indebted to Shun Ogawa for stimulating discussions. This work was supported by RIKEN Center for Brain Science, Brain/MINDS from AMED under Grant Number JP19dm020700 and JSPS KAKENHI Grant Number JP18H05432.