Nonconvex homogenization for one-dimensional controlled random walks in random potential

TL;DR

This paper studies the homogenization of a nonconvex Hamilton-Jacobi equation arising from a controlled random walk in a random potential, identifying asymptotically optimal policies and the nature of the effective Hamiltonian.

Contribution

It provides a constructive proof of homogenization for a nonconvex stochastic PDE and characterizes the effective Hamiltonian's convexity properties across different control regimes.

Findings

Effective Hamiltonian is convex when control is absent (),

Effective Hamiltonian becomes nonconvex and piecewise linear when control is maximal (
),

Homogenization results depend on the control parameter
and the potential's properties.

Abstract

We consider a finite horizon stochastic optimal control problem for nearest-neighbor random walk $\{X_i\}$ on the set of integers. The cost function is the expectation of exponential of the path sum of a random stationary and ergodic bounded potential plus $\theta X_n$. The random walk policies are measurable with respect to the random potential, and are adapted, with their drifts uniformly bounded in magnitude by a parameter $\delta\in[0,1]$. Under natural conditions on the potential, we prove that the normalized logarithm of the optimal cost function converges. The proof is constructive in the sense that we identify asymptotically optimal policies given the value of the parameter $\delta$, as well as the law of the potential. It relies on correctors from large deviation theory as opposed to arguments based on subadditivity which do not seem to work except when $\delta = 0$. The Bellman equation associated to this control problem is a second-order Hamilton-Jacobi (HJ) stochastic partial difference equation with a separable random Hamiltonian which is nonconvex in $\theta$ unless $\delta = 0$. We prove that this equation homogenizes under linear initial data to a first-order HJ deterministic partial differential equation. When $\delta = 0$, the effective Hamiltonian is the tilted free energy of random walk in random potential and it is convex in $\theta$. In contrast, when $\delta = 1$, the effective Hamiltonian is piecewise linear and nonconvex in $\theta$. Finally, when $\delta \in (0,1)$, the effective Hamiltonian is expressed completely in terms of the tilted free energy for the $\delta=0$ case and its convexity/nonconvexity in $\theta$ is characterized by a simple inequality involving $\delta$ and the magnitude of the potential, thereby marking two qualitatively distinct control regimes.

Equations

$${\mathcal{P}}_n(\delta )=$$

$${\pi }_i(n,\omega ,y,-1)+{\pi }_i(n,\omega ,y,1)=1\text{and}|{\pi }_i(n,\omega ,y,1)-{\pi }_i(n,\omega ,y,-1)|\le \delta$$

$$\text{for every i∈[0,n−1], ω∈Ω and y∈Z}\}.$$

$$P_x^{\pi ,\omega }(X_0=x_0,X_1=x_1,\dots ,X_n=x_n)=\prod _{i=0}^{n-1}{\pi }_i(n,\omega ,x_i,z_{i+1}).$$

$$u(n,x,\omega |\delta ,\beta ,\theta )=\underset{\pi \in {\mathcal{P}}_n(\delta )}{\inf }\log E_x^{\pi ,\omega }\left[e^{\beta {\sum }_{i=0}^{n-1}V(T_{X_i}\omega )+\theta X_n}\right].$$

$$u_o(t,x|\delta ,\beta ,\theta )=\underset{ϵ\to 0}{\lim }ϵu\left([{ϵ}^{-1}t],[{ϵ}^{-1}x],\omega |\delta ,\beta ,\theta \right)$$

$$u_o(t,x|\delta ,\beta ,\theta )=t{\overline{H}}_{\delta ,\beta }(\theta )+\theta x.$$

$$\frac{\partial u_o}{\partial t}(t,x)={\overline{H}}_{\delta ,\beta }\left(\frac{\partial u_o}{\partial x}(t,x)\right).$$

$$V(\omega )$$

$$\mathbb{P}(\text{[0,ℓ] is an h-valley})>0\text{and}\mathbb{P}(\text{[0,ℓ] is an h-hill})>0\text{for every h∈(0,1) and ℓ∈N.}$$

$${\omega }_{s+2k-1}={\omega }_{s+2k}={\alpha }_k$$

$${\overline{H}}_{0,\beta }(\theta )={\Lambda }_{\beta }(\theta ):=\underset{n\to \infty }{\lim }\frac{1}{n}\log E_0\left[e^{\beta {\sum }_{i=0}^{n-1}V(T_{X_i}\omega )+\theta X_n}\right],$$

$${\overline{H}}_{1,\beta }(\theta )=\{\begin{array}{ll}0& \text{if}|\theta |<\beta \mathbb{E}[V(\cdot )],\\ \beta \mathbb{E}[V(\cdot )]-|\theta |& \text{if}|\theta |\ge \beta \mathbb{E}[V(\cdot )].\end{array}$$

$$c=\frac{1}{2}\log \left(\frac{1+\delta }{1-\delta }\right).$$

$${\overline{H}}_{\delta ,\beta }(\theta )=\{\begin{array}{ll}\beta -\log \cosh (c)& \text{if}|\theta |<c,\\ {\Lambda }_{\beta }(|\theta |-c)-\log \cosh (c)& \text{if}|\theta |\ge c.\end{array}$$

$${\Lambda }_{\beta }(\bar{\theta }(\beta ,c)-c)=\log \cosh (c),$$

$${\overline{H}}_{\delta ,\beta }(\theta )=\{\begin{array}{ll}0& \text{if}|\theta |<|\bar{\theta }(\beta ,c)|,\\ {\Lambda }_{\beta }(|\theta |-c)-\log \cosh (c)& \text{if}|\theta |\ge |\bar{\theta }(\beta ,c)|.\end{array}$$

$${\pi }_i^{(x_{\ast },h,\ell )}(n,\omega ,y,1)=\{\begin{array}{ll}\frac{1+\delta }{2}& \text{if y<x∗},\\ \frac{1-\delta }{2}& \text{if y≥x∗}.\end{array}$$

$${\overleftarrow{\pi }}_i(n,\omega ,y,1)\equiv \frac{1-\delta }{2}\text{and}{\overrightarrow{\pi }}_i(n,\omega ,y,1)\equiv \frac{1+\delta }{2}.$$

$$e^{u(n,x,\omega )}=\underset{\pi \in {\mathcal{P}}_n(\delta )}{\inf }{E}_x^{\pi ,\omega }\left[e^{\beta {\sum }_{i=0}^{n-1}V(T_{X_i}\omega )+\theta X_n}\right].$$

$$u(n+1,x,\omega )=\beta V(T_x\omega )+\underset{q\in [\frac{1-\delta }{2},\frac{1+\delta }{2}]}{\inf }\log \left(q{e}^{u(n,x+1,\omega )}+(1-q)e^{u(n,x-1,\omega )}\right).$$

$${\nabla }_{1}u(n,x,\omega )=\{\begin{array}{ll}\frac{1}{2}{\Delta }_{2}u(n,x,\omega )+\log \cosh ({\nabla }_{2}u(n,x,\omega ))+\beta V(T_x\omega )& \text{if δ=0,}\\ \frac{1}{2}{\Delta }_{2}u(n,x,\omega )+\log \cosh (|{\nabla }_{2}u(n,x,\omega )|-c)-\log \cosh (c)+\beta V(T_x\omega )& \text{if δ∈(0,1),}\\ \frac{1}{2}{\Delta }_{2}u(n,x,\omega )-|{\nabla }_{2}u(n,x,\omega )|+\beta V(T_x\omega )& \text{if δ=1.}\end{array}$$

$${\nabla }_{1}u(n,x,\omega )$$

$${\nabla }_{2}u(n,x,\omega )$$

$${\Delta }_{2}u(n,x,\omega )$$

$$H_{\delta ,\beta }(\theta ,x,\omega )=K_{\delta }(\theta )+\beta V(T_x\omega ),K_{\delta }(\theta )=\{\begin{array}{ll}\log \cosh (\theta )& \text{if δ=0,}\\ \log \cosh (|\theta |-c)-\log \cosh (c)& \text{if δ∈(0,1),}\\ -|\theta |& \text{if δ=1.}\end{array}$$

$$u_{ϵ}(t,x,\omega )=ϵu\left([{ϵ}^{-1}t],[{ϵ}^{-1}x],\omega \right).$$

$${\nabla }_1^{ϵ}{u}_{ϵ}(t,x,\omega )=\frac{ϵ}{2}{\Delta }_2^{ϵ}{u}_{ϵ}(t,x,\omega )+H_{\delta ,\beta }({\nabla }_2^{ϵ}{u}_{ϵ}(t,x,\omega ),[{ϵ}^{-1}x],\omega ),$$

$${\nabla }_1^{ϵ}{u}_{ϵ}(t,x,\omega )$$

$${\nabla }_2^{ϵ}{u}_{ϵ}(t,x,\omega )$$

$${\Delta }_2^{ϵ}{u}_{ϵ}(t,x,\omega )$$

$$\frac{\partial u_o}{\partial t}(t,x)={\overline{H}}_{\delta ,\beta }\left(\frac{\partial u_o}{\partial x}(t,x)\right),u_o(0,x)=\theta x.$$

$$\text{Hδ,β(θ) is convex in θ}⟺\log \cosh (c)\le \beta ⟺\delta \le \sqrt{1-e^{-2\beta }}.$$

Full text

The Bellman equation associated to this control problem is a second-order Hamilton-Jacobi (HJ) stochastic partial difference equation with a separable random Hamiltonian which is nonconvex in $\theta$ unless $\delta=0$. We prove that this equation homogenizes under linear initial data to a first-order HJ deterministic partial differential equation. When $\delta=0$, the effective Hamiltonian is the tilted free energy of random walk in random potential and it is convex in $\theta$. In contrast, when $\delta=1$, the effective Hamiltonian is piecewise linear and nonconvex in $\theta$. Finally, when $\delta\in(0,1)$, the effective Hamiltonian is expressed completely in terms of the tilted free energy for the $\delta=0$ case and its convexity/nonconvexity in $\theta$ is characterized by a simple inequality involving $\delta$ and the magnitude of the potential, thereby marking two qualitatively distinct control regimes.

A. Yilmaz was partially supported by European Union FP7 Marie Curie Career Integration Grant no. 322078 and by the BAGEP Award of the Science Academy, Turkey.

O. Zeitouni was partially supported by an Israel Science Foundation grant

1. Introduction

1.1. Controlled random walks in random potential

Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space that is equipped with an ergodic invertible measure-preserving transformation $T:\Omega\to\Omega$. Elements of $\Omega$ are denoted by $\omega$ and referred to as environments. For every $n\in\mathbb{N}=\{1,2,\ldots\}$ and $\delta\in[0,1]$, define

[TABLE]

Each $\pi\in\mathcal{P}_{n}(\delta)$ is a (Markov) random walk policy whose drift is uniformly bounded in magnitude by $\delta$. Given any environment $\omega\in\Omega$ and starting point $x\in\mathbb{Z}$, $\pi$ induces a probability measure $P_{x}^{\pi,\omega}$ on the space of paths $x_{0,n}=(x_{0},x_{1},\ldots,x_{n})\in\mathbb{Z}^{n+1}$ with $x_{0}=x$ and $z_{i+1}=x_{i+1}-x_{i}\in\{-1,1\}$, defined by

[TABLE]

Expectation under $P_{x}^{\pi,\omega}$ is denoted by $E_{x}^{\pi,\omega}$.

Let $V:\Omega\to[0,1]$ be a nonconstant measurable function. $V(T_{y}\omega)$ is referred to as the potential at the point $y$ in the environment $\omega$. Here and throughout, $T_{0}=I$, $T_{k}=T\circ T_{k-1}$ and $T_{-k}=(T_{k})^{-1}$ for $k\in\mathbb{N}$. For every $n\in\mathbb{N}$, $x\in\mathbb{Z}$, $\omega\in\Omega$, $\delta\in[0,1]$, $\beta>0$ and $\theta\in\mathbb{R}$, let

[TABLE]

Note that the left-hand side of (1.1) would not change if we took the infimum on the right-hand side over the larger set of adapted (but not necessarily Markov) random walk policies with drifts still uniformly bounded in magnitude by $\delta$. (See [7, Proposition 11.7].)

1.2. Overview of our results

We show in Section 2.1 that, under natural assumptions, for $\mathbb{P}$-a.e. $\omega$ the limit

[TABLE]

exists for every $t>0$ and $x\in\mathbb{R}$ (where $[\cdot]$ denotes the floor function), and it is of the form

[TABLE]

$\overline{H}_{\delta,\beta}(\theta):=u_{o}(1,0\,|\,\delta,\beta,\theta)$ is a deterministic quantity for which we provide a formula. In fact, for $\delta>0$ we express $\overline{H}_{\delta,\beta}(\cdot)$ completely in terms of $\overline{H}_{0,\beta}(\cdot)$. The existence of the latter was already known (see Section 1.4) and can be shown via subadditivity (see Appendix A). However, there is no subadditivity to be exploited when $\delta>0$, so instead we develop a constructive approach. In particular, in Section 2.2 we identify asymptotically optimal policies (as $n\to\infty$) for the control problem in (1.1).

We make two observations. First, the Bellman equation associated to the control problem in (1.1) is a second-order Hamilton-Jacobi (HJ) stochastic partial difference equation (see (2.8)). Second, the function $u_{o}(t,x)=u_{o}(t,x\,|\,\delta,\beta,\theta)$ (given in (1.3)) satisfies the following first-order HJ deterministic partial differential equation:

[TABLE]

Due to the limit in (1.2) under an appropriate scaling of time and space, the former equation (with linear initial data) is said to homogenize to the latter one. See Section 2.3 for details and also Section 2.4 for related results from the homogenization literature. Therefore, throughout the paper, $\overline{H}_{\delta,\beta}(\theta)$ will be referred to as the effective Hamiltonian.

1.3. Assumptions on the potential

Since the potential inside the expectation on the right-hand side of (1.1) is scaled by $\beta$, there is no loss of generality in assuming that

[TABLE]

Our results will further require the existence of arbitrarily long finite intervals where the potential is uniformly close to its essential infimum (resp. supremum). In order to make this condition precise, we introduce two terms.

Definition 1.1**.**

For any $\omega\in\Omega$ and $h\in(0,1)$, an interval $[k,\ell]\subset\mathbb{Z}$ is said to be an $h$-valley (resp. $h$-hill) if $V(T_{y}\omega)\leq h$ (resp. $V(T_{y}\omega)\geq h$) for every $y\in[k,\ell]$.

With this terminology, we will assume that

[TABLE]

Note that this assumption does not imply that the environment is mixing, as Example 1.3 below shows.

Example 1.2*.*

Let $\Omega=[0,1]^{\mathbb{Z}}$ and $\mathcal{F}$ the Borel $\sigma$-algebra on $\Omega$. Define $T:\Omega\to\Omega$ by $(T\omega)_{y}=\omega_{y+1}$ for any $\omega=(\omega_{y})_{y\in\mathbb{Z}}\in\Omega$. Assume that

• (i)

$\mathbb{P}$ is a probability measure on $(\Omega,\mathcal{F})$ that is stationary and ergodic under $T$, and

• (ii)

there exists a Borel probability measure $\mu$ on $[0,1]$ such that the product measure $\prod_{y\in\mathbb{Z}}\mu$ is absolutely continuous with respect to $\mathbb{P}$ on $\mathcal{F}_{0,\ell}=\sigma\{\omega_{0},\ldots,\omega_{\ell}\}$ for every $\ell\in\mathbb{N}$.

Consider the function $V:\Omega\to[0,1]$ given by $V(\omega)=\omega_{0}$. Then, (1.4) is equivalent to $\mu$ having full support, in which case (1.5) holds by the assumption of absolute continuity.

Example 1.3*.*

With $\Omega$, $\mathcal{F}$, $T$ and $V$ as in Example 1.2, let $\{s\}\cup\{\alpha_{k}:\,k\in\mathbb{Z}\}$ be an i.i.d. collection of ($\{0,1\}$-valued) Bernoulli trials with success probability $1/2$. Define $\omega=(\omega_{y})_{y\in\mathbb{Z}}$ by setting

[TABLE]

for every $k\in\mathbb{Z}$. This induces a probability measure $\mathbb{P}$ on $(\Omega,\mathcal{F})$. It is clear that $\mathbb{P}$ is stationary and ergodic under $T$. Moreover, (1.4) and (1.5) trivially hold. However, $\omega$ is not even weakly mixing under $\mathbb{P}$, since an elementary computation shows that with $A=\{\omega_{-1}=\omega_{0}\}$ one has that $\mathbb{P}(A\cap T^{-2k}A)=5/8$ for all $k\neq 0$ while $(\mathbb{P}(A))^{2}=9/16$.

1.4. Special case: No control

If $\delta=0$, then $\mathcal{P}_{n}(\delta)$ is a singleton whose unique element satisfies $\pi_{i}(n,\omega,y,\pm 1)\equiv 1/2$ and induces simple symmetric random walk (SSRW) on $\mathbb{Z}$. In this case, we simplify the notation and write $P_{x}$ (resp. $E_{x}$) instead of $P_{x}^{\pi,\omega}$ (resp. $E_{x}^{\pi,\omega}$).

Theorem 1.4** (No control).**

Assume (1.4) and (1.5). If $\delta=0$, $\beta>0$ and $\theta\in\mathbb{R}$, then for $\mathbb{P}$-a.e. $\omega$ the limit in (1.2) exists for every $t>0$ and $x\in\mathbb{R}$. Moreover, (1.3) holds and the effective Hamiltonian is given by

[TABLE]

the so-called tilted free energy.

The existence of the tilted free energy was shown in several previous works in much greater generality. Zerner [30] considered nearest-neighbor random walks (RWs) in i.i.d. random potential on $\mathbb{Z}^{d}$ (with any $d\geq 1$) and gave a subadditivity argument that proves the existence of certain Lyapunov exponents which in turn imply a large deviation principle (LDP) for the position of the walk. Then, Flury [13] used Zerner’s large deviation result to show the existence of the tilted free energy in the same setting. These two papers built upon earlier work by Sznitman [27] on Brownian motion in a Poissonian potential on $\mathbb{R}^{d}$. By another subadditivity argument, Varadhan [28] bypassed Lyapunov exponents and directly established a similar LDP for a closely related model, namely nearest-neighbor RW in stationary and ergodic (not necessarily i.i.d.) random environment on $\mathbb{Z}^{d}$. It is easy to adapt Varadhan’s argument to give a short proof of the existence of the tilted free energy for RW in random potential on $\mathbb{Z}^{d}$. We do this in a more general setup in Theorem A.1 of Appendix A for the sake of completeness and with future use in mind. There are alternative proofs of Theorem A.1 which provide variational formulas for the tilted free energy [29, 23, 22, 24]. See Remark A.2 for details.

In Section 4, we will take advantage of our one-dimensional setting to present a self-contained proof of Theorem 1.4 (which is not based on subadditivity) and give an implicit (non-variational) formula for the tilted free energy $\Lambda_{\beta}(\theta)$. We will also show some properties of $\Lambda_{\beta}(\theta)$ as a function of $\beta$ and $\theta$ (see Proposition 4.8). In particular, if $\delta=0$, then the effective Hamiltonian $\overline{H}_{0,\beta}(\theta)=\Lambda_{\beta}(\theta)$ is convex in $\theta$ for every $\beta>0$.

2. Results

2.1. The effective Hamiltonian

As we present below, for $\mathbb{P}$-a.e. $\omega$ the limit in (1.2) exists for every $t>0$ and $x\in\mathbb{R}$ under the assumptions (1.4) and (1.5). Recall from Section 1.4 that the special case of no control (i.e., $\delta=0$) is studied in detail in Section 4. The other extreme case is $\delta=1$, i.e., when we can fully control the trajectory of the particle performing the walk. The analysis of the latter case involves the same approach as the intermediate case $\delta\in(0,1)$ but it is technically simpler, so we present it first.

Theorem 2.1** (Full control).**

Assume (1.4) and (1.5). If $\delta=1$, $\beta>0$ and $\theta\in\mathbb{R}$, then for $\mathbb{P}$-a.e. $\omega$ the limit in (1.2) exists for every $t>0$ and $x\in\mathbb{R}$. Moreover, (1.3) holds and the effective Hamiltonian is given by

[TABLE]

When $\delta\in(0,1)$, we can only partially control the trajectory of the particle. In order to give a tidy formula for $\overline{H}_{\delta,\beta}(\theta)$, we introduce the parameter

[TABLE]

The comparison of $\beta$ and $\log\cosh(c)$ (or equivalently of $\sqrt{1-e^{-2\beta}}$ and $\delta$) turns out to play a critical role, giving rise to two qualitatively distinct regimes to which we will refer below as weak control and strong control.

Theorem 2.2** (Weak control).**

Assume (1.4) and (1.5). If $\delta\in(0,1)$, $\beta\geq\log\cosh(c)$ and $\theta\in\mathbb{R}$, then for $\mathbb{P}$-a.e. $\omega$ the limit in (1.2) exists for every $t>0$ and $x\in\mathbb{R}$. Moreover, (1.3) holds and the effective Hamiltonian is given by

[TABLE]

Theorem 2.3** (Strong control).**

Assume (1.4) and (1.5). If $\delta\in(0,1)$, $\beta<\log\cosh(c)$ and $\theta\in\mathbb{R}$, then for $\mathbb{P}$-a.e. $\omega$ the limit in (1.2) exists for every $t>0$ and $x\in\mathbb{R}$. Moreover, (1.3) holds, there exists a unique $\bar{\theta}(\beta,c)\in(0,c)$ such that

[TABLE]

and the effective Hamiltonian is given by

[TABLE]

Substituting $c=0$ in (2.3) reproduces the formula in (1.6). Similarly, taking $c\to\infty$ in (2.4) reproduces the formula in (2.1) by Proposition 4.8(d).

2.2. Asymptotically optimal policies

The proofs of Theorems 2.1, 2.2 and 2.3 are constructive in the sense that we identify RW policies that are asymptotically optimal in each case. We introduce these policies below.

For every $h\in(0,1)$, $\ell\in\mathbb{N}$ and $\mathbb{P}$-a.e. $\omega$, we choose an $h$-valley (recall from Definition 1.1) of the form $[x_{*}-\ell,x_{*}+\ell-1]$ with some $x_{*}\in\mathbb{Z}$ that is suitably close to the starting point of the RW (see Remark 5.1 for details). We define a RW policy $\pi^{(x_{*},h,\ell)}$ by setting

[TABLE]

Note that it is a bang-bang policy (see, e.g., [5]). We also consider the spatiotemporally constant bang-bang policies $\overleftarrow{\pi}$ and $\overrightarrow{\pi}$ given by

[TABLE]

In each of the three regimes of weak, strong and full control, the graph of $\overline{H}_{\delta,\beta}(\theta)$ against $\theta$ has a flat region centered at the origin (see Figure 1). When $\theta$ is in this flat region, it will turn out that the infimum in (1.1) can be taken over the set of $\pi^{(x_{*},h,\ell)}$ with arbitrarily small $h\in(0,1)$ and arbitrarily large $\ell\in\mathbb{N}$. Doing so creates a $o(n)$ difference which does not change the limit in (1.2). (When $\delta=1$, it suffices to take $\ell=1$.) On the other hand, when $\theta$ is to the right (resp. left) of the flat region centered at the origin, it will turn out that the infimum in (1.1) is asymptotically attained at $\overleftarrow{\pi}$ (resp. $\overrightarrow{\pi}$) up to a $o(n)$ term as $n\to\infty$.

Even though the regimes of weak and strong control share a common class of asymptotically optimal policies at (say) $\theta=0$, namely the policies $\pi^{(x_{*},h,\ell)}$, the value of $\overline{H}_{\delta,\beta}(0)$ is different in these two cases (see Theorems 2.2 and 2.3), which is caused by the difference in the large deviation behavior of the walk under $\pi^{(x_{*},h,\ell)}$. In this sense, our optimal control problem can be thought of as a two-person game where the players are (i) the controller and (ii) the particle exhibiting atypical behavior. This point will become clear in the proofs.

2.3. Homogenization of the Bellman equation

For every $n\in\mathbb{N}$, $x\in\mathbb{Z}$, $\omega\in\Omega$, $\delta\in[0,1]$, $\beta>0$ and $\theta\in\mathbb{R}$, we write $u(n,x,\omega)=u(n,x,\omega\,|\,\delta,\beta,\theta)$ for notational brevity and then arrange (1.1) as

[TABLE]

Decomposing the expectation in the corresponding expression for $e^{u(n+1,x,\omega)}$ with respect to the first step of the controlled walk and applying the Bellman principle gives

[TABLE]

Due to linearity in the parameter $q$ and the monotonicity of the logarithm function, the infimum on the right-hand side of (2.7) is attained at $\frac{1-\delta}{2}$ or $\frac{1+\delta}{2}$. (Therefore, the infimum in (1.1) can be taken over the set of bang-bang policies. We will recapitulate and use this in Section 6.1.) Evaluating this infimum, switching to the parameter $c$ introduced in (2.2) in the case $\delta\in(0,1)$, and finally substracting $u(n,x,\omega)$ from both sides of (2.7), we deduce that

[TABLE]

Here, we use the notation

[TABLE]

for these difference operators. Hence, (1.1) solves a second-order HJ stochastic partial difference equation, subject to the linear initial condition $u(0,x,\omega)=\theta x$, with the following separable random Hamiltonian:

[TABLE]

For every $\epsilon>0$, $t\geq 0$, $x\in\mathbb{R}$ and $\omega\in\Omega$, let

[TABLE]

After appropriate substitutions, (2.8) becomes

[TABLE]

where

[TABLE]

As we mentioned in Section 1.2, the function $u_{o}(t,x)=u_{o}(t,x\,|\,\delta,\beta,\theta)=t\overline{H}_{\delta,\beta}(\theta)+\theta x$ solves

[TABLE]

Our final result combines Theorems 1.4, 2.1, 2.2 and 2.3, improves the pointwise convergence (in $t>0$ and $x\in\mathbb{R}$) in their statements to uniform convergence on compact sets.

Theorem 2.4** (Homogenization with linear initial data).**

Assume (1.4) and (1.5). If $\delta\in[0,1]$, $\beta>0$ and $\theta\in\mathbb{R}$, then for $\mathbb{P}$-a.e. $\omega$ the function $u_{\epsilon}(\cdot,\cdot,\omega)$ converges to $u_{o}(\cdot,\cdot)$ as $\epsilon\to 0$, uniformly on compact subsets of $[0,\infty)\times\mathbb{R}$, with the effective Hamiltonian $\overline{H}_{\delta,\beta}(\theta)$ given in (1.6), (2.1), (2.3) and (2.4) in the cases of no, full, weak and strong control, respectively.

In the language of homogenization theory, Theorem 2.4 says that the second-order HJ stochastic partial difference equation in (2.10) with the initial condition $u_{\epsilon}(0,x,\omega)=\theta\epsilon[\epsilon^{-1}x]$ homogenizes to the first-order HJ deterministic partial differential equation in (2.11).

The original Hamiltonian $H_{\delta,\beta}(\theta,x,\omega)=K_{\delta}(\theta)+\beta V(T_{x}\omega)$ (given in (2.9)) is convex in $\theta$ in the case of no control, and it is nonconvex in the cases of weak, strong and full control. On the other hand, the effective Hamiltonian $\overline{H}_{\delta,\beta}(\theta)$ is convex in $\theta$ in the cases of no and weak control, and it is nonconvex in the cases of strong and full control. (See Figure 1.) We summarize this as follows:

[TABLE]

Remark 2.5*.*

Observe that

• (i)

$\log\cosh(c)$ is equal to the depth of the wells in the graph of $K_{\delta}(\theta)$ against $\theta$, and

• (ii)

$\beta=\sup\{\beta V(T_{x}\omega)-\beta V(T_{y}\omega):\,x,y\in\mathbb{Z}\}$ for $\mathbb{P}$-a.e. $\omega$ (by (1.4)).

Therefore, the first equivalence in (2.12) is a purely geometric characterization of the convexity of the effective Hamiltonian in terms of the original Hamiltonian.

2.4. Some previous results on the homogenization of HJ equations

Recall from Section 1.4 that the existence of the tilted free energy was already shown for a general class of RWs in random potentials on $\mathbb{Z}^{d}$ with any $d\geq 1$ (see [30, 13, 29, 23] and also Remark A.2). In light of Theorem 1.4, this existence result can be seen as “pointwise homogenization” at $(t,x)=(1,0)$ for a second-order HJ stochastic partial difference equation with linear initial data, where the Hamiltonian is given by the tilted free energy and hence convex in $\theta$. It is not hard to improve the pointwise convergence at $(t,x)=(1,0)$ to uniform convergence on compact subsets of $[0,\infty)\times\mathbb{R}$ (as we do so in Theorems 1.4 and 2.4 in the one-dimensional case with no control). To the best of our knowledge, there are no other previous results on the homogenization of second-order HJ stochastic partial difference equations.

There is a rich literature on the continuous analog of our discrete setting with no control and its suitable generalizations. Sznitman’s work [27] on large deviations for Brownian motion in a Poissonian potential on $\mathbb{R}^{d}$ employs the subadditive ergodic theorem and gives the first example of “pointwise homogenization” of a second-order HJ stochastic partial differential equation (PDE) with linear initial data, where the Hamiltonian is quadratic (and hence convex) in $\theta$. Homogenization of second-order HJ stochastic PDEs (with general uniformly continuous initial data which is what is meant by default) was later established in [21] (using the subadditive ergodic theorem) and independently in [18] (using the ergodic and minimax theorems) for wide classes of Hamiltonians that are convex in $\theta$. In fact, as we mention in Section 1.4 and Remark A.2, the existence of the tilted free energy for RWs in random potentials on $\mathbb{Z}^{d}$ was shown in [30, 13] and then [29, 23] by building upon the ideas in [27] and [18], respectively. For further details and references on the homogenization of (first- and second-order) HJ stochastic PDEs with convex Hamiltonians, see [17].

There are also several works that prove homogenization for certain HJ stochastic PDEs with nonconvex Hamiltonians in arbitrary dimensions. In the second-order case (which is relevant to our setting), the work of Fehrman [12] covers a class of “level-set convex” Hamiltonians, whereas Armstrong and Cardaliaguet [2] consider Hamiltonians that satisfy a finite range of dependence condition and are homogeneous in $\theta$. The Hamiltonian $H_{\delta,\beta}(\theta,x,\omega)$ in our setting (which is given in (2.9) and is nonconvex in $\theta$ when $\delta\in(0,1]$) satisfies none of these conditions.

In the first-order case, Armstrong, Tran and Yu [3] prove homogenization for a HJ stochastic PDE in arbitrary dimensions, where the Hamiltonian is of the form $H(\theta)+V(T_{x}\omega)$ with the specific choice $H(\theta)=(|\theta|^{2}-1)^{2}$. In a subsequent work [4], the same authors extend this result to any coercive $H(\theta)$ in one dimension. They also notice the relationship between (i) the convexity of the effective Hamiltonian $\overline{H}(\theta)$ and (ii) the size of the oscillations of $V(T_{x}\omega)$ in comparison to the depth of the wells of $H(\theta)$ (which is similar to Remark 2.5). Moreover, they give an implicit formula for $\overline{H}(\theta)$ under additional assumptions (see [4, Lemma 5.2]). The proofs in [4] rely on the existence of sublinear correctors in one dimension which is parallel to our approach (see Section 3 for a summary of our proofs), but are otherwise quite different since they use (first-order) nonlinear PDE techniques. The main homogenization result in [4] is extended by Gao [14] to general (i.e., not necessarily separable) coercive Hamiltonians in one dimension.

In a recent paper, Davini and Kosygina [9] consider first- and second-order HJ stochastic PDEs in arbitrary dimensions. Using a variant of the perturbed test function method which is originally due to Evans [11], they prove that “pointwise homogenization” at $(t,x)=(1,0)$ with linear initial data in fact implies homogenization with general uniformly continuous initial data. We expect that this result can be adapted to our discrete setting, too, and in particular extend Theorem 2.4 to uniformly continuous initial data. However, we did not pursue this direction since our starting point is controlled RWs in random potential for which the corresponding initial data is linear.

As an application of their main result in [9], Davini and Kosygina show homogenization for nonconvex Hamiltonians of the following form in one dimension: there exist finitely many $\theta_{1},\ldots,\theta_{n}$ such that the Hamiltonian is constant at these values and it is convex in $\theta$ on each of the intervals $(-\infty,\theta_{1})$, $(\theta_{1},\theta_{2})$, …, $(\theta_{n-1},\theta_{n})$, $(\theta_{n},+\infty)$. Due to the random additive term $\beta V(T_{x}\omega)$ in (2.9), the Hamiltonian $H_{\delta,\beta}(\theta,x,\omega)$ in our setting does not have this form.

Finally, Ziliotto [31] proves, by giving a counterexample, that first-order HJ stochastic PDEs do not always homogenize. His counterexample comes from a zero-sum differential game in two dimensions. The Hamiltonian is coercive, Lipschitz continuous and (of course) nonconvex in $\theta$. The environment is stationary and ergodic (in fact, slowly mixing). Even though there are currently no such counterexamples in the second-order case, Ziliotto’s work suggests that one cannot prove homogenization results by purely qualitative arguments based on subadditivity when the Hamiltonian is nonconvex, and some kind of constructive approach (such as ours in this paper) is needed.

3. Summary of the proofs

In order to convey the essence and strategy of the proofs of Theorems 1.4, 2.1, 2.2 and 2.3 to the reader at a relatively early stage in the paper, we provide here an overview without giving full details, proper justifications or references (which can all be found in the subsequent sections).

3.1. No control

Similar to $u_{o}(t,x\,|\,\delta,\beta,\theta)$ in (1.2) with $\delta=0$, we define $\Lambda_{\beta}^{L}(\theta,t,x)$ and $\Lambda_{\beta}^{U}(\theta,t,x)$ in (4.1) and (4.2) but via $\liminf$ and $\limsup$, respectively.

For every $h\in(0,1)$ and $\ell\in\mathbb{N}$, there is an $h$-hill of the form $[x^{*}-\ell,x^{*}+\ell-1]$ that is suitably close to the starting point of the RW. The distance is controlled by a small parameter $a>0$. We consider the event that the particle marches deterministically to $x^{*}$ and then spends the rest of the time in this $h$-hill, which gives the lower bound

[TABLE]

after taking $a\to 0$, $h\to 1$ and $\ell\to\infty$.

If $\Lambda_{\beta}^{U}(\theta,t,x)=t\beta+\theta x$, then

[TABLE]

and we are done. Otherwise, we construct a bounded and centered cocycle $F_{\beta,\theta}:\Omega\times\{-1,1\}\to\mathbb{R}$ (referred to as the corrector) that satisfies

[TABLE]

for some $\lambda>\beta$. The sums $\sum F_{\beta,\theta}(T_{x_{i}}\omega,z_{i+1})$ over nearest-neighbor paths are uniformly sublinear in the number of steps. We use these sublinear path sums to modify the exponential expectations on the right-hand sides of (4.1) and (4.2) without changing the values of $\Lambda_{\beta}^{L}(\theta,t,x)$ and $\Lambda_{\beta}^{U}(\theta,t,x)$. After this modification, it follows from a repeated application of (3.2) that

[TABLE]

This completes the proof of Theorem 1.4. We also deduce that $\Lambda_{\beta}(\theta)\geq\beta$.

3.2. Full control

Similar to $u_{o}(t,x\,|\,\delta,\beta,\theta)$ in (1.2) with $\delta\in(0,1]$, we define $\overline{H}_{\delta,\beta}^{U}(\theta,t,x)$ and $\overline{H}_{\delta,\beta}^{L}(\theta,t,x)$ in (5.1) and (5.2) but via $\limsup$ and $\liminf$, respectively.

For every $h\in(0,1)$, there is an $h$-valley of the form $[x_{*}-1,x_{*}]$ that is suitably close to the starting point of the RW. The distance is controlled by $a>0$ as in Section 3.1. Under the policy $\pi^{(x_{*},h,1)}$ (given in (2.5)), the particle marches deterministically to $x_{*}$ and is then confined to $[x_{*}-1,x_{*}]$ for the rest of the time, which gives the upper bound

[TABLE]

after taking $a\to 0$ and $h\to 0$. On the other hand, the particle marches deterministically to the left and to the right under the policies $\overleftarrow{\pi}$ and $\overrightarrow{\pi}$, respectively (see (2.6)), which gives the upper bound

[TABLE]

by the Birkhoff ergodic theorem.

The upper bound in (3.4) is at least as good as the one in (3.3) when $\theta\geq\beta\mathbb{E}[V(\cdot)]$. In this case, we introduce a bounded and centered cocycle $G_{\beta}:\Omega\times\{-1,1\}\to\mathbb{R}$ (analogous to $F_{\beta,\theta}$ in Section 3.1 but simpler) that satisfies

[TABLE]

for every $p\in[0,1]$. The sums $\sum G_{\beta}(T_{x_{i}}\omega,z_{i+1})$ over nearest-neighbor paths are uniformly sublinear in the number of steps. We use these sublinear path sums to modify the exponential expectation on the right-hand side of (5.2) without changing the value of $\overline{H}_{1,\beta}^{L}(\theta,t,x)$. Then, it follows from the repeated application of (3.5) that $\overleftarrow{\pi}$ is optimal and

[TABLE]

This lower bound matches the upper bound in (3.4) when $\theta\geq\beta\mathbb{E}[V(\cdot)]$.

When $0<\theta<\beta\mathbb{E}[V(\cdot)]$, we introduce $\bar{\beta}=\bar{\beta}(\theta):=\frac{\theta}{\mathbb{E}[V(\cdot)]}<\beta$ and notice that

[TABLE]

where the second inequality follows from (3.6) since $\theta=\bar{\beta}\mathbb{E}[V(\cdot)]$. This lower bound matches the upper bound in (3.3). The last two lower bounds are adapted to the $\theta<0$ case by symmetry. The $\theta=0$ case is easy. This completes the proof of Theorem 2.1.

3.3. Partial control: Upper bounds

The infima in the definitions of $\overline{H}_{\delta,\beta}^{U}(\theta,t,x)$ and $\overline{H}_{\delta,\beta}^{L}(\theta,t,x)$ (see (5.1) and (5.2)) can be restricted to the set of bang-bang policies which take the values

[TABLE]

with the parameter $c$ introduced in (2.2). We use (3.8) to perform a change of measure and express $\overline{H}_{\delta,\beta}^{U}(\theta,t,x)$ and $\overline{H}_{\delta,\beta}^{L}(\theta,t,x)$ in terms of expectation with respect to SSRW (see (6.3) and (6.4)). This gives an alternative formulation of our control problem where the policies are now exponential tilts denoted by $\alpha$ and taking the values $\pm c$.

In this alternative formulation, the policies $\overleftarrow{\pi}$ and $\overrightarrow{\pi}$ (see (2.6)) correspond to $\overleftarrow{\alpha}$ and $\overrightarrow{\alpha}$ that are identically equal to $-c$ and $c$, respectively. Therefore, Theorem 1.4 gives the upper bound

[TABLE]

For every $h\in(0,1)$ and $\ell\in\mathbb{N}$, there is an $h$-valley of the form $[x_{*}-\ell,x_{*}+\ell-1]$ that is suitably close to the starting point of the RW, where the distance is controlled by $a>0$ as in Section 3.1. The policy $\pi^{(x_{*},h,\ell)}$ (see (2.5)) corresponds to $\alpha^{(x_{*},h,\ell)}$ that is equal to $c$ at points to the left of $x_{*}$ and equal to $-c$ elsewhere. When $\theta=c$, the combined tilt (of $\theta$ and the control) is $2c$ at points to the left of $x_{*}$ and zero elsewhere, which gives a simple upper bound for $\overline{H}_{\delta,\beta}^{U}(c,t,x)$ (see (6.6)). We dominate this upper bound using an exponential expectation involving the number of complete left excursions of a reflected RW on $[x_{*}-\ell,x_{*}+\ell-1]$ and show that

[TABLE]

This argument can be adapted to the $\theta=-c$ case. Finally, we use convexity to obtain the upper bound

[TABLE]

Observe that, in the weak control regime ($\beta\geq\log\cosh(c)$), the upper bound in (3.10) is at least as good as the one in (3.9) since, by Proposition 4.8, $\Lambda_{\beta}(\theta\pm c)-\log\cosh(c)\geq\beta-\log\cosh(c)=[\beta-\log\cosh(c)]^{+}$. On the other hand, there is no such uniform (in $\theta\in[-c,c]$) comparison in the strong control regime ($\beta<\log\cosh(c)$).

3.4. Partial control: Lower bounds

In the alternative formulation we mentioned in Section 3.3, $\overline{H}_{\delta,\beta}^{L}(\theta,t,x)$ is expressed in terms of an exponential expectation with respect to SSRW (see (6.4)). Observe that the combined tilt (of $\theta$ and the control) in this expectation defines a martingale. Therefore, we can ignore its contribution at a small exponential cost by the Azuma-Hoeffding inequality, use (3.1) which is now applicable, and deduce that

[TABLE]

This lower bound matches the upper bound in (3.10) in the weak control regime ($\beta\geq\log\cosh(c)$) when $\theta\in[-c,c]$, and it also matches the upper bound in (3.9) when $\Lambda_{\beta}(|\theta|-c)=\beta$ (regardless of weak or strong control).

When $\theta\geq 0$ and $\Lambda_{\beta}(\theta-c)>\beta$, we define

[TABLE]

where $F_{\beta,\theta-c}$ is the corrector we mentioned in Section 3.1. Under the extra assumption that $\theta>c$, we show that

[TABLE]

Then, analogous to Sections 3.1 and 3.2, we use the sublinear path sums $\sum F_{\beta,\theta-c}(T_{x_{i}}\omega,z_{i+1})$ to modify the exponential expectation on the right-hand side of (6.4) without changing the value of $\overline{H}_{\delta,\beta}^{L}(\theta,t,x)$. By repeated application of (3.11), we deduce that $\overleftarrow{\alpha}$ (i.e., $\overleftarrow{\pi}$ in the original formulation) is optimal and

[TABLE]

This lower bound matches the upper bound in (3.9). It is adapted to the $\theta<-c$ case by symmetry. This completes the proof of Theorem 2.2 (weak control).

It remains to obtain good lower bounds in the strong control regime ($\beta<\log\cosh(c)$) when $\theta\in(-c,c)$ and $\min\{\Lambda_{\beta}(\theta-c),\Lambda_{\beta}(\theta+c)\}>\beta$. (We know that $\Lambda_{\beta}(0)=\beta$.) The upper bound in (3.9) is at least as good as the one in (3.10) when $0<\theta<c$ and $\beta<\Lambda_{\beta}(\theta-c)\leq\log\cosh(c)$. In this case, we show that (3.11) continues to hold and it implies (by the same argument) the lower bound in (3.12), which matches the upper bound in (3.9).

When $0<\theta<c$ and $\beta<\log\cosh(c)<\Lambda_{\beta}(\theta-c)$, there exists a unique $\bar{\theta}(\beta,c)\in(0,c)$ such that

[TABLE]

and a unique $\bar{\beta}=\bar{\beta}(\theta,c)<\beta$ such that $\theta=\bar{\theta}(\bar{\beta},c)$. Analogous to (3.7) in Section 3.2,

[TABLE]

by (3.12) since $\bar{\beta}<\Lambda_{\bar{\beta}}(\theta-c)=\log\cosh(c)$. This lower bound matches the upper bound in (3.10). The last two lower bounds are adapted to the $-c<\theta<0$ case by symmetry. The $\theta=0$ case is easy. This completes the proof of Theorem 2.3 (strong control).

4. No control

4.1. The tilted free energy

In this section, we provide a self-contained proof of Theorem 1.4 (see Section 1.4 for references to the literature on the existence of the tilted free energy). While doing so, we obtain some intermediate results which will be central to the proofs of Theorems 2.1, 2.2, 2.3 and 2.4.

We start by recalling an elementary result regarding SSRW.

Lemma 4.1**.**

For every $\ell\in\mathbb{N}$,

[TABLE]

Proof.

This follows immediately from the eigenvalues and eigenvectors of the adjacency matrix of SSRW on $[-\ell,\ell-1]$ with absorbing boundary conditions (see [26, p. 239]). ∎

Assume that (1.4) and (1.5) hold. For every $\beta>0$, $\theta\in\mathbb{R}$, $t>0$ and $x\in\mathbb{R}$, let

[TABLE]

Strictly speaking, we should write $\Lambda_{\beta}^{L}(\theta,t,x,\omega)$ and $\Lambda_{\beta}^{U}(\theta,t,x,\omega)$ to indicate the dependence on $\omega$, too. However, it is clear from the ellipticity of SSRW that $\Lambda_{\beta}^{L}(\theta,t,x,\omega)=\Lambda_{\beta}^{L}(\theta,t,x,T_{1}\omega)$, and therefore $\Lambda_{\beta}^{L}(\theta,t,x)$ is $\mathbb{P}$-a.s. constant by the ergodicity assumption. The same reasoning applies to $\Lambda_{\beta}^{U}(\theta,t,x)$. For the purpose of proving the existence of the tilted free energy $\Lambda_{\beta}(\theta)$ (see (1.6)), it suffices to take $t=1$ and $x=0$. The latter applies to the following lemma, too.

Lemma 4.2**.**

For every $h\in(0,1)$, $\ell\in\mathbb{N}$, $a>0$, $B>0$ and $\mathbb{P}$-a.e. $\omega$, there exists an $n_{0}=n_{0}(\omega,h,\ell,a,B)$ such that the interval $[n(x-a),n(x+a)]$ contains an $h$-valley (resp. $h$-hill) of the form $[x_{*}-\ell,x_{*}+\ell-1]$ (resp. $[x^{*}-\ell,x^{*}+\ell-1]$) for every $x\in[-B,B]$ and $n\geq n_{0}$.

Proof.

Without loss of generality, we may and will assume that $a/B\leq 1$. For every $h\in(0,1)$, $k,\ell\in\mathbb{N}$ and $\mathbb{P}$-a.e. $\omega$, the number of $h$-valleys of length $2\ell-1$ contained in the interval $[0,k]$ (resp. the interval $[-k,0]$) is $kp_{h,\ell}+o(k)$ by the Birkhoff ergodic theorem, where

[TABLE]

by (1.5). Therefore, for $j\in[-\frac{B}{a},\frac{B}{a}-1]\cap\mathbb{Z}$, the number of $h$-valleys of length $2\ell-1$ contained in the interval $[0,nja]$ is $n|j|ap_{h,\ell}(1\pm a/4B)$, for all $n\geq n_{1}(\omega,h,\ell,a,B)$. It follows that for such $n$, the number of $h$-valleys of length $2\ell-1$ contained in $[nja,n(j+1)a]$ is at least $\frac{1}{2}nap_{h,\ell}$. In particular, it is positive for $n\geq n_{0}(\omega,h,\ell,a,B)$. For every $x\in[-B,B]$, the interval $[n(x-a),n(x+a)]$ contains $[nja,n(j+1)a]$ for at least one such $j$, and the desired result follows. The same argument applies to $h$-hills of length $2\ell-1$. ∎

Lemma 4.3**.**

$\Lambda_{\beta}^{L}(\theta,t,x)\geq t\beta+\theta x$* for every $\beta>0$, $\theta\in\mathbb{R}$, $t>0$ and $x\in\mathbb{R}$.*

Proof.

For every $h\in(0,1)$, $\ell\in\mathbb{N}$, $t>0$, $x\in\mathbb{R}$, $a\in(0,t)$, $\mathbb{P}$-a.e. $\omega$ and sufficiently small $\epsilon>0$, Lemma 4.2 implies the existence of an $h$-hill of the form $[x^{*}-\ell,x^{*}+\ell-1]$ that is contained in the interval $[\epsilon^{-1}(x-a),\epsilon^{-1}(x+a)]$. Let $A_{[\epsilon^{-1}t]}(x^{*},h,\ell)$ be the event that the particle marches deterministically from $[\epsilon^{-1}x]$ to $x^{*}$ and then spends the rest of the $[\epsilon^{-1}t]$ units of time in this $h$-hill. Restricting on this event and applying Lemma 4.1, we get the following lower bound:

[TABLE]

The desired result is obtained by first taking $\epsilon\log$ of both sides, then sending $\epsilon\to 0$, and finally taking $a\to 0$, $h\to 1$ and $\ell\to\infty$. ∎

Let $\tau_{k}=\inf\{i\geq 0:X_{i}=k\}$ denote the first time the particle is at $k\in\mathbb{Z}$. For every $\lambda>\beta$, define

[TABLE]

Note that

[TABLE]

Set $F_{\beta,\theta}^{\lambda}(\omega,-1)=-F_{\beta,\theta}^{\lambda}(T_{-1}\omega,1)$. Then, decomposing the expectation corresponding to $e^{-F_{\beta,\theta}^{\lambda}(\omega,1)}$ with respect to the first step of the RW, we see that

[TABLE]

for every $\omega\in\Omega$.

Lemma 4.4**.**

If $\theta\geq 0$ and $\beta<\lambda<t^{-1}\left(\Lambda_{\beta}^{U}(\theta,t,x)-\theta x\right)$ for some $t>0$ and $x\in\mathbb{R}$, then

[TABLE]

Proof.

Fix $\theta\geq 0$ and $\beta<\lambda<t^{-1}\left(\Lambda_{\beta}^{U}(\theta,t,x)-\theta x\right)$ for some $t>0$ and $x\in\mathbb{R}$. Then, for $\mathbb{P}$-a.e. $\omega$, there exists a subsequence $\epsilon_{k}\to 0$ and an error $o(\epsilon_{k}^{-1})$, both possibly depending on $\beta$, $\theta$, $t$, $x$ and $\omega$, so that

[TABLE]

On the other hand, for any $\epsilon>0$,

[TABLE]

We now claim that for any $\eta>0$ there is an $\epsilon_{0}=\epsilon_{0}(\omega,\beta,\theta,\lambda,\eta)$ so that if $\epsilon\leq\epsilon_{0}$ then for all $0\leq k\leq[\epsilon^{-1}t]-1$,

[TABLE]

Combining (4.6)–(4.8), we conclude that $0<\Lambda_{\beta}^{U}(\theta,t,x)-t\lambda-\theta x\leq-t\mathbb{E}[F_{\beta,\theta}^{\lambda}(\cdot,1)]$, which completes the proof of the lemma.

It remains to prove (4.8). Note that the Birkhoff ergodic theorem gives an $n_{0}=n_{0}(\omega,\beta,\theta,\lambda,\eta)$ such that $n\geq n_{0}$ implies

[TABLE]

Hence, for $0\leq k\leq[\epsilon^{-1}t]-1$,

[TABLE]

whenever $|[\epsilon^{-1}x]|\geq n_{0}$ and $|[\epsilon^{-1}x]+k|\geq n_{0}$. Otherwise, we can shift the indices of the sum in (4.9) by $2n_{0}$, recall (4.4) and use the triangle inequality to deduce that the left side of (4.9) is bounded by $\epsilon^{-1}\eta(t+2|x|)+4n_{0}\|F^{\lambda}_{\beta,\theta}(\cdot,1)\|_{\infty}$. Choosing $\epsilon_{0}<\eta/(4n_{0}\|F^{\lambda}_{\beta,\theta}(\cdot,1)\|_{\infty})$ then gives (4.8). ∎

Lemma 4.5**.**

If $\theta\geq 0$, then the map $\lambda\mapsto\mathbb{E}[F_{\beta,\theta}^{\lambda}(\cdot,1)]$ is continuous and strictly increasing for $\lambda>\beta$. Moreover,

[TABLE]

Proof.

Note that $F_{\beta,\theta}^{\lambda+\Delta\lambda}(\omega,1)\geq F_{\beta,\theta}^{\lambda}(\omega,1)+\Delta\lambda$, since $\tau_{1}\geq 1$. The rest follows from the uniform (in $\omega$) bounds in (4.4) and the dominated convergence theorem. ∎

Proof of Theorem 1.4.

Assume without loss of generality that $\theta\geq 0$.

If $\Lambda_{\beta}^{U}(\theta,t,x)=t\beta+\theta x$ for every $t>0$ and $x\in\mathbb{R}$, then Lemma 4.3 gives

[TABLE]

If $\Lambda_{\beta}^{U}(\theta,t^{\prime},x^{\prime})>t^{\prime}\beta+\theta x^{\prime}$ for some $t^{\prime}>0$ and $x^{\prime}\in\mathbb{R}$, then Lemmas 4.4 and 4.5 (and the intermediate value theorem) imply the existence of a unique $\lambda\geq(t^{\prime})^{-1}\left(\Lambda_{\beta}^{U}(\theta,t^{\prime},x^{\prime})-\theta x^{\prime}\right)$ such that $\mathbb{E}[F_{\beta,\theta}^{\lambda}(\cdot,1)]=0$. Then, $F_{\beta,\theta}^{\lambda}$ is a bounded and centered cocycle (see Definition B.1 in Appendix B). Therefore, for every $t>0$, $x\in\mathbb{R}$ and $\mathbb{P}$-a.e. $\omega$,

[TABLE]

Here, the first equality follows from the uniformly sublinear (in $[\epsilon^{-1}t]$) growth of sums (over nearest-neighbor paths of length $[\epsilon^{-1}t]$) of bounded and centered cocycles (see Lemma B.2) and the last equality is obtained by the repeated application of (4.5). Taking $\epsilon\log$ of both sides and sending $\epsilon\to 0$, we conclude that

[TABLE]

The existence of the limit in (1.2) and the validity of the identity in (1.3) follow immediately from (4.10) and (4.11). Finally, setting $t=1$ and $x=0$, we deduce (1.6). ∎

4.2. The corrector and an implicit formula

When $\theta>0$ and $\Lambda_{\beta}(\theta)>\beta$, we will henceforth write $F_{\beta,\theta}=F_{\beta,\theta}^{\Lambda_{\beta}(\theta)}$ to simplify the notation in the previous section. We extend this definition to the $\theta<0$ case and recapitulate it as follows:

[TABLE]

Note that this definition leaves out $\theta=0$ because $\Lambda_{\beta}(0)=\beta$ (see Proposition 4.8(c)). We record the following results for future reference.

Proposition 4.6**.**

Assume (1.4), (1.5), and that $\theta$ is such that $\Lambda_{\beta}(\theta)>\beta$. Then $F_{\beta,\theta}:\Omega\times\{-1,1\}\to\mathbb{R}$, defined in (4.12), satisfies

[TABLE]

i.e., it is a centered cocycle (see Definition B.1 in Appendix B). Moreover, for every $\omega\in\Omega$,

[TABLE]

Proof.

The equality (4.13) follows from the definition of $\Lambda_{\beta}(\theta)$, building on the proof of Theorem 1.4. When $\theta>0$, the desired results (4.14) and (4.15) have been shown in (4.4) and (4.5), respectively. When $\theta<0$, the proofs are identical since the law of the underlying SSRW is symmetric. ∎

In light of the exact equality in (4.15), $F_{\beta,\theta}$ is referred to as the corrector. We will say more about this choice of terminology in Appendix C (see Remark C.2) where we present and analyze two variational formulas for $\Lambda_{\beta}(\theta)$. The following result gives an implicit (non-variational) formula for $\Lambda_{\beta}(\theta)$.

Proposition 4.7**.**

Assume (1.4) and (1.5). Then, $\Lambda_{\beta}(\theta)\geq\beta$ for every $\theta\in\mathbb{R}$. Moreover,

[TABLE]

whenever $\Lambda_{\beta}(\theta)>\beta$.

Proof.

We already know from Lemma 4.3 (with $t=1$ and $x=0$) that $\Lambda_{\beta}(\theta)\geq\beta$. The symmetry of the law of SSRW implies that $\Lambda_{\beta}(\theta)$ is even in $\theta$. (We will list various properties of the tilted free energy in Proposition 4.8 below.) Therefore, (4.16) follows from (4.12) and (4.13) whenever $\Lambda_{\beta}(\theta)>\beta$. ∎

4.3. Some properties of the tilted free energy

Proposition 4.8**.**

Assume (1.4) and (1.5). Then, the following hold.

• (a)

$\Lambda_{\beta}(\theta)$* is increasing in $\beta$, and even and convex in $\theta$.*

• (b)

$\Lambda_{\beta}(\theta)\geq\max\left\{\beta,\beta\mathbb{E}[V(\cdot)]+\log\cosh(\theta)\right\}$* for every $\theta\in\mathbb{R}$.*

• (c)

If $|\theta|\leq\beta(1-\mathbb{E}[V(\cdot)])$, then $\Lambda_{\beta}(\theta)=\beta$. Hence, the set $\{\theta\in\mathbb{R}:\,\Lambda_{\beta}(\theta)=\beta\}$ is a symmetric and closed interval with nonempty interior.

• (d)

$\Lambda_{\beta}(\theta)-\log\cosh(\theta)\to\beta\mathbb{E}[V(\cdot)]$* as $|\theta|\to\infty$.*

• (e)

The map $\theta\mapsto\Lambda_{\beta}(\theta)$ is continuously differentiable on the complement of $\{\theta\in\mathbb{R}:\,\Lambda_{\beta}(\theta)=\beta\}$.

Proof.

• (a)

These three properties follow from $V(\cdot)\geq 0$, the symmetry of the law of SSRW and a standard application of Hölder’s inequality, respectively.

• (b)

Consider the nearest-neighbor RW with probability $p(\theta)=e^{\theta}/(e^{\theta}+e^{-\theta})$ of jumping to the right. It induces a probability measure $\hat{P}_{0}^{\theta}$ on paths starting at [math]. Let $\hat{E}_{0}^{\theta}$ denote expectation under $\hat{P}_{0}^{\theta}$. Note that

[TABLE]

for every bounded and $\mathcal{F}$-measurable function $f:\Omega\to\mathbb{R}$. In other words, under $\hat{P}_{0}^{\theta}$, the probability measure $\mathbb{P}$ is invariant for the so-called environment Markov chain $(T_{X_{i}}\omega)_{i\geq 0}$, and hence ergodic (with respect to temporal shifts) by Kozlov’s lemma (see [19] for details). Therefore,

[TABLE]

by Jensen’s inequality, the Birkhoff ergodic theorem and the bounded convergence theorem. Recalling the first part of Proposition 4.7, we get the desired lower bound.

• (c)

If $|\theta|\leq\beta(1-\mathbb{E}[V(\cdot)])$, then, with the $o(\cdot)$ notation denoting error terms that may depend on $\omega$,

[TABLE]

for $\mathbb{P}$-a.e. $\omega$ by the Birkhoff ergodic theorem and the observation that the particle visits each $x$ between [math] and $X_{n}$ at least once. Therefore, $\Lambda_{\beta}(\theta)\leq\beta$ and one concludes by appealing to part (b).

• (d)

Similar to part (c), if $|\theta|\geq\beta(1-\mathbb{E}[V(\cdot)])$, then

[TABLE]

for $\mathbb{P}$-a.e. $\omega$. Therefore,

[TABLE]

by part (b), and the desired result follows.

• (e)

Recall from (4.3) that

[TABLE]

for every $\omega\in\Omega$, $\theta>0$ and $\lambda>\beta$. Since $0\leq V(\cdot)\leq 1$, it follows from an application of the dominated convergence theorem (DCT) that the map $\lambda\mapsto F_{\beta,\theta}^{\lambda}(\omega,1)$ is differentiable. By a second application of the DCT, we deduce that the map $\lambda\mapsto\mathbb{E}[F_{\beta,\theta}^{\lambda}(\cdot,1)]$ is differentiable and

[TABLE]

Resorting to the DCT for a third time, we see that $\lambda\mapsto\mathbb{E}[F_{\beta,\theta}^{\lambda}(\cdot,1)]$ is in fact continuously differentiable. The map $\theta\mapsto\mathbb{E}[F_{\beta,\theta}^{\lambda}(\cdot,1)]$ is linear, and hence continuously differentiable, too. Recall from (4.13) that

[TABLE]

whenever $\theta>0$ and $\Lambda_{\beta}(\theta)>\beta$. Thus, by the implicit function theorem, the map $\theta\mapsto\Lambda_{\beta}(\theta)$ is continuously differentiable on the set $\{\theta\in\mathbb{R}:\,\theta>0\ \text{and}\ \Lambda_{\beta}(\theta)>\beta\}$. Since $\Lambda_{\beta}(-\theta)=\Lambda_{\beta}(\theta)$ and $\Lambda_{\beta}(0)=\beta$ by parts (a) and (c), this concludes the proof.∎

Proposition 4.8 does not answer the question of whether $\theta\mapsto\Lambda_{\beta}(\theta)$ is differentiable at the endpoints of the interval $\{\theta\in\mathbb{R}:\,\Lambda_{\beta}(\theta)=\beta\}$. We provide a negative answer to this question in Appendix D under a very mild additional assumption on the potential (see Theorem D.3 for the precise statement). This nondifferentiability is reflected in the sketches in Figure 1, but it is not actually used anywhere in the paper.

5. Full control

For every $\delta\in(0,1]$, $\beta>0$, $\theta\in\mathbb{R}$, $t\geq 0$ and $x\in\mathbb{R}$, let

[TABLE]

In this section, we assume that (1.4) and (1.5) hold, take $\delta=1$, provide matching upper and lower bounds for (5.1) and (5.2), respectively, and prove Theorem 2.1. In fact, we go beyond Theorem 2.1 and obtain error bounds for the limit in (1.2) which will be used in the proof of Theorem 2.4.

5.1. Upper bounds

For every $h\in(0,1)$, $t\geq 0$, $x\in\mathbb{R}$, $a>0$, $\mathbb{P}$-a.e. $\omega$ and sufficiently small $\epsilon>0$, Lemma 4.2 implies the existence of an $h$-valley of the form $[x_{*}-1,x_{*}]$ that is contained in the interval $[\epsilon^{-1}(x-a),\epsilon^{-1}(x+a)]$. Consider the policy $\pi^{(x_{*},h,1)}$ (given in (2.5)) with this specific choice of $x_{*}$ (see Remark 5.1). Under this policy, the particle marches deterministically to $x_{*}$ and is then confined to the $h$-valley $[x_{*}-1,x_{*}]$ for the rest of the $[\epsilon^{-1}t]$ units of time (if it gets to $x_{*}$). This gives the following bound:

[TABLE]

Sending $\epsilon\to 0$, $h\to 0$ and $a\to 0$ in this order, we deduce that

[TABLE]

Remark 5.1*.*

In Section 2.2, we introduced the RW policy $\pi^{(x_{*},h,\ell)}$ using an $h$-valley of the form $[x_{*}-\ell,x_{*}+\ell-1]$. When the walk starts at the origin (e.g., in (5.1) with $x=0$), we can work with a fixed $x_{*}=x_{*}(\omega,h,\ell)$ for all sufficiently small $\epsilon>0$. However, when the walk starts at $[\epsilon^{-1}x]$ with some $x\neq 0$, we need to take $x_{*}=x_{*}(\omega,h,\ell,x,a,\epsilon)$ as in Lemma 4.2. In particular, the policy $\pi^{(x_{*},h,\ell)}$ depends on $\epsilon$ in the latter case.

When $\theta\geq 0$, consider the policy $\overleftarrow{\pi}$ (given in (2.6)) under which the particle marches deterministically to the left for $[\epsilon^{-1}t]$ units of time.

[TABLE]

By the Birkhoff ergodic theorem, we deduce the following bound: for $\mathbb{P}$-a.e. $\omega$,

[TABLE]

5.2. Lower bounds when $\theta\geq 0$

5.2.1. Lower bound when $\theta\geq\beta\mathbb{E}[V(\cdot)]$

Define $G_{\beta}:\Omega\times\{-1,1\}\to\mathbb{R}$ by

[TABLE]

Then, $\mathbb{E}[G_{\beta}(\cdot,\pm 1)]=0$, and $G_{\beta}$ is a bounded and centered cocycle (see Definition B.1 in Appendix B). Analogous to $F_{\beta,\theta}$ (see Proposition 4.6) in the case of no control, $G_{\beta}$ will serve as the corrector in the case of full control.

For every $p\in[0,1]$, let

[TABLE]

Lemma 5.2**.**

If $\theta\geq\beta\mathbb{E}[V(\cdot)]$, then

[TABLE]

for every $p\in[0,1]$ and $\omega\in\Omega$.

Proof.

Since $V(\cdot)\geq 0$, we have

[TABLE]

Therefore, $\beta V(\omega)+\theta+G_{\beta}(\omega,1)\geq\beta V(\omega)-\theta+G_{\beta}(\omega,-1)$, and the inequality in (5.8) follows. The equality in (5.8) follows from direct substitution. ∎

For every $t\geq 0$, $x\in\mathbb{R}$ and $\pi\in\mathcal{P}_{[\epsilon^{-1}t]}(1)$, use Lemma B.2 to give the following bound, where the $o([\epsilon^{-1}t])$ error terms depend on $(\omega,t,x,\beta)$ but not on $\pi$ :

[TABLE]

and the last inequality used Lemma 5.2. Iterating, one obtains

[TABLE]

First taking $\epsilon\log$ of both sides, then taking infimum over $\pi\in\mathcal{P}_{[\epsilon^{-1}t]}(1)$, and finally sending $\epsilon\to 0$, we conclude that

[TABLE]

5.2.2. Lower bound when $0<\theta<\beta\mathbb{E}[V(\cdot)]$

We use scaling properties. Let $\bar{\beta}=\bar{\beta}(\theta)=\frac{\theta}{\mathbb{E}[V(\cdot)]}<\beta$. Then,

[TABLE]

and

[TABLE]

for every $t\geq 0$ and $x\in\mathbb{R}$. Here, the first inequality uses the fact that $V(\cdot)\geq 0$, and the second inequality follows from (5.10) which is applicable since $\theta=\bar{\beta}\mathbb{E}[V(\cdot)]$.

5.2.3. Lower bound when $\theta=0$

Since $V(\cdot)\geq 0$, we have

[TABLE]

for every $\epsilon>0$, $t\geq 0$ and $x\in\mathbb{R}$. Taking $\epsilon\to 0$, we conclude that

[TABLE]

5.3. The effective Hamiltonian

Proof of Theorem 2.1.

If $0\leq\theta<\beta\mathbb{E}[V(\cdot)]$, then the bounds (5.4), (5.12) and (5.14) match for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

and taking the infimum in (1.1) over the set $\{\pi^{(x_{*},h,1)}:\,0<h<h_{0}\}$ for any $h_{0}>0$ does not change the limit in (1.2). (Regarding the choice of $x_{*}$, see Remark 5.1.)

If $\theta\geq\beta\mathbb{E}[V(\cdot)]$, then the bounds (5.6) and (5.10) match for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

and $\overleftarrow{\pi}$ is asymptotically optimal as $\epsilon\to 0$.

If $\theta<0$, the analogous results follow from symmetry. The existence of the limit in (1.2) and the validity of the identity in (1.3) follow immediately from (5.15) and (5.16). Finally, setting $t=1$ and $x=0$, we deduce (2.1). ∎

6. Partial control: Alternative formulation and upper bounds

In this section, we consider the case $\delta\in(0,1)$ under the assumptions (1.4) and (1.5).

6.1. Alternative formulation

Recall from our discussion in Section 2.3, cf. (2.7), that the infimum in (1.1) can be taken over

[TABLE]

i.e., the set of bang-bang policies. For every $\pi\in\mathcal{P}_{n}^{BB}(\delta)$, define $\alpha=(\alpha_{0},\ldots,\alpha_{n-1})$ by setting

[TABLE]

The parameter $c$ was introduced in (2.2). Note that

[TABLE]

We perform a change of measure: for every $x\in\mathbb{Z}$,

[TABLE]

Then, (1.1) becomes

[TABLE]

where the infimum is taken over

[TABLE]

Similarly, (5.1) and (5.2) become

[TABLE]

6.2. General upper bound

The policies $\overleftarrow{\pi},\overrightarrow{\pi}\in\mathcal{P}_{n}^{BB}(\delta)$ (given in (2.6)) correspond to $\overleftarrow{\alpha},\overrightarrow{\alpha}\in\mathcal{A}_{n}(c)$ with

[TABLE]

respectively. For every $\theta\in\mathbb{R}$, $t>0$ and $x\in\mathbb{R}$, substituting each of these policies (with $n=[\epsilon^{-1}t]$) in the expectation on the right-hand side of (6.3) and using Theorem 1.4, we deduce the following bound:

[TABLE]

6.3. Upper bound when $|\theta|\leq c$

For every $h\in(0,1)$, $\ell\in\mathbb{N}$, $x\in\mathbb{R}$ and $a>0$, the RW policy $\pi^{(x_{*},h,\ell)}\in\mathcal{P}_{n}^{BB}(\delta)$ (given in (2.5)) corresponds to $\alpha^{(x_{*},h,\ell)}\in\mathcal{A}_{n}(c)$ with

[TABLE]

When the walk starts at $[\epsilon^{-1}x]$ with a sufficiently small $\epsilon>0$, recall from Lemma 4.2 and Remark 5.1 that $[x_{*}-\ell,x_{*}+\ell-1]\subset[\epsilon^{-1}(x-a),\epsilon^{-1}(x+a)]$. Assume without loss of generality that $x_{*}=[\epsilon^{-1}x]$, i.e., $[[\epsilon^{-1}x]-\ell,[\epsilon^{-1}x]+\ell-1]$ is an $h$-valley. (Starting the walk from $x_{*}$ instead of $[\epsilon^{-1}x]$ changes the right-hand side of (5.1) by at most $a(\log 2+\beta+2|\theta|+2c)$, which goes to [math] as $a\to 0$.) When $\theta=c$, substituting $\alpha^{(x_{*},h,\ell)}$ in the expectation on the right-hand side of (6.3), we get

[TABLE]

where we shifted the starting point of the RW $(X_{i})_{i\geq 0}$ to the origin.

Due to each complete left excursion starting from the origin, the $\sum 2cZ_{i+1}{{\rm 1\mkern-1.5mu}\!{\rm I}}_{\{X_{i}<0\}}$ term in the exponent inside the expectation on the right-hand side of (6.6) increases precisely by $2c$, and this sum does not increase (but it can decrease) due to an incomplete left excursion. On the other hand, complete and incomplete right excursions starting from the origin have no effect on this sum. We deduce that

[TABLE]

where

[TABLE]

counts the number of complete left excursions of a nearest-neighbor path $x_{0,n}$ with $x_{0}=0$ and $n\in\mathbb{N}$.

For every $j,k\in\mathbb{N}\cup\{0\}$, let $S_{j}=\sum_{i=1}^{j}{{\rm 1\mkern-1.5mu}\!{\rm I}}_{\{-\ell\leq X_{i}\leq\ell-1\}}$, $\sigma_{k}=\inf\{i\geq 0:S_{i}=k\}$ and $Y_{k}^{\ell}=X_{\sigma_{k}}$. It is easy to see that $(Y_{k}^{\ell})_{k\geq 0}$ is a Markov process on $[-\ell,\ell-1]$ starting from the origin, and it has the following transition probabilities:

[TABLE]

In words, $(Y_{k}^{\ell})_{k\geq 0}$ is a reflected RW on $[-\ell,\ell-1]$ and subject to geometric holding times (with rate $1/2$) at $-\ell$ and $\ell-1$. With this notation and observations, we control the right-hand side of (6.7) as follows:

[TABLE]

Here,

[TABLE]

The proof of the following proposition is deferred to Appendix E.

Proposition 6.1**.**

For every $c\in(0,\infty)$, the limit

[TABLE]

exists. Moreover,

[TABLE]

Putting together (6.6), (6.7) and (6.10) (and adapting the same argument to the $\theta=-c$ case), we get

[TABLE]

Finally, for any $\theta\in[-c,c]$, let $r=\frac{\theta+c}{2c}\in[0,1]$. Then, $\theta=(1-r)(-c)+rc$ is a convex combination. For every $h\in(0,1)$ and $\ell\in\mathbb{N}$, Hölder’s inequality gives

[TABLE]

Therefore, by (6.11),

[TABLE]

7. Partial control: Lower bounds and the effective Hamiltonian

As in the previous section, we consider the case $\delta\in(0,1)$ under the assumptions (1.4) and (1.5).

7.1. Uniform lower bound

For every $\epsilon>0$, $t>0$, $x\in\mathbb{R}$, $\omega\in\Omega$, $\theta\in\mathbb{R}$ and $\alpha\in\mathcal{A}_{[\epsilon^{-1}t]}(c)$ (see (6.2)),

[TABLE]

defines a martingale $(M_{j})_{0\leq j\leq[\epsilon^{-1}t]}$ under $P_{[\epsilon^{-1}x]}$, with $|M_{j}-M_{j-1}|\leq|\theta|+c$. Therefore, for every $b>0$,

[TABLE]

by the Azuma-Hoeffding inequality (see [15, Section 12.2]).

For every $h\in(0,1)$, $\ell\in\mathbb{N}$, $a\in(0,t)$, $\mathbb{P}$-a.e. $\omega$ and sufficiently small $\epsilon>0$, we know by Lemma 4.2 that there exists an $h$-hill of the form $[x^{*}-\ell,x^{*}+\ell-1]$ contained in $[\epsilon^{-1}(x-a),\epsilon^{-1}(x+a)]$. Recall from the proof of Lemma 4.3 that $A_{[\epsilon^{-1}t]}(x^{*},h,\ell)$ is the event that the particle marches deterministically from $[\epsilon^{-1}x]$ to $x^{*}$ and then spends the rest of the $[\epsilon^{-1}t]$ units of time in this $h$-hill. It follows from Lemma 4.1 that

[TABLE]

Take $a\in(0,t)$ sufficiently small and $\ell\in\mathbb{N}$ sufficiently large (both depending on $b$) so that

[TABLE]

Then,

[TABLE]

Restricting the expectation on the right-hand side of (6.4) on this set difference gives

[TABLE]

Taking $\epsilon\log$ of both sides, then sending $\epsilon\to 0$, $a\to 0$, $h\to 1$, $\ell\to\infty$, and finally taking $b\to 0$, we get the following uniform lower bound:

[TABLE]

7.2. Lower bounds when $\theta\geq 0$ and $\Lambda_{\beta}(\theta-c)>\beta$

We begin with a preliminary computation. For every $\xi\in\mathbb{R}$, let

[TABLE]

where $F_{\beta,\theta-c}$ is the corrector defined in (4.12).

Lemma 7.1**.**

If $\Lambda_{\beta}(\theta-c)>\beta$, then

[TABLE]

for every $\omega\in\Omega$. Moreover, the following equivalence holds:

[TABLE]

Proof.

The equality in (7.2) is immediate from (4.15). The equivalence in (7.3) is shown as follows:

[TABLE]

7.2.1. Lower bound when $\theta>c$ and $\Lambda_{\beta}(\theta-c)>\beta$

For every $\omega\in\Omega$,

[TABLE]

holds by (4.14). Hence,

[TABLE]

by Lemma 7.1. Therefore, for every $\epsilon>0$, $t>0$, $x\in\mathbb{R}$, $\alpha\in\mathcal{A}_{[\epsilon^{-1}t]}(c)$ and $\mathbb{P}$-a.e. $\omega$,

[TABLE]

Here, the first equality follows from Lemma B.2 (in Appendix B). Recalling (6.4), we conclude that

[TABLE]

7.2.2. Lower bound when $0<\theta<c$ and $\beta<\Lambda_{\beta}(\theta-c)\leq\log\cosh(c)$

For every $\omega\in\Omega$,

[TABLE]

Here, the first inequality follows from (4.14), and the first equality is shown in (E.1) (in Appendix E). Therefore,

[TABLE]

and (7.4) follows from Lemma 7.1. Hence, the argument immediately below (7.4) is applicable, and

[TABLE]

for every $\epsilon>0$, $t>0$, $x\in\mathbb{R}$, $\alpha\in\mathcal{A}_{[\epsilon^{-1}t]}(c)$ and $\mathbb{P}$-a.e. $\omega$. Recalling (6.4) as before, we conclude that

[TABLE]

7.2.3. Lower bound when $0<\theta<c$ and $\beta<\log\cosh(c)<\Lambda_{\beta}(\theta-c)$

It follows from Proposition 4.8(a,b,c) and the intermediate value theorem that there exists a unique $\bar{\theta}(\beta,c)\in(0,c)$ such that

[TABLE]

By Proposition 4.8(a), the map $\beta\mapsto\bar{\theta}(\beta,c)$ is increasing for $\beta\in(0,\log\cosh(c))$, with $\bar{\theta}(0^{+},c)=0$. For every $\theta\in(0,\bar{\theta}(\beta,c))$, there is a unique $\bar{\beta}=\bar{\beta}(\theta,c)\in(0,\beta)$ such that $\theta=\bar{\theta}(\bar{\beta},c)$. Using these quantities, we get the following bound: for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

Here, the first inequality uses the fact that $V(\cdot)\geq 0$, and the second inequality follows from (7.6) which is applicable since $\bar{\beta}<\Lambda_{\bar{\beta}}(\theta-c)=\log\cosh(c)$.

7.2.4. Lower bound when $\theta=0$

Since $V(\cdot)\geq 0$, it is clear from (5.2) that, for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

7.3. The effective Hamiltonian

Proof of Theorem 2.2.

If $0\leq\theta\leq c$, then the bounds (6.12) and (7.1) match for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

and taking the infimum in (1.1) over the set $\{\pi^{(x_{*},h,\ell)}:\,0<h<h_{0},\ \ell>\ell_{0}\}$ for any $h_{0}>0$ and $\ell_{0}\in\mathbb{N}$ does not change the limit in (1.2). (Regarding the choice of $x_{*}$, see Remark 5.1.)

If $\theta\geq 0$ and $\Lambda_{\beta}(\theta-c)=\beta$, then the bounds (6.5) and (7.1) match for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

and $\overleftarrow{\pi}$ is asymptotically optimal as $\epsilon\to 0$.

If $\theta>c$ and $\Lambda_{\beta}(\theta-c)>\beta$, then the bounds (6.5) and (7.5) match for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

and $\overleftarrow{\pi}$ is asymptotically optimal as $\epsilon\to 0$.

If $\theta<0$, the analogous results follow from symmetry. The existence of the limit in (1.2) and the validity of the identity in (1.3) follow immediately from (7.9), (7.10) and (7.11). Finally, setting $t=1$ and $x=0$, we deduce (2.3). ∎

Proof of Theorem 2.3.

Recall from Section 7.2.3 that there exists a unique $\bar{\theta}(\beta,c)\in(0,c)$ such that

[TABLE]

If $0\leq\theta<\bar{\theta}(\beta,c)$, then the bounds (6.12), (7.7) and (7.8) match for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

and taking the infimum in (1.1) over the set $\{\pi^{(x_{*},h,\ell)}:\,0<h<h_{0},\ \ell>\ell_{0}\}$ for any $h_{0}>0$ and $\ell_{0}\in\mathbb{N}$ does not change the limit in (1.2). (Regarding the choice of $x_{*}$, see Remark 5.1.)

If $\theta\geq 0$ and $\Lambda_{\beta}(\theta-c)=\beta$, then the bounds (6.5) and (7.1) match for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

and $\overleftarrow{\pi}$ is asymptotically optimal as $\epsilon\to 0$.

If $\theta\geq\bar{\theta}(\beta,c)$ and $\Lambda_{\beta}(\theta-c)>\beta$, then the bounds (6.5), (7.5) and (7.6) match for every $t>0$ and $x\in\mathbb{R}$,

[TABLE]

and $\overleftarrow{\pi}$ is asymptotically optimal as $\epsilon\to 0$.

If $\theta<0$, the analogous results follow from symmetry. The existence of the limit in (1.2) and the validity of the identity in (1.3) follow immediately from (7.12), (7.13) and (7.14). Finally, setting $t=1$ and $x=0$, we deduce (2.4). ∎

8. Homogenization of the Bellman equation

We start with a lemma which excludes the full control regime.

Lemma 8.1**.**

For every $\omega\in\Omega$, $\delta\in[0,1)$, $\beta>0$ and $\theta\in\mathbb{R}$, the function $u(\cdot,\cdot,\omega)=u(\cdot,\cdot,\omega\,|\,\delta,\beta,\theta)$ (defined in (1.1)) satisfies the following Lipschitz condition: for every $m,n\in\mathbb{N}$ and $x,y\in\mathbb{Z}$,

[TABLE]

Proof.

It follows easily from (1.1) that

[TABLE]

for every $m,n\in\mathbb{N}$ and $x,y\in\mathbb{Z}$. The second inequality is obtained by considering the event that the particle marches from $x$ to $y$ in $|x-y|$ steps. A suitable combination of these inequalities gives (8.1). ∎

Proof of Theorem 2.4.

If $\delta\in[0,1]$, $\beta>0$ and $\theta\in\mathbb{R}$, then for $\mathbb{P}$-a.e. $\omega$,

[TABLE]

for every $t>0$ and $x\in\mathbb{R}$ by Theorems 1.4, 2.1, 2.2 and 2.3. Moreover, at $t=0$,

[TABLE]

It remains to improve this pointwise limit on $[0,\infty)\times\mathbb{R}$ to a uniform limit on compact subsets of $[0,\infty)\times\mathbb{R}$.

If $\delta\in[0,1)$, then Lemma 8.1 gives the following bounds: for every $\epsilon>0$, $s,t\geq 0$ and $x,y\in\mathbb{R}$,

[TABLE]

For every $\epsilon^{\prime}>0$, $t_{max}>0$ and $B>0$, partition the rectangle $[0,t_{max}]\times[-B,B]$ into finitely many (say $N$) identical squares with side length $\frac{\epsilon^{\prime}}{12}(\beta+|\theta|-\textstyle{\log(\frac{1-\delta}{2})})^{-1}$. Fix a point $(s_{i},y_{i})$ in the $i$th square. By pointwise convergence, there exists an $\epsilon_{i}>0$ such that $\epsilon_{i}(2\beta+2|\theta|-\textstyle{\log(\frac{1-\delta}{2})})<\frac{\epsilon^{\prime}}{6}$ and $|u_{\epsilon}(s_{i},y_{i},\omega)-u_{o}(s_{i},y_{i})|<\epsilon^{\prime}/3$ whenever $0<\epsilon<\epsilon_{i}$. If $(t,x)$ is any point in the $i$th square, then

[TABLE]

by (8.2). Taking $\epsilon_{0}=\min\{\epsilon_{1},\ldots,\epsilon_{N}\}$ concludes the proof of uniform convergence on $[0,t_{max}]\times[-B,B]$.

If $\delta=1$, then the walk under bang-bang policies is not elliptic and Lemma 8.1 is not applicable. Therefore, we prove the desired uniform convergence by revisiting Sections 5.1 and 5.2 where we obtained upper and lower bounds for $u_{\epsilon}(t,x,\omega)$ with error bounds. Fix $t_{max}>0$ and $B>0$.

• •

For every $h\in(0,1)$, $a>0$, $\mathbb{P}$-a.e. $\omega$ and sufficiently small $\epsilon>0$ (depending on $\omega,h,a,B$), (5.3) gives

[TABLE]

uniformly for $(t,x)\in[0,t_{max}]\times[-B,B]$. Here, the uniformity in $x$ comes from Lemma 4.2.

• •

When $\theta\geq 0$, for $\mathbb{P}$-a.e. $\omega$, (LABEL:fuatoz) gives

[TABLE]

uniformly for $(t,x)\in[0,t_{max}]\times[-B,B]$. Here, the uniformity in $x$ comes from Lemma B.2 (in Appendix B) which is applicable since $G_{\beta}$ (defined in (5.7)) is a bounded and centered cocycle.

• •

When $\theta\geq\beta\mathbb{E}[V(\cdot)]$, for $\mathbb{P}$-a.e. $\omega$, (5.9) gives

[TABLE]

uniformly for $(t,x)\in[0,t_{max}]\times[-B,B]$. Again, the uniformity in $x$ comes from Lemma B.2.

• •

When $0<\theta<\beta\mathbb{E}[V(\cdot)]$, (5.11) and the lower bound in the previous case give

[TABLE]

uniformly for $(t,x)\in[0,t_{max}]\times[-B,B]$.

• •

When $\theta=0$, (5.13) gives

[TABLE]

uniformly for $(t,x)\in[0,\infty)\times\mathbb{R}$.

Combining these upper and lower bounds, uniform convergence on $[0,t_{max}]\times[-B,B]$ follows. ∎

Acknowledgments

We thank E. Kosygina for suggesting to us that nonconvex homogenization can be of interest in the discrete setup, for valuable discussions which motivated this project, for her help in formulating the discrete problem treated here, and for her friendly and useful feedback on an earlier version of this manuscript.

Appendices

Appendix A Proof of existence of the tilted free energy via subadditivity

With future use in mind, we consider a more general model of RW in random potential on $\mathbb{Z}^{d}$ with $d\geq 1$. The proof of Theorem A.1 that we give below follows [28, Section 2] closely and does not require any additional effort due to this generality.

Denote by $(X_{i})_{i\geq 0}$ the SSRW on $\mathbb{Z}^{d}$ with $Z_{i+1}:=X_{i+1}-X_{i}\in U:=\{\pm e_{1},\ldots,\pm e_{d}\}$. Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space on which a collection $\{T_{z}:\Omega\to\Omega\}_{z\in U}$ of invertible measure-preserving transformations act ergodically. Fix a bounded and measurable function $\Psi:\Omega\times U\to\mathbb{R}$. For every $n\in\mathbb{N}\cup\{0\}$, $x,y\in\mathbb{Z}^{d}$ and $\omega\in\Omega$, define

[TABLE]

Here, $E_{x}$ stands for expectation with respect to the law of $(X_{i})_{i\geq 0}$ when $X_{0}=x$.

Theorem A.1**.**

For $\mathbb{P}$-a.e. $\omega$, the limit

[TABLE]

exists. Moreover, $\Lambda(\Psi)$ is a deterministic quantity.

Proof.

Assume without loss of generality that $\Psi:\Omega\times U\to[\psi_{o},0]$ for some $\psi_{o}>-\infty$. (Otherwise, we can subtract an appropriate constant from $\Psi$, take the limit in (A.1), and add the constant back.) For every $c>0$, $t\geq 0$, $x,y\in\mathbb{Z}^{d}$ and $\omega\in\Omega$, define

[TABLE]

We make several observations. First,

[TABLE]

since $\Psi(\cdot,\cdot)\leq 0$. Second, it is clear from (A.2) that

[TABLE]

Third, for every $z,z^{\prime}\in U$,

[TABLE]

since the probability of moving from $x+z$ to $x$ (resp. from $y$ to $y+z^{\prime}$) in one step is equal to $\frac{1}{2d}$. Therefore, there exists a constant $c^{\prime}=c+|\psi_{o}|+\log(2d)>0$ such that

[TABLE]

where $|\cdot|_{1}$ denotes the $\ell_{1}$-norm on $\mathbb{R}^{d}$. Fourth,

[TABLE]

since

[TABLE]

It follows from [28, Theorem 2.1] (which is in turn based on Liggett’s subadditive ergodic theorem [20]) that (A.3) - (A.7) ensure the existence of a deterministic, Lipschitz continuous and concave function $\lambda_{c}:\mathbb{R}^{d}\to(-\infty,0]$ such that

[TABLE]

We are ready to establish upper and lower bounds that will imply the existence of the limit in (A.1). For every $c>0$ and $\mathbb{P}$-a.e. $\omega$,

[TABLE]

Here, the last equality follows from (A.8) and the continuity of $\xi\mapsto\lambda_{c}(\xi)$ as in the proof of Varadhan’s integral lemma (see [10, Theorem 4.3.1]). It is clear from (A.2) that $f_{c}(\cdot,\cdot,\cdot,\cdot)$ decreases as $c$ increases, and so does $\lambda_{c}(\xi)$. Therefore,

[TABLE]

exists. Moreover, $c\mapsto\lambda_{c}(\xi)$ is convex since it is the limit of the supremum of a collection of linear functions. Using Sion’s minimax theorem (see [16]), we deduce the following upper bound:

[TABLE]

Obtaining a matching lower bound is equivalent to showing that

[TABLE]

for every $\xi\in\mathbb{R}^{d}$ such that $|\xi|_{1}\leq 1$. There is nothing to prove if $\lambda(\xi)=-\infty$. (In fact, this case can be ruled out.) Assume $\lambda(\xi)=-\ell>-\infty$. Fix an arbitrary $\epsilon>0$ and choose $c\geq(\ell+1)\epsilon^{-1}$. Then, $\lambda_{c}(\xi)\geq-\ell$ by monotonicity in $c$. Recalling (A.2) and (A.8), we see that

[TABLE]

for $\mathbb{P}$-a.e. $\omega$ and every $(x_{n})_{n\geq 1}$ such that $\frac{x_{n}}{n}\to\xi$. If $|m-n|\geq n\epsilon$, then

[TABLE]

Therefore,

[TABLE]

Observe that $|\log F(m,0,\omega)-\log F(n,0,\omega)|\leq|m-n||\psi_{o}|$, which gives

[TABLE]

Since $\epsilon>0$ is arbitrary, the desired lower bound (A.9) follows. ∎

Remark A.2*.*

There are alternative proofs of Theorem A.1. One of the authors established in [29] a so-called level-2 LDP from the point of view of the particle performing nearest-neighbor random walk in random environment (RWRE) on $\mathbb{Z}^{d}$, from which the existence of the limit in (A.1) follows as a corollary by Varadhan’s integral lemma. That paper built upon the Ph.D. thesis of Rosenbluth [25] who in turn adapted the work of Kosygina, Rezakhanlou and Varadhan [18] on the homogenization of second-order HJ stochastic PDEs with convex Hamiltonians. This approach is certainly more technical than the short and subadditivity-based proof we gave above, but it has the advantage of providing two variational formulas for $\Lambda(\Psi)$ (see Appendix C for these formulas in our setting). This result was subsequently generalized in [23] to random walks with arbitrary set of allowed steps (including directed walks). In the latter setting, Rassoul-Agha and Seppäläinen [22] also gave a proof of existence via subadditivity. Finally, assuming the existence of $\Lambda(\Psi)$, several variational formulas for it were given in [24] via a potential-theoretic approach which results in much shorter proofs than those in [25, 29, 23].

Appendix B Centered cocycles and sublinearity of path sums

Definition B.1**.**

A function $F:\Omega\times\{-1,1\}\to\mathbb{R}$ is said to be a cocycle if $F(\cdot,1)$ is $\mathcal{F}$-measurable and $F(\omega,-1)=-F(T_{-1}\omega,1)$ for every $\omega\in\Omega$. $F$ is said to be a centered cocycle if $\mathbb{E}[F(\cdot,1)]=0$.

The set of centered cocycles is denoted by $\mathcal{K}_{0}$.

Lemma B.2**.**

If $F\in\mathcal{K}_{0}$ is bounded, then for every $B>0$ and $\mathbb{P}$-a.e. $\omega$,

[TABLE]

Proof.

For every $y\in\mathbb{Z}$, define

[TABLE]

Since $F$ is bounded, there exists a $K>0$ such that

[TABLE]

for every $y,y^{\prime}\in\mathbb{Z}$. Since $F$ is centered, $f(\omega,y)=o(|y|)$ for $\mathbb{P}$-a.e. $\omega$ by the Birkhoff ergodic theorem. Hence, for every $\epsilon,B>0$ and $k\in\mathbb{N}$, there exists an $n_{0}(\omega,\epsilon,B,k)$ such that

[TABLE]

for every integer $j\in[-k,k]$ and $n\geq n_{0}$. Combining (B.1) and (B.2), we deduce that

[TABLE]

Since $F$ is a cocycle, telescoping gives $\sum_{i=0}^{n-1}F(T_{x_{i}}\omega,z_{i+1})=f(\omega,x_{n})-f(\omega,[nx])$ for any nearest-neighbor path $x_{0,n}$ with $x_{0}=[nx]$. Note that $|x_{n}|\leq|[nx]|+n\leq n(B+1)$. Therefore, the desired result follows from (B.3). ∎

Appendix C Variational formulas for the tilted free energy

We present here two variational formulas for the tilted free energy (defined in (1.6)) in our one-dimensional nearest-neighbor setting. These are provided for completeness and are not used elsewhere in the paper, except that some notation is used also in Appendix D.

The variational formulas are

[TABLE]

In (C.2), $p(\theta)=e^{\theta}/(e^{\theta}+e^{-\theta})$,

[TABLE]

and the supremum is taken over all $\mathcal{F}$-measurable $q:\Omega\to(0,1)$ and $\phi:\Omega\to(0,\infty)$ such that $\mathbb{E}[\phi(\cdot)]=1$ and

[TABLE]

for $\mathbb{P}$-a.e. $\omega$. The last equality implies that the probability measure $\phi d\mathbb{P}$ is invariant for the so-called environment Markov chain $(T_{X_{i}}\omega)_{i\geq 0}$ induced by the RWRE with probability $q(T_{x}\omega)$ of jumping to the right at the point $x$ in the environment $\omega$. These variational formulas follow e.g. from [29, Theorem 2.1]. See Remark A.2 for further references.

When $\Lambda_{\beta}(\theta)>\beta$, the variational problems in (C.1) and (C.2) are solved as follows. Assume without loss of generality that $\theta>0$. (Recall from Proposition 4.8(c) that $\Lambda_{\beta}(0)=\beta$.) Define

[TABLE]

Then, (4.15) readily implies

[TABLE]

Note that

[TABLE]

by (4.14). We consider the RWRE with probability $q_{\beta,\theta}(T_{x}\omega)$ of jumping to the right at the point $x$ in the environment $\omega$. It induces a probability measure $\hat{P}_{0}^{\beta,\theta,\omega}$ on nearest-neighbor paths starting at [math]. $\hat{E}_{0}^{\beta,\theta,\omega}$ denotes expectation under $\hat{P}_{0}^{\beta,\theta,\omega}$. With this notation, it is clear from (C.3) that $\mathbb{E}\left[\hat{E}_{0}^{\beta,\theta,\omega}[\tau_{1}]\right]<\infty$. Therefore,

[TABLE]

satisfies $\mathbb{E}[\psi_{\beta,\theta}(\cdot)]<\infty$ (see [29, Theorem 5.17]) and we define

[TABLE]

Proposition C.1**.**

Assume (1.4) and (1.5). If $\theta>0$ and $\Lambda_{\beta}(\theta)>\beta$, then

• (a)

the infimum in (C.1) is attained at $F_{\beta,\theta}$, and

• (b)

the supremum in (C.2) is attained at $(q_{\beta,\theta},\phi_{\beta,\theta})$.

Proof.

(a) This follows immediately from (4.15).

(b) It is easy to show that

[TABLE]

for $\mathbb{P}$-a.e. $\omega$ (see [29, Theorem 5.17]). By Kozlov’s lemma (see [19]), the probability measure $\phi_{\beta,\theta}d\mathbb{P}$ is ergodic for the RWRE defined by $q_{\beta,\theta}$. Therefore, for $\mathbb{P}$-a.e. $\omega$ and $\hat{P}_{0}^{\beta,\theta,\omega}$-a.s.,

[TABLE]

Here, the first equality holds by the Birkhoff ergodic theorem, and the second equality follows from Lemma B.2. We use (C.4) to deduce that

[TABLE]

Remark C.2*.*

The components of the vector $\left(\frac{1}{2}e^{\beta V(\omega)+\theta-\Lambda_{\beta}(\theta)},\frac{1}{2}e^{\beta V(\omega)-\theta-\Lambda_{\beta}(\theta)}\right)$ do not add up to $1$. $F_{\beta,\theta}$ is called the corrector precisely because it enables us to modify this vector and obtain the transition kernel $(q_{\beta,\theta}(\omega),1-q_{\beta,\theta}(\omega))$. The sublinearity in Lemma B.2 ensures that this modification does not alter the asymptotics in the $\frac{1}{n}\log$ scale. We refer to [6, Section 2, Remark 2] for a discussion of the relation between these type of correctors and the classical Kipnis-Varadhan correctors used to construct martingales in the proofs of invariance principles for Markov processes.

Appendix D Nondifferentiability of the tilted free energy at the endpoints of $\{\theta\in\mathbb{R}:\,\Lambda_{\beta}(\theta)=\beta\}$

Recall the notation from the previous section. When $\theta>0$ and $\lambda=\Lambda_{\beta}(\theta)>\beta$, the equality in (4.17) can be expressed as follows:

[TABLE]

where the second equality is due to telescoping. Define

[TABLE]

Then,

[TABLE]

by (C.3). Therefore, the law of large numbers holds for this RWRE and the limiting velocity $v_{\beta,\theta}$ satisfies

[TABLE]

(see [1, Theorem 4.1]). Recalling (D.1) and the proof of Proposition 4.8(e), we deduce by the implicit function theorem (together with $d/d\theta\left(\mathbb{E}\left[F_{\beta,\theta}^{\Lambda_{\beta}(\theta)}(\cdot,1)\right]\right)=0$ due to (4.18) and $\partial/\partial\theta(F_{\beta,\theta}^{\lambda}(\omega,1))=-1$ due to (4.3)) that

[TABLE]

In order to prove that $\theta\mapsto\Lambda_{\beta}(\theta)$ is not differentiable at $\pm\theta_{b}$ defined by

[TABLE]

it suffices to obtain an upper bound for $\mathbb{E}[(1+r_{\beta,\theta}(\cdot))S_{\beta,\theta}(\cdot)]$ that is uniform in $\theta>\theta_{b}$. To this end, observe that

[TABLE]

Since $V$ takes values in $[0,1]$, we have $\bar{r}_{\beta}(\omega)\leq 1$ for every $\omega\in\Omega$. Moreover, (1.4) implies that $\mathbb{E}[\bar{r}_{\beta}(\cdot)]<1$. As the proof of the following warm-up result demonstrates, the advantage of working with $\bar{r}_{\beta}(\omega)$ instead of $r_{\beta,\theta}(\omega)$ is that the former does not depend on $\theta$ and it depends on the potential only through $V(\omega)$.

Proposition D.1**.**

If $\left(V(T_{j}\omega)\right)_{j\in\mathbb{Z}}$ are i.i.d. under $\mathbb{P}$, then

[TABLE]

for every $\theta>\theta_{b}$. Hence, $\theta\mapsto\Lambda_{\beta}(\theta)$ is not differentiable at $\pm\theta_{b}$.

Proof.

Since $\bar{r}_{\beta}(\omega)$ is a function of $V(\omega)$, the random variables $\left(\bar{r}_{\beta}(T_{j}\omega)\right)_{j\in\mathbb{Z}}$ are i.i.d. under $\mathbb{P}$, too. Therefore, using that $r_{\beta,\theta}(\cdot)\leq\bar{r}_{\beta}(\cdot)$ by (D),

[TABLE]

whenever $\theta>\theta_{b}$. We use (D.2) to deduce that

[TABLE]

Recall Proposition 4.8. If the even and convex map $\theta\mapsto\Lambda_{\beta}(\theta)$ were differentiable at $\pm\theta_{b}$, it would be continuously differentiable (by the monotonocity of the derivative and an application of Darboux’s theorem). However, the latter is ruled out by (D.4) and the fact that the derivative vanishes on the nonempty interior of the closed interval $\{\theta\in\mathbb{R}:\,\Lambda_{\beta}(\theta)=\beta\}$. This concludes the proof. ∎

We can relax the i.i.d. assumption in Proposition D.1. To this end, let

[TABLE]

for every $h\in(0,1)$, where

[TABLE]

We can use this event to introduce a stationary discrete point process $N=N(\omega)=(N_{n}(\omega))_{n\geq 0}$ with $N_{0}(\omega)=0$ and $N_{n}(\omega)=\sum_{j=1}^{n}{{\rm 1\mkern-1.5mu}\!{\rm I}}_{A_{h}}(T_{j}\omega)$. For every $k\geq 0$, define

[TABLE]

In particular,

[TABLE]

Lemma D.2**.**

$\mathbb{E}[t_{h,1}]=\frac{1}{2}\mathbb{P}(A_{h})\mathbb{E}\left[\left.t_{h,1}(t_{h,1}+1)\,\right|\,A_{h}\right]$* and $\mathbb{E}[t_{h,k}]\leq\mathbb{P}(A_{h})\mathbb{E}\left[\left.t_{h,1}^{2}\,\right|\,A_{h}\right]$ for every $k\geq 2$.*

Proof.

The sequence $(t_{h,k})_{k\geq 1}$ is stationary under $\mathbb{P}(\cdot\,|\,A_{h})$. Moreover,

[TABLE]

for any nonnegative measurable function $g$ of the point process $N=(N_{n})_{n\geq 0}$ (see [8, Theorem 13.3.I]). Therefore,

[TABLE]

for every $k\geq 2$. ∎

Theorem D.3**.**

If $\mathbb{E}[t_{h,1}]<\infty$ for some $h\in(0,1)$, then $\theta\mapsto\Lambda_{\beta}(\theta)$ is not differentiable at $\pm\theta_{b}$.

Proof.

If $\mathbb{E}[t_{h,1}]<\infty$ for some $h\in(0,1)$, then

[TABLE]

for every $k\geq 2$ by Lemma D.2. Therefore,

[TABLE]

whenever $\theta>\theta_{b}$, where (D) was used in the first inequality. The rest of the proof is identical to that of Proposition D.1. ∎

Appendix E Large deviation estimates for the number of left excursions of RWs

Let $(X_{i})_{i\geq 0}$ denote SSRW on $\mathbb{Z}$. Similar to $\tau_{y}=\inf\{i\geq 0:\,X_{i}=y\}$ with $y\in\mathbb{Z}$, define

[TABLE]

Lemma E.1**.**

For every $\lambda\geq 0$,

[TABLE]

Proof.

The desired equalities clearly hold when $\lambda=0$. For $\lambda>0$ and $x\in\mathbb{Z}$, let $v_{\lambda}(x)=E_{x}[e^{-\lambda\tau_{0}}{{\rm 1\mkern-1.5mu}\!{\rm I}}_{\{\tau_{0}<\infty\}}]$. Then,

[TABLE]

$v_{\lambda}(0)=1$, and $\lim_{|x|\to\infty}v_{\lambda}(x)=0$. We substitute $v_{\lambda}(x)=e^{-a|x|}$ into (E.3) and find after an elementary computation that $e^{\pm a}=e^{\lambda}\pm\sqrt{e^{2\lambda}-1}$. Consequently, (E.1) follows and

[TABLE]

Proof of Proposition 6.1.

Recall from (6.8) that $\mathcal{L}_{0}(X_{0,n})$ counts the number of complete left excursions of SSRW starting from the origin, up to time $n$. For every $\xi\geq 0$, define

[TABLE]

It is clear that $I(0)=0$ and $I(\xi)=\infty$ if $\xi>1/2$. For $\xi\in(0,1/2]$,

[TABLE]

by Cramér’s theorem (see [10, Theorem 2.2.3]), where $\tau_{-1,0}^{i}$ are independent copies of $\tau_{-1,0}$. We substitute the expression on the right-hand side of (E.2) into the last expression in (E.4), check that the infimum there is attained when $\sqrt{1-e^{-2\lambda}}=2\xi$, and obtain the following formula (with the convention $0\log 0=0$):

[TABLE]

Note that $I$ is continuous and strictly increasing on $[0,1/2]$. Set $I(\xi)=\infty$ for every $\xi<0$. It follows that $\left(P_{0}(\frac{1}{n}\mathcal{L}_{0}(X_{0,n})\in\cdot\,)\right)_{n\geq 1}$ satisfies the large deviation principle with rate function $I:\mathbb{R}\to[0,\infty]$. Finally,

[TABLE]

by Varadhan’s integral lemma (see [10, Theorem 4.3.1]) and a routine computation.

Recall from Section 6.3 that $(Y_{k}^{\ell})_{k\geq 0}$ is a reflected RW on $[-\ell,\ell-1]$ subject to geometric holding times (with rate $1/2$) at $-\ell$ and $\ell-1$. Its transition probabilities are given in (6.9). For $y\in\mathbb{Z}$, let

[TABLE]

For $\lambda>0$, let $\tilde{v}_{\lambda}^{\ell}(x)=E_{x}[e^{-\lambda\tilde{\tau}_{0}^{\ell}}{{\rm 1\mkern-1.5mu}\!{\rm I}}_{\{\tilde{\tau}_{0}^{\ell}<\infty\}}]$. Then, similar to (E.3),

[TABLE]

$\tilde{v}_{\lambda}^{\ell}(0)=1$, and $\tilde{v}_{\lambda}^{\ell}(x)\leq e^{-\lambda|x|}$ for $x\in[-\ell,\ell-1]$. By the maximum principle, $\tilde{v}_{\lambda}^{\ell}(x)-v_{\lambda}(x)\to 0$ as $\ell\to\infty$. Therefore,

[TABLE]

We record this as follows: for every $\lambda>0$,

[TABLE]

satisfy

[TABLE]

For every $\ell,m\in\mathbb{N}$ and $\lambda>0$,

[TABLE]

where $\tilde{\tau}_{-1,0}^{\ell,i}$ are independent copies of $\tilde{\tau}_{-1,0}^{\ell}$. Therefore,

[TABLE]

It follows from (E.2) that $\varphi(\log\cosh(c))=-2c$. Since $J_{\ell}$ is clearly Lipschitz continuous with Lipschitz constant $1/2$,

[TABLE]

Recalling (E.5), we deduce that

[TABLE]

On the other hand, it is clear from the definition of $(Y_{k}^{\ell})_{k\geq 0}=(X_{\sigma_{k}})_{k\geq 0}$ that $\mathcal{L}_{0}(Y_{0,m}^{\ell})\geq\mathcal{L}_{0}(X_{0,m})$ for every $m\geq 1$ and every realization of the SSRW path $X_{0,m}$. Therefore,

[TABLE]

for every $\ell\in\mathbb{N}$. Combining (E.6) and (E.7) concludes the proof. ∎