On uniqueness of solutions to viscous HJB equations with a subquadratic nonlinearity in the gradient

TL;DR

This paper proves the uniqueness of positive solutions to certain viscous Hamilton-Jacobi-Bellman equations with subquadratic gradient nonlinearities under mild growth conditions on the potential function, using a novel measure-based approach.

Contribution

It introduces a new method involving infinite dimensional linear programming for measures, establishing uniqueness without strong regularity assumptions.

Findings

Unique positive solution for a broad class of coercive functions

Method applicable to larger Hamiltonian classes studied by Ichihara

Verification of optimality in ergodic control problems without parabolic equations

Abstract

Uniqueness of positive solutions to viscous Hamilton-Jacobi-Bellman (HJB) equations of the form $-\Delta u(x) + \frac{1}{\gamma} |D{u}(x)|^\gamma = f(x) - \lambda$, with $f$ a coercive function and $\lambda$ a constant, in the subquadratic case, that is, $\gamma\in(1,2)$, appears to be an open problem. Barles and Meireles [Comm. Partial Differential Equations 41 (2016)] show uniqueness in the case that $f(x) \approx |x|^\beta$ and $|D f(x)|\lessapprox |x|^{(\beta-1)_+}$ for some $\beta>0$, essentially matching earlier results of Ichihara, who considered more general Hamiltonians but with better regularity for $f$. Without enforcing this assumption, to our knowledge, there are no results on uniqueness in the literature. In this short article, we show that the equation has a unique positive solution for any locally Lipschitz continuous, coercive $f$ which satisfies $|D f(x)| \le \kappa\bigl(1 +|f(x)|^{2-\frac{1}{\gamma}}\bigr)$ for some positive constant $\kappa$. Since $2-\frac{1}{\gamma}>1$, this assumption imposes very mild restrictions on the growth of the potential $f$. We also show that this solution fully characterizes optimality for the associated ergodic problem. Our method involves the study of an infinite dimensional linear program for elliptic equations for measures, and is very different from earlier approaches. It also applies to the larger class of Hamiltonians studied by Ichihara, and we show that it is well suited to provide verification of optimality results for the associated ergodic control problems, even in a pathwise sense, and without resorting to the parabolic problem.

Equations

$$-\Delta u(x)+\frac{1}{\gamma }|Du(x){|}^{\gamma }=f(x)-\lambda ,$$

$$\lambda^{\mspace{-2.0mu}*}\,\coloneqq\,\sup\,\bigl{\{}\lambda\in{\mathds{R}^{d}}\,\colon\text{\lx@cref{creftype~refnum}{EP}\ has a subsolution}\bigr{\}}\,.$$

$$c^{-1}\lvert x\rvert^{\beta}-c\,\leq\,f(x)\,\leq\,c\bigl{(}1+\lvert x\rvert^{\beta}\bigr{)}\,,\quad\text{and\ \ }\bigl{\lvert}Df(x)\bigr{\rvert}\,\leq\,c^{-1}\Bigl{(}1+\lvert x\rvert^{(\beta-1)_{+}}\Bigr{)}$$

$$\bigl{\lvert}Df(x)\bigr{\rvert}\,\leq\,\kappa_{0}\Bigl{(}1+\lvert f(x)\rvert^{2-\frac{1}{\gamma}}\Bigr{)}\qquad\forall\,x\in{\mathds{R}^{d}}\,.$$

$$\mathcal{A}g(x,\xi ):=-\Delta g(x)+\xi \cdot Dg(x),\text{for }(x,\xi )\in \text{mathds}{R}^d\times \text{mathds}{R}^d.$$

$$F(x,\xi ):=f(x)+\frac{1}{{\gamma }^{\ast }}|\xi {|}^{{\gamma }^{\ast }},\text{with }{\gamma }^{\ast }:=\frac{\gamma }{\gamma -1}.$$

$$\mu (F):={\int }_{\text{mathds}{R}^d\times \text{mathds}{R}^d}F(x,\xi )\mu (\mathrm{d}x,\mathrm{d}\xi ),$$

$${\mathcal{M}}_{F}\,\coloneqq\,\bigl{\{}\mu\in{\mathcal{M}}\,\colon\mu(F)<\infty\bigr{\}}\,.$$

$${\mathcal{M}}_{F,r}\,\coloneqq\,\bigl{\{}\mu\in{\mathcal{M}}\,\colon\mu(F)\leq r\bigr{\}}$$

$$\overline{\lambda }:=\underset{\mu \in \mathcal{M}}{\inf }\mu (F).$$

$$\sup_{z\in B_{r}(x)}\,\lvert D{u}(z)\rvert\,\leq\,C\,\biggl{(}r^{-\frac{1}{\gamma-1}}+\sup_{z\in B_{2r}(x)}\,\bigl{(}f(z)-\lambda\bigr{)}_{+}^{\nicefrac{{1}}{{\gamma}}}+\sup_{z\in B_{2r}(x)}\,\bigl{\lvert}Df(z)\bigr{\rvert}^{\nicefrac{{1}}{{(2\gamma-1)}}}\biggr{)}\quad\forall\,x\in{\mathds{R}^{d}}\,,$$

$$\sup_{z\in B_{r}(x)}\,\lvert D{u}(z)\rvert\,\leq\,C_{0}\,\biggl{(}r^{-\frac{1}{\gamma-1}}+\sup_{z\in B_{2r}(x)}\,\bigl{(}f(y)-\lambda\bigr{)}_{+}^{\nicefrac{{1}}{{\gamma}}}\biggr{)}\quad\forall\,x\in{\mathds{R}^{d}}\,,\ \ \forall\,r>0\,.$$

$$-\Delta u_{r}(y)+\frac{1}{\gamma}\bigl{\lvert}D{u}_{r}(y)\bigr{\rvert}^{\gamma}\,=\,f_{r}(y)\,.$$

$$\underset{y\in B_1(x)}{\sup }|D{u}_r(y)|$$

$$\Gamma_{x}\,\coloneqq\,\bigl{[}f(x)\bigr{]}^{-\frac{1}{\gamma^{*}}}\,,\quad x\in{\mathds{R}^{d}}\,.$$

$$\inf_{y\in B_{r}}\,u\bigl{(}x+\Gamma_{x}\,y)\,\geq\,\kappa\,\bigl{[}f(x)\bigr{]}^{\frac{\gamma^{*}-2}{\gamma^{*}}}\quad\forall\,x\in{\mathds{R}^{d}}\,.$$

$$\frac{|Du(x){|}^2}{u(x)}\le M_{0}f(x)\forall x\in \text{mathds}{R}^d.$$

$$\lvert Df(x_{n})\rvert\,\leq\,2\kappa_{0}\bigl{[}f(x_{n})\bigr{]}^{\frac{\gamma^{*}+1}{\gamma^{*}}}\qquad\forall\,x\in B_{R}^{c}\,.$$

$$\inf_{y\in B_{r}}\,u(x_{n}+\Gamma_{x_{n}}y)\bigl{[}f(x_{n})\bigr{]}^{\frac{2-\gamma^{*}}{\gamma^{*}}}\,\xrightarrow[n\to\infty]{}\,0$$

$$-\Delta u(x)+b(x)\cdot Du(x)=f(x)-\lambda$$

$$u_{n}(y)\,\coloneqq\,\frac{u\bigl{(}x_{n}+\Gamma_{n}y\bigl{)}}{\bigl{[}f(x_{n})\bigr{]}^{\frac{\gamma^{*}-2}{\gamma^{*}}}}\,,\quad b_{n}(y)\,\coloneqq\,\Gamma_{n}\,b\bigl{(}x_{n}+\Gamma_{n}y\bigl{)}\,,\quad\text{and\ \ }f_{n}(y)\,\coloneqq\,\frac{f\bigl{(}x_{n}+\Gamma_{n}y\bigl{)}-\lambda}{f(x_{n})}\,,$$

$$-\Delta u_n(y)+b_n(y)\cdot D{u}_n(y)=f_n(y).$$

$$\bigl{[}f(x_{n}+\Gamma_{n}y)\bigr{]}^{-\frac{1}{\gamma^{*}}}-\bigl{[}f(x_{n})\bigr{]}^{-\frac{1}{\gamma^{*}}}\,\geq\,-\frac{2\kappa_{0}}{\gamma^{*}}\,\lvert y\rvert\,\Gamma_{n}\,.$$

$$\bigl{(}\tfrac{2}{3}\bigr{)}^{\gamma^{*}}\,f(x_{n})\,\leq\,f(x_{n}+\Gamma_{n}y)\,\leq\,2^{\gamma^{*}}f(x_{n})\quad\forall\,y\in B_{4r}\,,\quad\forall\,x_{n}\in B_{R}^{c}\,.$$

$$\lvert D{u}(x_{n}+\Gamma_{n}y)\rvert\,\leq\,C(1+\Gamma_{n}^{1-\gamma^{*}})\,=\,C\bigl{(}1+\bigl{[}f(x_{n})\bigr{]}^{\nicefrac{{1}}{{\gamma}}}\bigr{)}\quad\forall\,y\in B_{2r}\,.$$

$$\underset{y\in B_{2r}}{\sup }|Du(x_n+{\Gamma }_{n}y)|$$

$$\displaystyle\,=\,C_{0}\,\biggl{(}(2r)^{-\frac{1}{\gamma-1}}\bigl{[}f(x_{n})\bigr{]}^{\frac{1}{\gamma}}+\sup_{y\in B_{4r}}\,\bigl{(}f(x_{n}+\Gamma_{n}y)-\lambda\bigr{)}_{+}^{\nicefrac{{1}}{{\gamma}}}\biggr{)}\,.$$

$$\underset{x\in \text{mathds}{R}^d}{\sup }\underset{y\in B_{4r}}{\sup }\frac{f(x+{\Gamma }_{x}y)-\lambda }{f(x)}<\infty .$$

$$\frac{|Du(x){|}^2}{u(x)f(x)}$$

$$\displaystyle\,\leq\,2C_{0}\Biggl{(}(\nicefrac{{r}}{{2}})^{-\frac{1}{\gamma-1}}\frac{\bigl{[}f(x)\bigr{]}^{\frac{\gamma^{*}-2}{\gamma^{*}}}}{u(x)}+\sup_{y\in B_{r}}\,\frac{\bigl{[}f(x+\Gamma_{x}y)\bigr{]}^{\frac{\gamma^{*}-2}{\gamma^{*}}}}{u(x)}\times\frac{f(x+\Gamma_{x}y)}{f(x)}\Biggr{)}\quad\forall\,x\in{\mathds{R}^{d}}\,.$$

$$\bigl{\lvert}Df(x)\bigr{\rvert}\,\leq\,\kappa_{0}\Bigl{(}1+\lvert f(x)\rvert^{\frac{4\gamma-3}{3\gamma-2}}\Bigr{)}\qquad\forall\,x\in{\mathds{R}^{d}}\,.$$

Full text

1. Introduction

We consider the viscous Hamilton–Jacobi–Bellman (HJB) equation

[TABLE]

for $(u,\lambda)\in{C}^{2}({\mathds{R}^{d}})\times\mathds{R}$, with $\gamma>1$. Here, $f\in{\mathscr{W}}_{\text{loc}}^{1,\infty}({\mathds{R}^{d}})$ and is coercive. By coercive, sometimes also called inf-compact, we refer to a function $f$ whose sublevel sets $\{x\in{\mathds{R}^{d}}\,\colon f(x)\leq r\}$ are compact (or empty) for every $r\in\mathds{R}$. As shown in [Barles-16], Eq. EP has a classical solution $u$ for any $\lambda\leq\lambda^{\mspace{-2.0mu}*}$, where

[TABLE]

This equation which has a long history in the literature, has been studied in [Ichihara-12, Cirant-14] for somewhat more general Hamiltonians, and was recently revisited by Barles in [Barles-16]. What is of interest here, is to characterize the solutions of Eq. EP which are bounded from below, that is, without loss of generality, the positive solutions. Naturally, when we refer to this equation having a unique positive solution, we mean that the solution is unique up to an additive constant. In the superquadratic case ($\gamma\geq 2$), [Barles-16, Theorem 2.6] shows that Eq. EP has a unique positive solution for any coercive $f$, and in addition, for this solution, $\lambda=\lambda^{\mspace{-2.0mu}*}$. In the subquadratic case ($\gamma\in(1,2)$), [Barles-16] adopts assumption (H2) in [Ichihara-12], which states that $f$ satisfies a bound of the form

[TABLE]

for some positive constants $\beta$ and $c$ for all $x\in{\mathds{R}^{d}}$. Without enforcing this assumption, to our knowledge, there are no results on uniqueness in the literature, and therefore also no verification of optimality results. Note that [Cirant-14] introduces an additional stable drift to study the subquadratic case.

There is substantial literature on viscous HJB equations, other than [Barles-16, Ichihara-12, Cirant-14] mentioned above. It is not our intent to review this literature, since it does not address the problem studied in this paper, but we should at least mention [Barles-01, Ben-Fre, BenNag-91, Barles-10, Fujita-06a, Ichihara-13a, BenFre-92, Lasry-89].

We adopt the following assumption for $\gamma\in(1,2)$.

• (A1)

The function $f$ is locally Lipschitz continuous and coercive, and there exists a constant $\kappa_{0}$ such that

[TABLE]

We show that, under (A1), there exists a unique positive solution $u$ to Eq. EP. In addition, this solution fully characterizes the ergodic control problem in the sense that a stationary Markov control is optimal if and only if it agrees a.e. on ${\mathds{R}^{d}}$ with the function $\xi_{u}$ in Eq. 2.19 (see Theorem 4.1). The method we follow covers the more general Hamiltonians studied in [Ichihara-12, Cirant-14], and also improves the existing results for the superquadratic case. This is discussed in Section 3.

1.1. Brief summary of the method

Consider the operator ${\mathcal{A}}\colon{C}^{2}({\mathds{R}^{d}})\to{C}({\mathds{R}^{d}}\times{\mathds{R}^{d}})$ defined by

[TABLE]

Let ${\mathcal{P}}({\mathds{R}^{d}}\times{\mathds{R}^{d}})$ denote the space of probability measures on the Borel $\sigma$-algebra of ${\mathds{R}^{d}}\times{\mathds{R}^{d}}$, denoted as ${\mathfrak{B}}({\mathds{R}^{d}}\times{\mathds{R}^{d}})$, endowed with the Prokhorov topology. We say that $\mu\in{\mathcal{P}}({\mathds{R}^{d}}\times{\mathds{R}^{d}})$ is infinitesimally invariant for the operator ${\mathcal{A}}$ if $\int{\mathcal{A}}g\,\mathrm{d}\mu=0$ for all $g\in{C}_{c}^{2}({\mathds{R}^{d}})$, the latter denoting the functions in ${C}^{2}({\mathds{R}^{d}})$ with compact support, and denote the set of these probability measures by ${\mathcal{M}}$. Let

[TABLE]

For $\mu\in{\mathcal{P}}({\mathds{R}^{d}}\times{\mathds{R}^{d}})$ we use the simple notation

[TABLE]

and define

[TABLE]

In other words, ${\mathcal{M}}_{F}$ is the subset of ${\mathcal{M}}$ consisting of those probability measures under which $F$ is integrable. It is simple to show that ${\mathcal{M}}_{F}$ is always nonempty. Thus, since $F$ is coercive, the set

[TABLE]

is compact for all $r>0$ sufficiently large. Clearly, it is also convex.

Consider the minimization problem

[TABLE]

The lower semicontinuity of $\mu\mapsto\mu(F)$ then implies that the infimum of Eq. LP is attained in ${\mathcal{M}}$. We let ${\mathcal{M}}_{F}^{*}$ denote the set of points in ${\mathcal{M}}$ which attain this infimum.

Our approach to the proof of uniqueness of positive solutions of Eq. EP is as follows: First, we show that if $(u,\lambda)\in{C}^{2}({\mathds{R}^{d}})\times\mathds{R}$ is any pair solving Eq. EP, with $u$ a positive function, then $\lambda=\overline{\lambda}$ and some measure $\mu\in{\mathcal{M}}$ taking the form $\mu(\mathrm{d}{x},\mathrm{d}\xi)=\nu(\mathrm{d}{x})\delta_{D{u}(x)}(\mathrm{d}\xi)$, with $\nu\in{\mathcal{P}}({\mathds{R}^{d}})$ and $\delta_{D{u}(x)}$ denoting the Dirac mass at $D{u}(x)$, attains the infimum in Eq. LP, that is, it belongs to ${\mathcal{M}}_{F}^{*}$. Next, we show that ${\mathcal{M}}_{F}^{*}$ is a singleton, thus establishing the uniqueness of a positive solution to Eq. EP.

1.2. Notation

The standard Euclidean norm in $\mathds{R}^{d}$ is denoted by $\lvert\,\cdot\,\rvert$, and $\mathds{N}$ stands for the set of natural numbers. The closure, the boundary and the complement of a set $A\subset\mathds{R}^{d}$ are denoted by $\bar{A}$, $\partial{A}$ and $A^{c}$, respectively. The open ball of radius $r$ in $\mathds{R}^{d}$, centered at $x\in{\mathds{R}^{d}}$, is denoted by $B_{r}(x)$, and $B_{r}$ is the ball centered at [math]. We use $a_{\pm}\coloneqq\max(\pm a,0)$ for $a\in\mathds{R}$.

For a Borel space $Y$, ${\mathcal{P}}(Y)$ denotes the set of probability measures on its Borel $\sigma$-algebra, and $\delta_{y}$ denotes the Dirac mass at $y\in Y$. For $\mu\in{\mathcal{P}}(Y)$ and a measurable function $g\colon Y\to\mathds{R}$ which is integrable under $\mu$, we often use the simplifying notation $\mu(g)\coloneqq\int_{Y}f\,\mathrm{d}\mu$.

2. Main results

Throughout this section we assume $\gamma\in(1,2)$, unless otherwise explicitly mentioned. Also, without loss of generality we assume that $f\geq 1$, and we scale a solution of Eq. EP, which is bounded from below, by an additive constant so that $\inf_{\mathds{R}^{d}}\,u=1$.

We start with the very useful gradient estimate in [Ichihara-12, Theorem B.1], stated under weaker regularity in [Barles-16, Theorem A.2] for Eq. EP. It appears that the Bernstein approach for this estimate originates in [Lasry-89, Theorem A.1]. We need to use this estimate on small balls, so we scale it as follows.

Corollary 2.1**.**

There exists a constant $C$ such that for any solution $u$ of Eq. EP we have

[TABLE]

and for all $r>0$. In particular, under (A1), with perhaps a different constant $C_{0}$, we have

[TABLE]

Proof.

Fix any $x\in{\mathds{R}^{d}}$. For $r>0$, let $u_{r}(y)\coloneqq r^{\frac{2-\gamma}{\gamma-1}}u(x+ry)$ and f_{r}(y)\coloneqq r^{\gamma^{*}}\bigl{(}f(x+ry)-\lambda\bigr{)}. The function $u_{r}$ satisfies

[TABLE]

By [Ichihara-12, Theorem B.1], there exists a constant $C$ such that any solution $u_{r}$ of Eq. 2.3 satisfies

[TABLE]

from which Eq. 2.1 follows. ∎

We continue by proving a useful lower bound for positive solutions of Eq. EP. Define

[TABLE]

Lemma 2.1**.**

Assume (A1). Then, for every positive solution $u$ of Eq. EP, the following hold.

1. (a)

There exist positive constants $r$ and $\kappa$ such that

[TABLE] 2. (b)

There exists a positive constant $M_{0}$ such that

[TABLE]

Proof.

Note that by (A1) there exists some $R>0$ such that

[TABLE]

Choose $r$ positive and small enough such that $r\leq\frac{\gamma^{*}}{8\kappa_{0}}$. We claim that the assertion in part (a) holds for this $r$. To prove this, we use contradiction. Suppose that

[TABLE]

along some sequence $\{x_{n}\}_{n\in\mathds{N}}\subset{\mathds{R}^{d}}$, such that $\lvert x_{n}\rvert\to\infty$ as $n\to\infty$. We write Eq. EP as

[TABLE]

with $b(x)\coloneqq\frac{1}{\gamma}\lvert D{u}\rvert^{\gamma-2}D{u}(x)$. Simplifying the notation, let \Gamma_{n}\equiv\Gamma_{x_{n}}=\bigl{[}f(x_{n})\bigr{]}^{-\frac{1}{\gamma^{*}}}, and define the sequence of scaled functions

[TABLE]

for $y\in{\mathds{R}^{d}}$ and $n\in\mathds{N}$. Then we obtain from Eq. EP that

[TABLE]

Integrating Eq. 2.6, we obtain

[TABLE]

Computing also the lower bound inherited from Eq. 2.6 and combining it with Eq. 2.9, we obtain

[TABLE]

This shows that $f_{n}$ and $f_{n}^{-1}$ are bounded in $B_{4r}$ uniformly in $n\in\mathds{N}$. To establish a bound for $b_{n}$ on $B_{2r}$, it is enough to show that, for some constant $C$, we have

[TABLE]

By Eq. 2.2 we have

[TABLE]

Thus Eq. 2.11 follows by Eqs. 2.10 and 2.12.

Therefore, since, as we have shown, $f_{n}$, $f_{n}^{-1}$ and $b_{n}$ are bounded in $B_{2r}$ uniformly in $n\in\mathds{N}$, then, by using for example [AA-Harnack, Lemma 3.6], we see that equation Eq. 2.8 contradicts the hypothesis in Eq. 2.7 that $\inf_{y\in B_{r}}\,u_{n}(y)\to 0$. This completes the proof of part (a).

Moving to part (b), let $r$ be as chosen in the proof of part (a). We have shown above that

[TABLE]

On the other hand, using the estimate Eq. 2.12 on $B_{\nicefrac{{r}}{{2}}}$, with $r$ and $R$ as in part (a), we have

[TABLE]

Therefore, Eq. 2.5 follows by Eqs. 2.4 and 2.13. This completes the proof. ∎

Remark 2.1*.*

The estimate in Lemma 2.1 is not suitable for the superquadratic case. A different scaling can be used when $\gamma\geq 2$. First, we replace (A1) by

[TABLE]

Then, under Eq. 2.14, we obtain

[TABLE]

for some positive constants $r$ and $\kappa$. To prove this, we use \Gamma_{x}=\bigl{[}f(x)\bigr{]}^{\frac{1-\gamma}{3\gamma-2}}, and follow the proof of Lemma 2.1.

To continue, we need the following notation.

Notation 2.1**.**

For $r>0$, we let $\chi_{r}$ be a concave ${C}^{2}(\mathds{R})$ function such that $\chi_{r}(t)=t$ for $t\leq r$, and $\chi^{\prime}_{r}(t)=0$ for $t\geq 3r$. Then $\chi^{\prime}_{r}$ and $-\chi^{\prime\prime}_{r}$ are nonnegative, and the latter is supported on $[r,3r]$. In addition, we select $\chi_{r}$ so that

[TABLE]

This is always possible. For example, we can specify $\chi^{\prime\prime}_{r}$ as

[TABLE]

Recall the definitions in Eqs. 1.4, 1.5 and LP.

Lemma 2.2**.**

Assume (A1). For any positive solution $u\in{C}^{2}({\mathds{R}^{d}})$ of Eq. EP with eigenvalue $\lambda$ and $\mu\in{\mathcal{M}}_{F}$, we have

[TABLE]

In particular, $\lambda\leq\overline{\lambda}$.

Proof.

Since $u$ is coercive by Lemma 2.1 (a), it follows that $\chi_{r}(u)-r-1$ is compactly supported. Thus we have

[TABLE]

by the definition of ${\mathcal{M}}$. On the other hand, we have

[TABLE]

Therefore,

[TABLE]

for all $\mu\in{\mathcal{M}}$. By Lemma 2.1 we have $\frac{\lvert D{u}\rvert^{2}}{u}\leq M_{0}f$ for some constant $M_{0}$. Using this together with Eq. 2.16, we obtain

[TABLE]

since $\int f\mathrm{d}\mu<\infty$ by the definition of ${\mathcal{M}}_{F}$. Thus letting $r\nearrow\infty$, and applying the monotone convergence theorem, we obtain Eq. 2.17, thus completing the proof. ∎

It is convenient to express the operator ${\mathcal{A}}$ in Eq. 1.3 in terms of a family of operators $\{{\mathcal{L}}_{\xi}\}_{\xi\in{\mathds{R}^{d}}}$ defined by

[TABLE]

It is clear from the Legendre–Fenchel transform

[TABLE]

that, for any solution $u$ of Eq. EP, we have

[TABLE]

and that the maximum is realized at $\xi(x)=\lvert Du(x)\rvert^{\gamma-2}Du(x)$.

The next lemma applies to any $\gamma>1$.

Lemma 2.3**.**

Let $\gamma>1$. Let $u$ be a coercive solution of Eq. EP with eigenvalue $\lambda$. Define

[TABLE]

Then, there exists a Borel probability measure $\nu_{u}$ on ${\mathfrak{B}}({\mathds{R}^{d}})$, such that

[TABLE]

and

[TABLE]

In particular, $\overline{\lambda}\leq\lambda$.

Proof.

Using the definition in Eq. 2.18, we write

[TABLE]

Since $u$ is coercive, we can apply [Bogachev-01d, Theorem 1.2] to assert the existence of a unique $\nu_{u}\in{\mathcal{P}}({\mathds{R}^{d}})$ which satisfies

[TABLE]

Thus Eq. 2.20 follows from Eq. 2.23 and the definition of $\mu_{u}$, whereas Eq. 2.21 follows by integrating Eq. 2.22 with respect to $\nu_{u}$ using a cut-off function as in the proof of Lemma 2.2, which shows that $\int_{\mathds{R}^{d}}{\mathcal{L}}_{\xi_{u}(x)}g(x)\,\mathrm{d}\mu_{u}\leq 0$ for any positive function $g\in{C}^{2}({\mathds{R}^{d}})$. ∎

Remark 2.2*.*

Lemma 2.3 can also be established by a simple probabilistic argument. Viewing Eq. 2.20 as a Foster–Lyapunov equation, it is well known that the coercivity of $u$ implies that the diffusion with extended generator ${\mathcal{L}}_{\xi_{u}(x)}$ is positive recurrent. The measure $\nu_{u}$ can then be specified as the unique invariant probability measure of this diffusion.

We now discuss some properties of the set ${\mathcal{M}}$ of infinitesimally invariant measures which are needed for the proof of Theorem 2.1 below. It is clear that every $\mu\in{\mathcal{M}}$ can be disintegrated into a probability measure $\nu(\mathrm{d}{x})\in{\mathcal{P}}({\mathds{R}^{d}})$ and a Borel measurable probability kernel $\eta(x,\mathrm{d}\xi)$ on ${\mathds{R}^{d}}\times{\mathfrak{B}}({\mathds{R}^{d}})$. We denote this disintegration by $\mu=\nu\circledast\eta$. For $\nu\circledast\eta\in{\mathcal{M}}$, define $\bar{\eta}(x)\coloneqq\int_{\mathds{R}^{d}}\xi\,\eta(x,\mathrm{d}\xi)$. Then $\bar{\eta}\colon{\mathds{R}^{d}}\to{\mathds{R}^{d}}$ is a Borel measurable map. It is straightforward to verify that $\nu\circledast\delta_{\bar{\eta}}$ is in ${\mathcal{M}}$. Since, by convexity, we have

[TABLE]

it is clear that the infimum in Eq. LP is attained at some $\mu\in{\mathcal{M}}$ whose disintegration results in a kernel $\eta(x,\,\cdot\,)$ which is Dirac for each $x\in{\mathds{R}^{d}}$. Then, $\eta$ can be represented as a Borel measurable map $v\colon{\mathds{R}^{d}}\to{\mathds{R}^{d}}$, and vice-versa. We denote the class of such measures as $\overline{\mathcal{M}}$, and abusing the notation we represent them as $\mu=\nu\circledast v$. Consider such a $\mu=\nu\circledast v$ in $\overline{\mathcal{M}}\cap{\mathcal{M}}_{F}$. It follows by Eq. 2.17 that $\int_{\mathds{R}^{d}}\lvert v(x)\rvert^{\gamma^{*}}\,\eta(\mathrm{d}{x})<\infty$. Since $\gamma^{*}>2$, this implies that $\int_{\mathds{R}^{d}}\lvert v(x)\rvert^{2}\,\eta(\mathrm{d}{x})<\infty$, and thus $\nu$ has density $\varrho\in{L}^{\nicefrac{{d}}{{(d-1)}}}({\mathds{R}^{d}})$ with respect to the Lebesgue measure by [Bogachev-96, Theorem 1.1].

We continue with our main theorem. Recall the definition of $\lambda^{\mspace{-2.0mu}*}$ in Eq. 1.1, and that ${\mathcal{M}}_{F}^{*}$ denotes the subset of ${\mathcal{M}}$ consisting of points that attain the infimum in Eq. LP.

Theorem 2.1**.**

Assume (A1). The following hold.

1. (a)

For any positive solution $u$ of Eq. EP with eigenvalue $\lambda$ we have

[TABLE]

with $\mu_{u}$ as in Lemma 2.3. 2. (b)

The set ${\mathcal{M}}_{F}^{*}$ is a singleton. 3. (c)

There exists a unique positive solution of Eq. EP.

Proof.

By Lemma 2.1 every positive solution of Eq. EP is coercive. Therefore, the first two equalities in part (a) follow by Lemmas 2.2 and 2.3. By [Barles-16, Theorem 2.6], there exists a solution with eigenvalue $\lambda^{\mspace{-2.0mu}*}$ which is bounded from below. This of course implies $\overline{\lambda}=\lambda^{\mspace{-2.0mu}*}$ thus completing the proof of part (a).

Let $\nu_{u}$ and $\xi_{u}$ be as in Lemma 2.3. Let $\mu=\nu\circledast v$ be any element of ${\mathcal{M}}_{F}^{*}\cap\overline{\mathcal{M}}$. By the discussion in the paragraph preceding the theorem, $\nu$ has a density $\varrho\in{L}^{\nicefrac{{d}}{{(d-1)}}}({\mathds{R}^{d}})$ with respect to the Lebesgue measure. Let $\varrho_{u}$ denote the density of $\nu_{u}$, which, as well known, is strictly positive. Let

[TABLE]

It is straightforward to verify that $\bar{\nu}\circledast\bar{v}\in{\mathcal{M}}_{\circ}$.

By optimality, we have

[TABLE]

by convexity. Thus $\bar{\nu}\circledast\bar{v}\in{\mathcal{M}}_{F}^{*}$. Since $\rho_{u}$ is strictly positive, (2.24) implies that $v=\xi_{u}$ on the support of $\varrho$. It is clear that if $v$ is modified outside the support of $\varrho$, then the modified measure is also infinitesimally invariant for ${\mathcal{A}}$. Therefore $\nu\circledast\xi_{u}\in{\mathcal{M}}_{F}^{*}$. The uniqueness of a probability measure satisfying Eq. 2.23 then implies that $\nu=\nu_{u}$, which in turn implies (since $v=\xi_{u}$ on the support of $\nu$) that $v=\xi_{u}$ a.e. in ${\mathds{R}^{d}}$. This completes the proof of part (b).

Turning to part (c), existence of a positive solution follows from [Barles-16, Theorem 2.6]. By part (b), for any positive solutions $u$ and $w$, we have $\xi_{u}=\xi_{w}$ a.e. in ${\mathds{R}^{d}}$, implying that $Du=Dw$ on ${\mathds{R}^{d}}$. Thus the solution is unique up to an additive constant. This completes the proof. ∎

3. More general Hamiltonians

In this section we consider viscous equations taking the form

[TABLE]

with more general Hamiltonians $H$. We adopt the following assumptions.

• (A2)

The function $f$ is in ${C}^{2}({\mathds{R}^{d}})$ and is coercive. The Hamiltonian $H$ satisfies the following.

1. (i)

H\in{C}^{2}({\mathds{R}^{d}}\times\bigl{(}{\mathds{R}^{d}}\setminus\{0\})\bigr{)}, and $p\mapsto H(x,p)$ is strictly convex for all $x\in{\mathds{R}^{d}}$. 2. (ii)

There exist constants $h_{0}>0$ and $\gamma>1$, such that

[TABLE]

The hypothesis (A2) is equivalent to (A2′) below for the Lagrangian $L$, which is related to $H$ via the Fenchel–Legendre transform, that is,

[TABLE]

• (A2′)

The function $f$ is in ${C}^{2}({\mathds{R}^{d}})$ and is coercive. The Lagrangian $L$ satisfies the following.

1. (i)

L\in{C}^{2}({\mathds{R}^{d}}\times\bigl{(}{\mathds{R}^{d}}\setminus\{0\})\bigr{)}, and $\xi\mapsto L(x,\xi)$ is strictly convex for all $x\in{\mathds{R}^{d}}$. 2. (ii)

There exist constants $l_{0}>0$ and $\gamma^{*}>1$, such that

[TABLE]

In addition, under (A2) or (A2′), there exists positive constants $h_{1}$ and $l_{1}$ such that

[TABLE]

for all $(x,p,\xi)\in\mathds{R}^{3d}$, and

[TABLE]

with equality if and only if $\xi=D_{p}H(x,p)$ or $p=D_{\xi}L(x,\xi)$.

The model above is slightly more general than the model in [Ichihara-12, Cirant-14]. A more restrictive assumption on $H$ is used in [Ichihara-12], while $H$ does not depend on $x$ in [Cirant-14]. For the properties mentioned above see [Ichihara-12, Theorem 3.4] and [Cirant-14, Proposition 4.1].

As mentioned earlier, [Ichihara-12] imposes the assumptions in Eq. 1.2 for $f$, for both the subquadratic and superquadratic cases. Barles in [Barles-16] uses Eq. 1.2 only for the subquadratic case, while [Cirant-14] does not consider unbounded $f$ for the subquadratic case. Analogous is the model in [Ben-Fre, Section 4.6].

The results for this Hamiltonian are essentially the same as those in Section 2. We need the following ramification of [Ichihara-12, Theorem B.1] analogous to Corollary 2.1 valid for solutions of Eq. 3.1.

Corollary 3.1**.**

Assume (A2). Then, there exists a constant $C$ such that any solution $u$ of Eq. 3.1 satisfies Eq. 2.1.

Proof.

A closer inspection of the proof of [Ichihara-12, Theorem B.1] reveals that the following is established. Let $g\in{C}^{2}({\mathds{R}^{d}})$ be a coercive function. There exists a function $C\colon(0,\infty)^{2}\to(0,\infty)$ such that if $\varphi\in{C}^{3}({\mathds{R}^{d}})$ satisfies

[TABLE]

for a pair of positive constants $(c_{1},c_{2})$, then

[TABLE]

We use scaling. With $u$ a solution of Eq. 3.1, we define $u_{r}(y)\coloneqq r^{\frac{2-\gamma}{\gamma-1}}u(x+ry)$. Using (A2) (ii) and Eq. 3.3, we deduce that $u_{r}$ and g\equiv r^{\gamma^{*}}\bigl{(}f(x+ry)-\lambda\bigr{)} satisfy Eq. 3.5 for all $r\in(0,1]$ and for constants $c_{1}$ and $c_{2}$ which do not depend on $r$. The result then follows by Eq. 3.6. ∎

For the model in Eq. 3.1, we define

[TABLE]

and ${\mathcal{M}}_{F}$ and $\overline{\lambda}$ as in Eq. 1.5 and Eq. LP, respectively, relative to $F$ in Eq. 3.7. We also let

[TABLE]

Recall that ${\mathcal{M}}_{F}^{*}$ is the set of measures in ${\mathcal{M}}$ that attain the infimum in Eq. LP.

Theorem 3.1**.**

Assume (A1)–(A2) and $\gamma\in(1,2)$. Then

1. (a)

The conclusions of Lemma 2.1 hold. 2. (b)

For any positive solution $u\in{C}^{2}({\mathds{R}^{d}})$ of Eq. 3.1 with eigenvalue $\lambda$ and $\mu\in{\mathcal{M}}_{F}$, we have

[TABLE]

there exists a Borel probability measure $\nu_{u}$ on ${\mathfrak{B}}({\mathds{R}^{d}})$, such that, with $\xi_{u}$ as defined in Eq. 3.8, we have

[TABLE]

and

[TABLE]

In particular, $\lambda=\overline{\lambda}$. 3. (c)

We have ${\mathcal{M}}_{F}^{*}=\{\mu_{u}\}$, with $\mu_{u}$ as in Eq. 3.10. 4. (d)

There exists at most one positive solution of Eq. 3.1.

Proof.

Part (a) follows as in Lemma 2.1 with a slight modification. Instead of Eq. 2.8, we use the inequality

[TABLE]

with \xi_{n,u}(y)\,\coloneqq\,\Gamma_{n}\,\xi_{u}\bigl{(}x_{n}+\Gamma_{n}y\bigl{)}, and $\xi_{u}$ as in Eq. 3.8. Then we apply Eq. 3.3 and Corollary 3.1, and follow the proof of Lemma 2.1.

For part (b), we define

[TABLE]

and write

[TABLE]

and following the proof of Lemma 2.2, we obtain

[TABLE]

The remaining assertions in (b) follow by Lemma 2.3, with $\xi_{u}$ as defined in Eq. 3.8.

Part (c) follows as in Lemma 2.3 with a slight difference. For $\gamma>2$ we don’t know a priori that $\lvert\xi\rvert^{2}$ is integrable under a measure in ${\mathcal{M}}_{F}$. So instead of the densities $\zeta$ and $\zeta_{u}$ we use the Radon–Nikodym derivatives.

Part (d) follows from parts (b) and (c). This concludes the proof. ∎

Remark 3.1*.*

Theorem 3.1 does not address existence of a positive solution to Eq. 3.1. For Hamiltonians not depending on $x$, existence is asserted in [Cirant-14]. In general, under some additional assumptions, we can show that there exists a positive solution to Eq. 3.1. In addition to (A1)–(A2), we also assume that for any bounded $C^{2}$ domain $D$, there exists a constant $\beta>0$ satisfying the following: for every $\varepsilon>0$ there exists $\delta>0$ such that

[TABLE]

This is same as [LP16, (2.23)]. It then follows from [LP16, Theorem 2.15] that there exists a unique $v_{D}\in C^{2}(D)$ and a constant $c_{D}$ satisfying

[TABLE]

Furthermore, $c_{D}$ is characterized as follows:

[TABLE]

Thus $c_{D}$ is monotone decreasing as a function of $D$. Denote by $(v_{n},c_{n})$ the solution pair of Eq. 3.13 corresponding to $D=B_{n}$, and let $x_{n}\in\operatorname*{Arg\,min}v_{n}$. It follows from the equation above that

[TABLE]

which implies that

[TABLE]

Let $c=\lim_{n\to\infty}c_{n}$ which exists by the above estimate. It also clear that $v_{n}$ attains its minimum in a compact set independent of $n$. Thus we can follow a standard argument (see [Barles-16, Theorem 2.6]) to show that $v_{n}-\min_{\mathds{R}^{d}}v_{n}\to v$ as $n\to\infty$, and

[TABLE]

Assumption (3.12) is satisfied by a large class of Hamiltonian. For instance, consider

[TABLE]

for some bounded function $b$. Then we can choose $\beta=\frac{1}{\gamma}$ above. Note that (3.12) follows from the estimate below.

[TABLE]

where the constant $C$ depends on $\lVert b\rVert_{\infty}$ and $\gamma$. Now choose $\delta=C^{-\frac{\gamma-1}{\gamma}}\varepsilon$.

3.1. Remarks on the superquadratic case

Cirant in [Cirant-14] is adopting (A2), except that in his model the Hamiltonian does not depend on $x$, that is, $H(x,p)=H(p)$. He also assumes that $f$ and $Df$ have at most polynomial growth. He shows that there always exists a positive solution to Eq. 3.1, and that this has a at least linear growth.

For this model we can establish that $\int\lvert u\rvert^{2}\,D\mu<\infty$ for all $\mu\in{\mathcal{M}}_{F}$, and that therefore, Eq. 3.9 holds. However, the proof of this differs from the proof of Lemma 2.2. We choose instead a smooth concave function $\chi$ such that $\chi(s)=s$ for $s\leq 0$, and $\chi(s)=1$ for $s\geq 2$, and we scale it by defining $\chi_{t}(s)\coloneqq t+\chi(s-t)$ for $t\in\mathds{R}$. Since $\gamma\geq 2$, and $Du$ has polynomial growth, while $u$ has at least linear growth, we can follow the argument in the proof of [ABBK, Theorem 4.1] to conclude that $\int\lvert u\rvert^{2}\,D\mu<\infty$ for all $\mu\in{\mathcal{M}}_{F}$. In [Cirant-14], the set of admissible controls are required to satisfy $\limsup_{T\to\infty}\,\frac{\operatorname{\mathbb{E}}^{x}[u(X_{t})]}{T}=0$. This is an unnecessary restriction on the class of admissible controls, and can be avoided. Without assuming that $H(p)$ is strictly convex, which might result in non-uniqueness for $u$, the approach summarized above, shows that $\xi_{u}$ is an optimal Markov control and the corresponding infinitesimal measure is a minimizer of Eq. LP. Thus, we have a strong notion of optimality as explained in Section 4. Under the additional assumption that $H(p)$ is strictly convex, the positive solution $u$, and therefore also the optimal Markov control are unique.

4. Implications for the ergodic control problem

The problem Eq. EP is associated with an ergodic control problem for the diffusion $X=(X_{t})_{t\geq 0}$ given by the Itô stochastic differential equation

[TABLE]

This equation is specified on a complete, filtered probability space \bigl{(}\Omega,{\mathfrak{F}},\operatorname{\mathbb{P}},({\mathfrak{F}}_{t})_{t\geq 0}\bigr{)}, with $(W_{t})_{t\geq 0}$ an $({\mathfrak{F}}_{t})$-adapted $d$-dimensional Brownian motion. An admissible control is an ${\mathds{R}^{d}}$-valued $({\mathfrak{F}}_{t})$-progressively measurable process $\xi=(\xi_{t})_{t\geq 0}$, such that \operatorname{\mathbb{E}}\Bigl{[}\int_{0}^{T}\lvert\xi_{t}\rvert^{\gamma^{*}}\,\mathrm{d}{t}\Bigr{]}<\infty for all $T>0$, and we let $\mathscr{A}$ denote the class of such controls. The running cost function is given by $F$ in Eq. 1.4.

In [Ichihara-12], optimality is established via the study of the parabolic problem. In view of the optimality results concerning Eq. LP, we can state a stronger version of optimality. We state this result for the model in Eq. EP, noting that an identical argument can be used to establish this for Eq. 3.1 under (A2). We let $\operatorname{\mathbb{E}}^{x}_{\xi}$ denote the expectation operator for the diffusion in Eq. 4.1 controlled by $\xi\in\mathscr{A}$ with initial condition $X_{0}=x$.

Theorem 4.1**.**

Assume (A1). Let $u\in{C}^{2}({\mathds{R}^{d}})$ be the unique positive solution of Eq. EP in the subquadratic case, or as in Section 3.1 for the superquadratic case. With $\xi_{u}$ as in Eq. 2.19, we have

[TABLE]

Moreover, Eq. 4.2 holds without the expectation operators in the a.s. pathwise sense. In addition a Markov control $v\colon{\mathds{R}^{d}}\to{\mathds{R}^{d}}$ is optimal, if and only if it agrees with $\xi_{u}$ a.e. in ${\mathds{R}^{d}}$.

Proof.

The inequality in Eq. 4.2 follows from the fact that limit points of mean empirical measures in ${\mathcal{P}}({\mathds{R}^{d}}\times{\mathds{R}^{d}})$ are infinitesimal measures for the operator ${\mathcal{A}}$ (see Lemma 3.4.6 and Theorem 3.4.7 in [ABG12]) together with the definition of $\overline{\lambda}$ in Eq. LP. The equality follows by the ergodicity of the process under the control $\xi_{u}$ and the fact that $F$ is integrable under the invariant probability measure as asserted in Lemma 2.3. The pathwise results also follow from results in [ABG12] referenced above. The verification part of the theorem follows from Theorem 2.1. ∎

Acknowledgements

The work of Ari Arapostathis and Luis Caffarelli was supported in part by the National Science Foundation through grant DMS-1715210. The work of Ari Arapostathis was also supported in part by the Army Research Office through grant W911NF-17-1-001. The research of Anup Biswas was supported in part by an INSPIRE faculty fellowship and DST-SERB grant EMR/2016/004810.