Mean Field Games with state constraints: from mild to pointwise solutions of the PDE system

TL;DR

This paper develops a mathematical framework for analyzing Mean Field Games with state constraints by establishing a global semiconcavity property of the value function, enabling a better understanding of the associated PDE system.

Contribution

It introduces a novel approach to define solutions for the PDE system of constrained Mean Field Games using semiconcavity properties.

Findings

Proves global semiconcavity of the value function with state constraints

Provides a new interpretation of the PDE system for constrained Mean Field Games

Bridges the gap between control problems and PDE analysis in this context

Abstract

Mean Field Games with state constraints are differential games with infinitely many agents, each agent facing a constraint on his state. The aim of this paper is to provide a meaning of the PDE system associated with these games, the so-called Mean Field Game system with state constraints. For this, we show a global semiconvavity property of the value function associated with optimal control problems with state constraints.

Equations

$$(MFG)\{\begin{array}{l}-{\partial }_{t}u+H(x,Du)=F(x,m)\\ {\partial }_{t}m-div(m{D}_{p}H(x,Du))=0\\ m(0)=m_{0}u(x,T)=G(x,m(T)).\end{array}$$

$$\frac{1}{2}u(t+\sigma ,x+h)+\frac{1}{2}u(t-\sigma ,x-h)-u(t,x)\le c(|h|+\sigma {)}^{\frac{3}{2}}$$

$$\{\begin{array}{l}-{\partial }_{t}u+H(x,Du)=F(x,m(t))\text{in}(0,T)\times \overline{\Omega },\\ u(x,T)=G(x,m(T))\text{in}\overline{\Omega },\end{array}$$

$$\{\begin{array}{ll}{\partial }_{t}m+div(Vm)=0,& \text{mbox}in(0,T)\times \overline{\Omega },\\ m(0,x)=m_0(x),& \text{mbox}in\overline{\Omega }\end{array}$$

$$V(t,x)=-D_{p}H(x,Du(t,x)).$$

$$V(t,x)=-D_{p}H\Bigl{(}x,D^{\tau}_{x}u(t,x)+\lambda_{+}(t,x)\nu(x)\Bigr{)},$$

$$-{\partial }_{t}u(t,x)+H(x,D_x^{\tau }u(t,x)+{\lambda }_{+}(t,x)\nu (x))=F(x,m(t)),$$

$$||A||=\underset{|x|=1}{\max }|Ax|(x\in {\mathbb{R}}^n).$$

$$1_S(x)=\{\begin{array}{ll}1& x\in S,\\ 0& x\in S^c.\end{array}$$

$$\|\phi\|_{k,\infty}:=\sup_{\tiny\begin{array}[]{c}x\in U\\ |\alpha|\leq k\end{array}}|D^{\alpha}\phi(x)|<\infty$$

$$d_{\Omega }(x):=\underset{y\in \overline{\Omega }}{\inf }|x-y|(x\in {\mathbb{R}}^n).$$

$$b_{\Omega }(x)=d_{\Omega }(x)-d_{{\Omega }^c}(x)(x\in {\mathbb{R}}^n).$$

$$b_{\Omega}(\cdot)\in C^{2}_{b}\ \ \text{on}\ \ \Sigma_{\rho_{0}}=\Big{\{}y\in B(x,\rho_{0}):x\in\partial\Omega\Big{\}}.$$

$$supp(\eta):=\Big{\{}x\in X:\eta(V)>0\ \mbox{for each neighborhood V of $x$}\Big{\}}.$$

$$\underset{i\to \infty }{\lim }{\int }_{X}f(x)d{\eta }_i(x)={\int }_{X}f(x)d\eta \forall f\in C_b(X).$$

$$d_{1}(m,m^{\prime})=sup\Big{\{}\int_{X}f(x)\,dm(x)-\int_{X}f(x)\,dm^{\prime}(x)\ \Big{|}\ f:X\rightarrow\mathbb{R}\ \ \mbox{is 1-Lipschitz}\Big{\}},$$

$$d_1(m(t),m(s))\le C|t-s|,\forall t,s\in [0,T],$$

$$\lambda u(x)+(1-\lambda )u(y)-u(\lambda x+(1-\lambda )y)\le \lambda (1-\lambda )|x-y|\omega (|x-y|)$$

$$D^{-}u(x)$$

$$D^{+}u(x)$$

$$D^{+}u(x)=D^{-}u(x)=\{Du(x)\}.$$

$$u(y)-u(x)-⟨p+\lambda \nu (x),y-x⟩=u(y)-u(x)-⟨p,y-x⟩-\lambda ⟨\nu (x),y-x⟩\le o(|y-x|).$$

$$u(y)-u(x)-⟨p,y-x⟩\le |y-x|\omega (|y-x|)$$

$$D_x^{+}u(t,x):=\left\{\eta \in {\mathbb{R}}^n:\underset{h\to 0}{\mathrm{limsup}}\frac{u(t,x+h)-u(t,x)-⟨\eta ,h⟩}{h}\le 0\right\}.$$

$$T_{\overline{\Omega}}(x)=\Big{\{}\lim_{i\rightarrow\infty}\frac{x_{i}-x}{t_{i}}:x_{i}\in\Omega,x_{i}\rightarrow x,\ t_{i}\in\mathbb{R}_{+},\ t_{i}\downarrow 0\Big{\}}.$$

$$\displaystyle\mbox{if}\ \ \ x\in\Omega\ \$$

$$\text{mbox}ifx\in \partial \Omega$$

$$\partial^{\uparrow}u(x;\theta)=\limsup_{\tiny\begin{array}[]{c}h\rightarrow 0^{+}\\ \theta^{\prime}\rightarrow\theta\\ x+h\theta^{\prime}\in\overline{\Omega}\end{array}}\frac{u(x+h\theta^{\prime})-u(x)}{h}$$

$$\partial^{\downarrow}u(x;\theta)=\liminf_{\tiny\begin{array}[]{c}h\rightarrow 0^{+}\\ \theta^{\prime}\rightarrow\theta\\ x+h\theta^{\prime}\in\overline{\Omega}\end{array}}\frac{u(x+h\theta^{\prime})-u(x)}{h},$$

$$\partial^{+}_{\theta}u(x)=\lim_{\tiny\begin{array}[]{c}h\rightarrow 0^{+}\\ \theta^{\prime}\rightarrow\theta\\ x+h\theta^{\prime}\in\overline{\Omega}\end{array}}\frac{u(x+h\theta^{\prime})-u(x)}{h}$$

$${\partial }^{\uparrow }u(x;\theta )=\underset{p\in D^{+}u(x)}{\min }⟨p,\theta ⟩={\partial }^{\downarrow }u(x;\theta ).$$

Full text

Keywords: semiconcave functions, state constraints, mean field games, viscosity solutions.

MSC Subject classifications: 49L25, 49N60, 49K40,49N90.

1 Introduction

The theory of Mean Field Games (MFG) has been developed simultaneously by Lasry and Lions ([26], [27], [28]) and by Huang, Malhamé and Caines ([22], [23]) in order to study differential games with an infinite number of rational players in competition. The simplest MFG models lead to systems of partial differential equations involving two unknown functions: the value function $u=u(t,x)$ of the optimal control problem that a typical player seeks to solve, and the time-dependent density $m=(m(t))$ of the population of players:

[TABLE]

In the largest part of the literature, this system is studied in the full space $(0,T)\times\mathbb{R}^{n}$ (or with space periodic data) assuming the time horizon $T$ to be finite. The system can also be associated with Neumann boundary conditions for the $u$-component, which corresponds to reflected dynamics for the players.

The aim of this paper is to study system (1.1) when players are confined in the closure of a bounded domain $\Omega\subset\mathbb{R}^{n}$—a set-up that arises naturally in applications. For instance, MFG appearing in macroeconomic models (the so-called heterogeneous agent models) almost always involve constraints on the state variables. Indeed, one could even claim that state constraints play a central role in the analysis since they explain heterogeneity in the economy: see, for instance, Huggett’s model of income and wealth distribution ([1, 2]). It is also very natural to introduce constraints in pedestrians MFG models, although this variant of the problem has so far been discussed just in an informal way. Here again, constraints are important to explain the behavior of crowds and it is largely believed that the analysis of constrained problems should help regulating traffic: see, for instance, [16, 19] on related issues. However, despite their relevance to applications, a general approach to MFG with state constraints seems to be missing up to now. To the best of our knowledge, the first reference on this subject is the analysis of Huggett’s model in [2]. Other contributions are [8] and [9], on which we comment below.

One of the main issues in the analysis of MFG models with state constraints is the interpretation of system (1.1). If, on the one hand, the meaning of the Hamilton-Jacobi equation associated with the underlying optimal control problem is well understood (see [29, 30] and [12, 24]), on the other hand this is not the case for the continuity equation. This fact is due to several reasons: first, in contrast with unconstrained problems, one cannot expect $m(t)$ to be absolutely continuous with respect to the Lebesgue measure, in general. In fact, for Huggett’s model ( [2]) measure $m$ always develops a singular part at the boundary of $\Omega$. Moreover, solutions of Hamilton-Jacobi equations fail to be of class $C^{1}$, in general. Thus, the gradient $Du(t,x)$—even when it exists—may develop discontinuities. In addition, the meaning of $Du(t,x)$, when the point $x$ belongs to the boundary of the domain, is totally unclear. For all these reasons, the interpretation of the continuity equation is problematic.

To overcome the above difficulties, the first two authors of this paper introduced in [8] the notion of relaxed MFG equilibrium. Such an equilibrium is not defined in terms of the MFG system, but follows instead the so-called Lagrangian formulation (see [4], [5], [6], [7], [14], [15]). The main result of [8] is the existence of MFG equilibria, which holds under fairly general assumptions. In the same paper, the uniqueness of the solution is also derived under an adapted version of the Lasry-Lions monotonicity condition ([28]). Once the existence of relaxed equilibria is ensured, the next issue to address should be regularity. In [8], the notion of solution was very general and yields a value function $u$ and a flow of measures $m=(m(t))$ which are merely continuous. However, in [9], we have shown how to improve the construction in [8] to obtain more regular solutions, that is, pairs $(u,m)$ such $u$ is Lipschitz on $[0,T]\times\overline{\Omega}$ and the flow of measures $m:[0,T]\to\mathcal{P}(\overline{\Omega})$ is Lipschitz with respect to the Kantorovich-Rubinstein metric on $\mathcal{P}(\overline{\Omega})$, the space of probability measures on $\overline{\Omega}$.

However, [8] and [9] leave the open problem of establishing a suitable sense in which the MFG system is satisfied. Such a necessity justifies the search for further regularity properties of generalized solutions. Indeed, despite its importance in nonsmooth analysis, Lipschitz regularity does not suffice to give the MFG system—in particular, the continuity equation—a satisfactory interpretation. A more useful regularity property, in connection with Hamilton-Jacobi equations, is known to be semiconcavity (see, for instance, [10]). However, even for problems in the calculus of variations or optimal control, very few semiconcavity results are available in the presence of state constraints and, in fact, it was so far known that global linear semiconcavity should not be expected ([13]).

Surprisingly, in this paper we show that global semiconcavity—with a fractional modulus—does hold true. More precisely, denoting by $u$ the value function of a constrained problem in the calculus of variation (see Subsection 2.2), we prove that

[TABLE]

for allt $(t,x)\in[0,T)\times\partial\Omega$ and $h$, $\sigma$ small enough (see Corollary 3.2). Actually, the above semiconcavity estimate is obtained as a corollary of a sensitivity relation (Theorem 3.1), for the proof of which key tools are provided by necessary optimality conditions in the formulation that was introduced in [9].

Using the above property, in this paper we give—for the first time—an interpretation of system (1.1) in the presence of state constraints, which goes as follows: if $(u,m)$ is a mild solution of the constrained MFG problem (see Definition 4.2 below), then—as expected—$u$ is a constrained viscosity solution of

[TABLE]

(in the sense of [29, 30]). Moreover—and this is our main result—there exists a bounded continuous vector field $V:(0,T)\times\overline{\Omega}\rightarrow\mathbb{R}^{n}$ such that $m$ satisfies the continuity equation

[TABLE]

in the sense of distributions. The vector field $V$ is related to $u$ in the following way: on the one hand, at any point $(t,x)$ such that $x$ is an interior point belonging to the support of $m(t)$, $u$ is differentiable and

[TABLE]

On the other hand, if $x$ is a boundary point on the support of $m(t)$, then one has that

[TABLE]

where $D^{\tau}_{x}u(t,x)$ is the tangential component of all elements of the superdifferential of $u$ and $\lambda_{+}(t,x)$ is the unique real number $\lambda$ for which $-D_{p}H(x,D^{\tau}_{x}u(t,x)+\lambda\nu(x))$ is tangential to $\Omega$ at $x$ (see Remark 4.6). We also prove that $u$ has time derivative at $(t,x)$ and $D^{\tau}_{x}u(t,x)+\lambda_{+}(t,x)\nu(x)$ can be interpreted as the correct space derivative of $u$ at $(t,x)$. For instance, we show that the Hamilton-Jacobi equation holds with an equality at any such point, that is,

[TABLE]

as is the case for points of differentiability of the solution in the interior. The continuity of the vector field $V$ is directly related to the semiconcavity of $u$. Such a rigidity result is reminiscent of the reformulation of the notion of viscosity solution of Hamilton-Jacobi equation with state-constraints in terms of flux-limited solutions, as described in the recent papers by Imbert and Monneau [25] and Guerand [21].

This paper is organized as follows. In Section 2, we introduce the notation and some preliminary results. In Section 3, under suitable assumptions, we deduce the local fractional semiconcavity of the value function associated to a variational problem with state constraints. In Section 4, we apply our semiconcavity result to constrained MFG problems. In particular, we give a new interpretation of the MFG system in the presence of state constraints. Finally, in the Appendix, we prove a technical result on directional derivatives.

2 Preliminaries

Throughout this paper we denote by $|\cdot|$ and $\langle\cdot\rangle$ , respectively, the Euclidean norm and scalar product in $\mathbb{R}^{n}$. We denote by $B_{R}$ the ball of radius $R>0$ and center [math]. Let $A\in\mathbb{R}^{n\times n}$ be a matrix. We denote by $||\cdot||$ the norm of $A$ defined as follows

[TABLE]

For any subset $S\subset\mathbb{R}^{n}$, $\overline{S}$ stands for its closure, $\partial S$ for its boundary, and $S^{c}$ for $\mathbb{R}^{n}\setminus S$. We denote by $\mathbf{1}_{S}:\mathbb{R}^{n}\rightarrow\{0,1\}$ the characteristic function of $S$, i.e.,

[TABLE]

We write $AC(0,T;\mathbb{R}^{n})$ for the space of all absolutely continuous $\mathbb{R}^{n}$-valued functions on $[0,T]$, equipped with the uniform norm $||\gamma||_{\infty}={\rm sup}_{[0,T]}\ |\gamma(t)|$. We observe that $AC(0,T;\mathbb{R}^{n})$ is not a Banach space.

Let $U$ be an open subset of $\mathbb{R}^{n}$. $C(U)$ is the space of all continuous functions on $U$ and $C_{b}(U)$ is the space of all bounded continuous functions on $U$. $C^{k}(U)$ is the space of all functions $\phi:U\rightarrow\mathbb{R}$ that are k-times continuously differentiable. Let $\phi\in C^{1}(U)$. The gradient vector of $\phi$ is denoted by $D\phi=(D_{x_{1}}\phi,\cdots,D_{x_{n}}\phi)$, where $D_{x_{i}}\phi=\frac{\partial\phi}{\partial x_{i}}$. Let $\phi\in C^{k}(U)$ and let $\alpha=(\alpha_{1},\cdots,\alpha_{n})\in\mathbb{N}^{n}$ be a multiindex. We define $D^{\alpha}\phi=D^{\alpha_{1}}_{x_{1}}\cdots D^{\alpha_{n}}_{x_{n}}\phi$. $C^{k}_{b}(U)$ is the space of all function $\phi\in C^{k}(U)$ and such that

[TABLE]

Throughout the paper, $\Omega$ is a bounded open subset of $\mathbb{R}^{n}$ with $C^{2}$ boundary. $C^{1,1}(\overline{\Omega})$ is the space of all the functions $C^{1}$ in a neighborhood $U$ of $\Omega$ and with locally Lipschitz continuous first order derivatives.

The distance function from $\overline{\Omega}$ is the function $d_{\Omega}:\mathbb{R}^{n}\rightarrow[0,+\infty[$ defined by

[TABLE]

We define the oriented boundary distance from $\partial\Omega$ by

[TABLE]

We recall that, since the boundary of $\Omega$ is of class $C^{2}$, there exists $\rho_{0}>0$ such that

[TABLE]

Throughout the paper, we suppose that $\rho_{0}$ is fixed so that (2.1) holds.

Let $X$ be a separable metric space. $C_{b}(X)$ is the space of all bounded continuous functions on $X$. We denote by $\mathscr{B}(X)$ the family of the Borel subset of $X$ and by $\mathcal{P}(X)$ the family of all Borel probability measures on $X$. The support of $\eta\in\mathcal{P}(X)$, $supp(\eta)$, is the closed set defined by

[TABLE]

We say that a sequence $(\eta_{i})\subset\mathcal{P}(X)$ is narrowly convergent to $\eta\in\mathcal{P}(X)$ if

[TABLE]

We denote by $d_{1}$ the Kantorovich-Rubinstein distance on $X$, which—when $X$ is compact—can be characterized as follows

[TABLE]

for all $m,m^{\prime}\in\mathcal{P}(X)$.

We write ${\rm Lip}(0,T;\mathcal{P}(\overline{\Omega}))$ for the space of all maps $m:[0,T]\rightarrow\mathcal{P}(\overline{\Omega})$ that are Lipschitz continuous with respect to $d_{1}$, i.e.,

[TABLE]

for some constant $C\geq 0$. We denote by ${\rm Lip}(m)$ the smallest constant that verifies (2.3).

2.1 Semiconcave functions and generalized gradients

Definition 2.1**.**

We say that $\omega:\mathbb{R}_{+}\rightarrow\mathbb{R}_{+}$ is a modulus if it is a nondecreasing upper semicontinuous function such that $\displaystyle\lim_{r\rightarrow 0^{+}}\omega(r)=0$.

Definition 2.2**.**

Let $\omega:\mathbb{R}_{+}\rightarrow\mathbb{R}_{+}$ be a modulus. We say that a function $u:\overline{\Omega}\rightarrow\mathbb{R}$ is semiconcave with modulus $\omega$ if

[TABLE]

for any pair $x$, $y\in\overline{\Omega}$, such that the segment $[x,y]$ is contained in $\overline{\Omega}$ and for any $\lambda\in[0,1]$. We call $\omega$ a modulus of semiconcavity for $u$ in $\overline{\Omega}$.

A function $u$ is called semiconvex in $\overline{\Omega}$ if $-u$ is semiconcave.

When the right-side of (2.4) is replaced by a term of form $C|x-y|^{2}$ we say that $u$ is semiconcave with linear modulus.

For any $x\in\overline{\Omega}$, the sets

[TABLE]

are called, respectively, the (Fréchet) subdifferential and superdifferential of $u$ at $x$.

We note that if $x\in\Omega$ then, $D^{+}u(x)$, $D^{-}u(x)$ are both nonempty if and only if $u$ is differentiable in $x$. In this case we have that

[TABLE]

Proposition 2.1**.**

Let $u$ be a real-valued function defined on $\overline{\Omega}$. Let $x\in\partial\Omega$ and let $\nu(x)$ be the outward unit normal vector to $\partial\Omega$ in $x$. If $p\in D^{+}u(x)$, $\lambda\leq 0$ then $p+\lambda\nu(x)$ belongs to $D^{+}u(x)$ for all $\lambda\leq 0$.

Proof.

Let $x\in\partial\Omega$ and let $\nu(x)$ be the outward unit normal vector to $\partial\Omega$ in $x$. Let $p\in D^{+}u(x)$. Let us take $\lambda\leq 0$ and $y\in\overline{\Omega}$. Since $p\in D^{+}u(x)$ and $\lambda\leq 0$, one has that

[TABLE]

Hence, $p+\lambda\nu(x)$ belongs to $D^{+}u(x)$. ∎

$D^{+}u(x)$, $D^{-}u(x)$ can be described in terms of test functions as shown in the next lemma.

Proposition 2.2**.**

Let $u\in C(\overline{\Omega})$, $p\in\mathbb{R}^{n}$, and $x\in\overline{\Omega}$. Then the following properties are equivalent:

1. (a)

$p\in D^{+}u(x)$* (resp. $p\in D^{-}u(x)$);* 2. (b)

$p=D\phi(x)$* for some function $\phi\in C^{1}(\mathbb{R}^{n})$ touching $u$ from above (resp. below);* 3. (c)

$p=D\phi(x)$* for some function $\phi\in C^{1}(\mathbb{R}^{n})$ such that $f-\phi$ attains a local maximum (resp. minimum) at $x$.*

In the proof of Proposition 2.2 it is possible to follow the same method of [10, Proposition 3.1.7]. The following statements are straightforward extensions to the constrained case of classical results: we refer again to [10] for a proof.

Proposition 2.3**.**

Let $u:\overline{\Omega}\rightarrow\mathbb{R}$ be semiconcave with modulus $\omega$ and let $x\in\overline{\Omega}$. Then, a vector $p\in\mathbb{R}^{n}$ belongs to $D^{+}u(x)$ if and only if

[TABLE]

for any point $y\in\overline{\Omega}$ such that $[y,x]\subset\overline{\Omega}$.

A direct consequence of Proposition 2.3 is the following result.

Proposition 2.4**.**

Let $u:\overline{\Omega}\rightarrow\mathbb{R}$ be a semiconcave function with modulus $\omega$ and let $x\in\overline{\Omega}$. Let $\{x_{k}\}\subset\overline{\Omega}$ be a sequence converging to $x$ and let $p_{k}\in D^{+}u(x_{k})$. If $p_{k}$ converges to a vector $p\in\mathbb{R}^{n}$, then $p\in D^{+}u(x)$.

Remark 2.1*.*

If the function $u$ depends on $(t,x)\in(0,T)\times\overline{\Omega}$, for some $T>0$, it is natural to consider the generalized partial differentials with respect to $x$ as follows

[TABLE]

2.1.1 Directional derivatives

Let $\Omega$ be a bounded open subset of $\mathbb{R}^{n}$ with $C^{2}$ boundary. Let us first recall the definition of contingent cone.

Definition 2.3**.**

Let $x\in\overline{\Omega}$ be given. The contingent cone (or Bouligand’s tangent cone) to $\Omega$ at $x$ is the set

[TABLE]

Remark 2.2*.*

Since $\Omega$ is a bounded open subset of $\mathbb{R}^{n}$ with $C^{2}$ boundary, then

[TABLE]

where $\nu(x)$ is the outward unit normal vector to $\partial\Omega$ in $x$.

Definition 2.4**.**

Let $x\in\overline{\Omega}$ and $\theta\in T_{\overline{\Omega}}(x)$. The upper and lower Dini derivatives of $u$ at $x$ in direction $\theta$ are defined as

[TABLE]

and

[TABLE]

*respectively.

The one-sided derivative of $u$ at $x$ in direction $\theta$ is defined as*

[TABLE]

Let $x\in\partial\Omega$ and let $\nu(x)$ be the outward unit normal vector to $\partial\Omega$ in $x$. In the next result, we show that any semiconcave function admits one-sided derivative in all $\theta$ such that $\langle\theta,\nu(x)\rangle\leq 0$.

Lemma 2.1**.**

Let $u:\overline{\Omega}\rightarrow\mathbb{R}$ be Lipschitz continuous and semiconcave with modulus $\omega$ in $\overline{\Omega}$. Let $x\in\partial\Omega$ and let $\nu(x)$ be the outward unit normal vector to $\partial\Omega$ in $x$. Then, for any $\theta\in\mathbb{R}^{n}$ such that $\langle\theta,\nu(x)\rangle\leq 0$ one has that

[TABLE]

For reader’s convenience the proof is given in Appendix.

Remark 2.3*.*

We observe that Lemma 2.1 also holds when $x\in\Omega$. In this case, (2.15) is a direct consequence of [11, Theorem 4.5].

Fix $x\in\partial\Omega$ and let $\nu(x)$ be the outward unit normal vector to $\partial\Omega$ in $x$. All $p\in D^{+}_{x}u(x)$ can be written as

[TABLE]

where $p^{\nu}$ is the normal component of $p$, i.e.,

[TABLE]

and $p^{\tau}$ is the tangential component of $p$ which satisfies

[TABLE]

Proposition 2.5**.**

Let $x\in\partial\Omega$ and let $\nu(x)$ be the outward unit normal vector to $\partial\Omega$ in $x$. Let $u:\overline{\Omega}\rightarrow\mathbb{R}$ be Lipschitz continuous and semiconcave with modulus $\omega$. Then,

[TABLE]

where

[TABLE]

Proof.

Let $x\in\partial\Omega$ and let $\nu(x)$ be the outward unit normal vector to $\partial\Omega$ in $x$. By Lemma 2.1 we obtain that

[TABLE]

This completes the proof. ∎

2.2 Necessary conditions

Let $\Omega\subset\mathbb{R}^{n}$ be a bounded open set with $C^{2}$ boundary. Let $\Gamma$ be the metric subspace of $AC(0,T;\mathbb{R}^{n})$ defined by

[TABLE]

For any $(t,x)\in[0,T]\times\overline{\Omega}$, we set

[TABLE]

Given $(t,x)\in[0,T]\times\overline{\Omega}$, we consider the constrained minimization problem

[TABLE]

We denote by $\Gamma^{*}_{t}[x]$ the set of solutions of (2.18), that is

[TABLE]

Let $U\subset\mathbb{R}^{n}$ be an open set such that $\overline{\Omega}\subset U$. We assume that $f:[0,T]\times U\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ and $g:U\rightarrow\mathbb{R}$ satisfy the following conditions.

1. (g1)

$g\in C^{1}_{b}(U)$ 2. (f0)

f\in C\big{(}[0,T]\times U\times\mathbb{R}^{n}\big{)} and for all $t\in[0,T]$ the function $(x,v)\longmapsto f(t,x,v)$ is differentiable. Moreover, $D_{x}f$, $D_{v}f$ are continuous on $[0,T]\times U\times\mathbb{R}^{n}$ and there exists a constant $M\geq 0$ such that

[TABLE] 3. (f1)

For all $t\in[0,T]$ the map $(x,v)\longmapsto D_{v}f(t,x,v)$ is continuously differentiable and there exists a constant $\mu\geq 1$ such that

[TABLE]

for all $(t,x,v)\in[0,T]\times U\times\mathbb{R}^{n}$, where $I$ denotes the identity matrix. 4. (f2)

For all $(x,v)\in U\times\mathbb{R}^{n}$ the function $t\longmapsto f(t,x,v)$ and the map $t\longmapsto D_{v}f(t,x,v)$ are Lipschitz continuous. Moreover there exists a constant $\kappa\geq 0$ such that

[TABLE]

for all $t$, $s\in[0,T]$, $x\in U$, $v\in\mathbb{R}^{n}$.

Remark 2.4*.*

By classical results in the calculus of variation (see, e.g., [18, Theorem 11.1i]), there exists at least one minimizer of (2.18) in $\Gamma$ for any fixed point $x\in\overline{\Omega}$.

We denote by $H:[0,T]\times U\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ the Hamiltonian

[TABLE]

In the next result we show the necessary conditions for our problem (for a proof see [9]).

Theorem 2.1**.**

Suppose that (g1), (f0)-(f2) hold. For any $x\in\overline{\Omega}$ and any $\gamma\in\Gamma^{*}_{t}[x]$ the following holds true.

1. (i)

$\gamma$* is of class $C^{1,1}([t,T];\overline{\Omega})$.* 2. (ii)

There exist:

1. (a)

a Lipschitz continuous arc $p:[t,T]\rightarrow\mathbb{R}^{n}$, 2. (b)

a constant $\nu\in\mathbb{R}$ such that

[TABLE]

which satisfy the adjoint system

[TABLE]

and

[TABLE]

where $\Lambda:[t,T]\times\Sigma_{\rho_{0}}\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ is a bounded continuous function independent of $\gamma$ and $p$.

Moreover,

1. (iii)

the following estimate holds

[TABLE]

*where $L^{\star}=L^{\star}(\mu,M^{\prime},M,\kappa,T,||Dg||_{\infty},||g||_{\infty})$. *

Remark 2.5*.*

The (feedback) function $\Lambda$ in (2.24) can be computed explicitly, see [9, Remark 3.4].

Following the terminology of control theory, given an optimal trajectory $\gamma$, any arc $p$ satisfying (2.24) and (2.25) is called a dual arc associated with $\gamma$.

Remark 2.6*.*

Following (2.25) and the regularity of $H$, the derivative of the optimal trajectory $\gamma$ can be expressed in function of the dual arc:

[TABLE]

3 Sensitivity relations and fractional semiconcavity

In this section, we investigate further the optimal control problem with state constraints introduced in Subsection 2.2 and show our main semiconcavity result of the value function. For this, we have to enforce the assumptions on the data.

Suppose that $f:[0,T]\times U\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ satisfies the assumptions (f0)-(f2) and

1. (f3)

for all $s\in[0,T]$, for all $x\in U$ and for all $v$, $w\in B_{R}$, there exists a constant $C(R)\geq 0$ such that

[TABLE] 2. (f4)

for any $R>0$ the map $x\longmapsto f(t,x,v)$ is semiconcave with linear modulus $\omega_{R}$, i.e., for any $(t,v)\in[0,T]\times B_{R}$ one has that

[TABLE]

for any pair $x$, $y\in U$ such that the segment $[x,y]$ is contained in $U$ and for any $\lambda\in[0,1]$.

Moreover, we assume that $g:U\rightarrow\mathbb{R}$ satisfies (g1). Define $u:[0,T]\times\overline{\Omega}\rightarrow\mathbb{R}$ as the value function of the minimization problem (2.18), i.e.,

[TABLE]

Remark 3.1*.*

We observe that the value function $u$ is Lipschitz continuous in $[0,T]\times\overline{\Omega}$ (see [9, Proposition 4.1]).

Under the above assumptions on $\Omega$, $f$ and $g$ the sensitivity relations for our problem can be stated as follows.

Theorem 3.1**.**

For any $\varepsilon>0$ there exists a constant $c_{\varepsilon}\geq 1$ such that for any $(t,x)\in[0,T-\varepsilon]\times\overline{\Omega}$ and for any $\gamma\in\Gamma^{*}_{t}[x]$, denoting by $p\in{\rm Lip}(t,T,\mathbb{R}^{n})$ a dual arc associated with $\gamma$, one has that

[TABLE]

for all $h\in\mathbb{R}^{n}$ such that $x+h\in\overline{\Omega}$, and for all $\sigma\in\mathbb{R}$ such that $0\leq t+\sigma\leq T-\varepsilon$.

Corollary 3.1**.**

Let $\Omega\subset\mathbb{R}^{n}$ be a bounded open set with $C^{2}$ boundary. Let $(t,x)\in[0,T)\times\overline{\Omega}$. Let $\gamma\in\Gamma^{*}_{t}[x]$ and let $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$ be a dual arc associated with $\gamma$. Then,

[TABLE]

A direct consequence of Theorem 3.1 is that $u$ is a semiconcave function.

Corollary 3.2**.**

Let $\Omega\subset\mathbb{R}^{n}$ be a bounded open set with $C^{2}$ boundary. The value function (3.2) is locally semiconcave with modulus $\omega(r)=Cr^{\frac{1}{2}}$ in $(0,T)\times\overline{\Omega}$.

Proof.

Let $\varepsilon>0$ and let $(t,x)\in[0,T-\varepsilon]\times\partial\Omega$. Let $\gamma\in\Gamma^{*}_{t}[x]$ and let $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$ be a dual arc assosiated with $\gamma$. Let $h\in\mathbb{R}^{n}$ be such that $x+h$, $x-h\in\overline{\Omega}$. Let $\sigma>0$ be such that $0\leq t-\sigma\leq t\leq t+\sigma\leq T-\varepsilon$. By Theorem 3.1, there exists a constant $c_{\varepsilon}\geq 1$ such that

[TABLE]

Inequality (3) yields (2.4) for $\lambda=\frac{1}{2}$. By [10, Theorem 2.1.10] this is enough to conclude that $u$ is semiconcave, because $u$ is continuous on $(0,T)\times\overline{\Omega}$. ∎

3.1 Proof of Theorem 3.1

It is convenient to divide the proof of Theorem 3.1 in several lemmas. First, we show that $u$ is semiconcave with modulus $\omega(r)=Cr^{\frac{1}{2}}$ in $\overline{\Omega}$.

Lemma 3.1**.**

For any $\varepsilon>0$ there exists a constant $c_{\varepsilon}\geq 1$ such that for any $(t,x)\in[0,T-\varepsilon]\times\overline{\Omega}$ and for any $\gamma\in\Gamma^{*}_{t}[x]$, denoting by $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$ a dual arc associated with $\gamma$, one has that

[TABLE]

*for all $h\in\mathbb{R}^{n}$ such that $x+h\in\overline{\Omega}$. *

Proof.

Let $\varepsilon>0$ and let $(t,x)\in[0,T-\varepsilon]\times\overline{\Omega}$. Let $\gamma\in\Gamma^{*}_{t}[x]$ and let $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$ be a dual arc associated with $\gamma$. Let $h\in\mathbb{R}^{n}$ be such that $x+h\in\overline{\Omega}$. Let $r\in(0,\varepsilon/2]$. We denote by $\gamma_{h}$ the trajectory defined by

[TABLE]

We observe that, if $|h|$ is small enough, then $d_{\Omega}(\gamma_{h}(s))\leq\rho_{0}$ for all $s\in[t,t+r]$, where $\rho_{0}$ is defined in (2.1). Indeed,

[TABLE]

Thus, we have that $d_{\Omega}(\gamma_{h}(s))\leq\rho_{0}$ for all $s\in[t,T]$ and for $|h|\leq\rho_{0}$. Denote by $\widehat{\gamma}_{h}$ the projection of $\gamma_{h}$ on $\overline{\Omega}$, i.e.,

[TABLE]

By construction $\widehat{\gamma}_{h}\in AC(0,T;\mathbb{R}^{n})$ and for $s=t$ one has that $\widehat{\gamma}_{h}(t)=x+h$. Moreover,

[TABLE]

Indeed,

[TABLE]

for all $s\in[t,T]$. Furthermore, recalling [8, Lemma 3.1], we have that

[TABLE]

for a.e. $s\in[t,t+r]$. Since $\gamma$ is an optimal trajectory for $u$ at $(t,x)$, by the dynamic programming principle, and by the definition of $\widehat{\gamma}_{h}$ we have that

[TABLE]

Integrating by parts, $\langle p(t),h\rangle$ can be rewritten as

[TABLE]

Recalling that $p$ satisfies (2.24) and (2.25), we deduce that

[TABLE]

Therefore, using (3.1), (3.1) can be rewritten as

[TABLE]

Using the assumptions (f1), (f3) and (f4) in (3.1) we have that

[TABLE]

for some constant $c\geq 0$. By (3.6) we observe that

[TABLE]

Moreover, recalling (3.7) one has that

[TABLE]

By [8, Lemma 3.1] we obtain that

[TABLE]

Recalling that $\gamma_{h}(t)$, $\gamma_{h}(t+r)\in\overline{\Omega}$, we observe that

[TABLE]

where $(s_{i},t_{i})\cap(s_{j},t_{j})=\emptyset$ for all $i\neq j$. Hence,

[TABLE]

Integrating by parts, we get

[TABLE]

Owing to $d_{\Omega}(\gamma_{h}(s_{i}))=d_{\Omega}(\gamma_{h}(t_{i}))=0$ for $i\in\mathbb{N}$, $d_{\Omega}(\gamma_{h}(t+r))=d_{\Omega}(\gamma(t+r))=0$ and $d_{\Omega}(\gamma_{h}(t))=d_{\Omega}(\gamma(t))=0$, one has that

[TABLE]

From now on, we assume that $|h|\leq r$. Then, recalling that $\gamma\in C^{1,1}([t,T];\overline{\Omega})$, one has that

[TABLE]

where the constant $C$ does not dependent on $h$ and $r$. Hence, we deduce that

[TABLE]

and so

[TABLE]

Moreover, we have that

[TABLE]

and

[TABLE]

for some constant $C\geq 0$ independent on $h$ and $r$. Since $\langle D^{2}b_{\Omega}(x),Db_{\Omega}(x)\rangle=0$ $\forall x\in\mathbb{R}^{n}$ one has that

[TABLE]

Hence,

[TABLE]

Moreover, using Young’s inequality, (3.14) and (3.11), we deduce that

[TABLE]

where $c$ is a constant independent of $h$ and $r$. Moreover, since

[TABLE]

and using (3.14) and (3.1) we have that

[TABLE]

Thus, choosing $r=|h|^{\frac{1}{2}}$ in (3.16), we conclude that (3.5) holds. Note that the constraint on the size of $|h|$—namely $|h|\leq\rho_{0}$, $|h|\leq r$ and $|h|=r^{2}\leq(\varepsilon/2)^{2}$—depends on $\varepsilon$ but not on $(t,x)$. This constraint can be removed by changing the constant $c_{\varepsilon}$ if necessary. This completes the proof. ∎

Lemma 3.2**.**

For any $\varepsilon>0$ there exists a constant $c_{\varepsilon}\geq 1$ such that for all $(t,x)\in[0,T-\varepsilon]\times\overline{\Omega}$ and for all $\gamma\in\Gamma^{*}_{t}[x]$, denoting by $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$ a dual arc associated with $\gamma$, one has that

[TABLE]

for any $h\in\mathbb{R}^{n}$ such that $x+h\in\overline{\Omega}$, and for any $\sigma>0$ such that $0\leq t+\sigma\leq T-\varepsilon$.

Proof.

Let $\varepsilon>0$ and let $(t,x)\in[0,T-\varepsilon]\times\overline{\Omega}$. Let $\sigma>0$ be such that $0\leq t\leq t+\sigma\leq T-\varepsilon$ and let $h\in\mathbb{R}^{n}$ be such that $x+h\in\overline{\Omega}$. Let $\gamma\in\Gamma^{*}_{t}[x]$ and let $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$ be a dual arc associated with $\gamma$. By dynamical programming principle one has that

[TABLE]

By Lemma 3.1 there exists a constant $c_{\varepsilon}\geq 1$ such that

[TABLE]

By Theorem 2.1, we have that

[TABLE]

Since $\gamma\in C^{1,1}([t,T];\overline{\Omega})$, $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$, we deduce that

[TABLE]

Using (3.18) and (3.1) in (3.17), one has that

[TABLE]

By the definition of $H$ we have that

[TABLE]

Since $\gamma\in C^{1,1}([t,T];\overline{\Omega})$ and $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$, we get

[TABLE]

where $C$ is a positive constant independent on $h$ and $\sigma$.

Using (3.1) in (3.1) we conclude that

[TABLE]

This completes the proof. ∎

Lemma 3.3**.**

For any $\varepsilon>0$ there exists a constant $c_{\varepsilon}\geq 1$ such that for any $(t,x)\in[0,T-\varepsilon]\times\overline{\Omega}$ and for any $\gamma\in\Gamma^{*}_{t}[x]$, denoting by $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$ a dual arc associated with $\gamma$, one has that

[TABLE]

for any $h\in\mathbb{R}^{n}$ such that $x+h\in\overline{\Omega}$, and for any $\sigma>0$ such that $0\leq t-\sigma\leq T-\varepsilon$.

Proof.

Let $\varepsilon>0$ and let $(t,x)\in[0,T-\varepsilon]\times\overline{\Omega}$. Let $\sigma>0$ be such that $0\leq t-\sigma\leq T-\sigma\leq T-\varepsilon$ and let $h\in\mathbb{R}^{n}$ be such that $x+h\in\overline{\Omega}$. Let $\gamma\in\Gamma^{*}_{t}[x]$ and let $p\in{\rm Lip}(t,T;\mathbb{R}^{n})$ be a dual arc associated with $\gamma$. We define $\gamma_{h}$ and $\widehat{\gamma}_{h}$ as in the proof of Lemma 3.1 for $r=(\sigma+|h|)^{\frac{1}{2}}$. By (3.6) and (3.14) we have, for any $s\in[t,T]$,

[TABLE]

We finally set

[TABLE]

and note that $\widehat{\gamma}_{h,\sigma}(t-\sigma)=\widehat{\gamma}_{h}(t)=x+h$. By the dynamic programming principle we obtain

[TABLE]

We start with the estimate of the first term on the right-hand side of (3.24). By using the two inequalities in (3.23) and the regularity of $f$, we have

[TABLE]

Therefore, recalling that $D_{v}f(s,\gamma(s),\dot{\gamma}(s))=-p(s)$, $p$ is uniformly Lipschitz continuous, and $\dot{\widehat{\gamma}}_{h}$ is bounded we obtain

[TABLE]

On the other hand, the second term in the right-hand side of (3.24) can be estimated by using Lemma 3.1 and the first inequality in (3.23):

[TABLE]

Combining (3.24), (3.25) and (3.26), we obtain that

[TABLE]

Then (2.25) and the optimality of $\gamma$ imply that

[TABLE]

where we used again the Lipschitz continuity of $s\to H(s,\gamma(s),p(s))$. This completes the proof. ∎

We observe that Theorem 3.1 is a direct consequence of Lemma 3.2 and Lemma 3.3.

4 The Mean Field Game system: from mild to pointwise solutions

In this section we return to mean field games with state constraints. Our aim is to give a meaning to system (1.1). For this, we first recall the notion of constrained MFG equilibria and mild solutions of the constrained MFG problem, as introduced in [8]. Then, we investigate further regularity properties of the value function $u$. We conclude by the interpretation of the continuity equation for $m$.

4.1 Assumptions

Let $\mathcal{P}(\overline{\Omega})$ be the set of all Borel probability measures on $\overline{\Omega}$ endowed with the Kantorovich-Rubinstein distance $d_{1}$ defined in (2.2). Let $U$ be an open subset of $\mathbb{R}^{n}$ and such that $\overline{\Omega}\subset U$. Assume that $F:U\times\mathcal{P}(\overline{\Omega})\rightarrow\mathbb{R}$ and $G:U\times\mathcal{P}(\overline{\Omega})\rightarrow\mathbb{R}$ satisfy the following hypotheses.

1. (D1)

For all $x\in U$, the functions $m\longmapsto F(x,m)$ and $m\longmapsto G(x,m)$ are Lipschitz continuous, i.e., there exists $\kappa\geq 0$ such that

[TABLE]

for any $m_{1}$, $m_{2}\in\mathcal{P}(\overline{\Omega})$. 2. (D2)

For all $m\in\mathcal{P}(\overline{\Omega})$, the functions $x\longmapsto G(x,m)$ and $x\longmapsto F(x,m)$ belong to $C^{1}_{b}(U)$. Moreover

[TABLE] 3. (D3)

For all $m\in\mathcal{P}(\overline{\Omega})$, the function $x\longmapsto F(x,m)$ is semiconcave with linear modulus, uniformly with respect to $m$.

Let $L:U\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ be a function that satisfies the following assumptions.

1. (L0)

$L\in C^{1}(U\times\mathbb{R}^{n})$ and there exists a constant $M\geq 0$ such that

[TABLE] 2. (L1)

$D_{v}L$ is differentiable on $U\times\mathbb{R}^{n}$ and there exists a constant $\mu\geq 1$ such that

[TABLE]

for all $(x,v)\in U\times\mathbb{R}^{n}$. 3. (L2)

For all $x\in U$ and for all $v$, $w\in B_{R}$, there exists a constant $C(R)\geq 0$ such that

[TABLE] 4. (L3)

For any $R>0$ the map $x\longmapsto L(x,v)$ is semiconcave with linear modulus, uniformly with respect to $v\in B_{R}$.

Remark 4.1*.*

For any given $m\in{\rm Lip}(0,T;\mathcal{P}(\overline{\Omega}))$, the function $f(t,x,v):=L(x,v)+F(x,m(t))$ satisfies assumptions (f0)-(f4).

We denote by $H:U\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ the Hamiltonian

[TABLE]

The assumptions on $L$ imply that $H$ satisfies the following conditions.

1. (H0)

$H\in C^{1}(U\times\mathbb{R}^{n})$ and there exists a constant $M^{\prime}\geq 0$ such that

[TABLE] 2. (H1)

$D_{p}H$ is differentiable on $U\times\mathbb{R}^{n}$ and satisfies

[TABLE]

where $\mu$ is the constant in (L1) and $C(\mu,M^{\prime})$ depends only on $\mu$ and $M^{\prime}$. 3. (H2)

For all $x\in U$ and for all $p$, $q\in B_{R}$, there exists a constant $C(R)\geq 0$ such that

[TABLE] 4. (H3)

For any $R>0$ the map $x\longmapsto H(x,p)$ is semiconvex with linear modulus, uniformly with respect to $p\in B_{R}$.

4.2 Constrained MFG equilibria and mild solutions

For any $t\in[0,T]$, we denote by $e_{t}:\Gamma\to\overline{\Omega}$ the evaluation map defined by

[TABLE]

For any $\eta\in\mathcal{P}(\Gamma)$, we define

[TABLE]

For any fixed $m_{0}\in\mathcal{P}(\overline{\Omega})$, we denote by ${\mathcal{P}}_{m_{0}}(\Gamma)$ the set of all Borel probability measures $\eta$ on $\Gamma$ such that $e_{0}\sharp\eta=m_{0}$. For all $\eta\in\mathcal{P}_{m_{0}}(\Gamma)$, we set

[TABLE]

For all $x\in\overline{\Omega}$ and $\eta\in\mathcal{P}_{m_{0}}(\Gamma)$, we define

[TABLE]

where $\Gamma_{0}[x]=\{\gamma\in\Gamma:\gamma(0)=x\}$.

Definition 4.1**.**

Let $m_{0}\in\mathcal{P}(\overline{\Omega})$. We say that $\eta\in\mathcal{P}_{m_{0}}(\Gamma)$ is a contrained MFG equilibrium for $m_{0}$ if

[TABLE]

We denote by $\mathcal{P}_{m_{0}}^{\rm Lip}(\Gamma)$ the set of $\eta\in\mathcal{P}_{m_{0}}(\Gamma)$ such that $m^{\eta}(t)=e_{t}\sharp\eta$ is Lipschitz continuous. Let $\eta\in\mathcal{P}_{m_{0}}^{\rm Lip}(\Gamma)$ and fix $x\in\overline{\Omega}$. Then we have that

[TABLE]

where $L_{0}=L_{0}(\mu,M^{\prime},M,\kappa,||G||_{\infty},||DG||_{\infty})$ (see [9, Proposition 4.1]).

We recall the definition of mild solution of the constrained MFG problem given in [8].

Definition 4.2**.**

We say that $(u,m)\in C([0,T]\times\overline{\Omega})\times C([0,T];\mathcal{P}(\overline{\Omega}))$ is a mild solution of the constrained MFG problem in $\overline{\Omega}$ if there exists a constrained MFG equilibrium $\eta\in\mathcal{P}_{m_{0}}(\Gamma)$ such that

1. (i)

$m(t)=e_{t}\sharp\eta$* for all $t\in[0,T]$;* 2. (ii)

$u$* is given by*

[TABLE]

for $(t,x)\in[0,T]\times\overline{\Omega}$.

Remark 4.2*.*

Suppose that (L0),(L1), (D1) and (D2) hold true. Then,

there exists at least one constrained MFG equilibrium; 2. 2.

there exists at least one mild solution $(u,m)$ of the constrained MFG problem in $\overline{\Omega}$ such that

1. (i)

$u$ is Lipschitz continuous in $[0,T]\times\overline{\Omega}$; 2. (ii)

$m\in{\rm Lip}(0,T;\mathcal{P}(\overline{\Omega}))$ and ${\rm Lip}(m)\leq\ L_{0}$ where $L_{0}$ is given in (4.13).

For the proof see [9].

A direct consequence of Corollary 3.2 is the following result.

Corollary 4.1**.**

Let $\Omega$ be a bounded open subset of $\mathbb{R}^{n}$ with $C^{2}$ boundary. Suppose that (L0)-(L3), (D1)-(D3) hold true. Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$. Then, $u$ is locally semiconcave with modulus $\omega(r)=Cr^{\frac{1}{2}}$ in $[0,T)\times\overline{\Omega}$.

4.3 The Hamilton-Jacobi-Bellman equation

Let $\Omega$ be a bounded open subset of $\mathbb{R}^{n}$ with $C^{2}$ boundary. Assume that $H$, $F$ and $G$ satisfy the assumptions in Section 4.1. Let $m\in{\rm Lip}(0,T;\mathcal{P}(\overline{\Omega}))$. Consider the following equation

[TABLE]

We recall the definition of constrained viscosity solution.

Definition 4.3**.**

Let $u\in C((0,T)\times\overline{\Omega})$. We say that:

1. (i)

$u$* is a viscosity supersolution of (4.15) in $(0,T)\times\overline{\Omega}$ if*

[TABLE]

for any $\phi\in C^{1}(\mathbb{R}^{n+1})$ such that $u-\phi$ has a local minimum, relative to $(0,T)\times\overline{\Omega}$, at $(t,x)\in(0,T)\times\overline{\Omega}$; 2. (ii)

$u$* is a viscosity subsolution of (4.15) in $(0,T)\times\Omega$ if*

[TABLE]

for any $\phi\in C^{1}(\mathbb{R}^{n+1})$ such that $u-\phi$ has a local maximum, relative to $(0,T)\times\Omega$, at $(t,x)\in(0,T)\times\Omega$; 3. (iii)

$u$* is constrained viscosity solution of (4.15) in $(0,T)\times\overline{\Omega}$ if it is a subsolution in $(0,T)\times\Omega$ and a supersolution in $(0,T)\times\overline{\Omega}$.*

Remark 4.3*.*

Owing to Proposition 2.2, Definition 4.3 can be expressed in terms of subdifferential and superdifferential, i.e.,

[TABLE]

A direct consequence of the definition of mild solution is the following result.

Proposition 4.1**.**

Let $H$ and $F$ satisfy hypotheses $(H0)-(H3)$ and $(D1)-(D3)$, respectively. Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$. Then, $u$ is a constrained viscosity solution of (4.15) in $(0,T)\times\overline{\Omega}$.

Remark 4.4*.*

Given $m\in{\rm Lip}(0,T;\mathcal{P}(\overline{\Omega}))$, it is known that $u$ is the unique constrained viscosity solution of (4.15) in $\overline{\Omega}$ (see [12, 29, 30]).

From now on, we set

[TABLE]

We note that $Q_{m}\cap\partial Q_{m}=\emptyset$ and that $Q_{m}\cup\partial Q_{m}=supp(m)\cap((0,T)\times\overline{\Omega})$.

Theorem 4.1**.**

Let $H$ and $F$ satisfy hypotheses $(H0)-(H3)$ and $(D1)-(D3)$, respectively. Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$ and let $(t,x)\in Q_{m}$. Then,

[TABLE]

Proof.

Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$. Since $u$ is a constrained viscosity solution of (4.15) in $\overline{\Omega}$, we know that

[TABLE]

So, it suffices to prove that the converse inequality also holds. Let us take $(t,x)\in Q_{m}$ and $(p_{1},p_{2})\in D^{+}u(t,x)$. Since $(t,x)\in Q_{m}$, then there exists an optimal trajectory $\gamma:[0,T]\rightarrow\overline{\Omega}$ such that $\gamma(t)=x$. Let $r\in\mathbb{R}$ be small enough and such that $0\leq t-r\leq t$. Since $(p_{1},p_{2})\in D^{+}u(t,x)$ one has that

[TABLE]

Since

[TABLE]

we get

[TABLE]

By the dynamic programming principle and (4.18) one has that

[TABLE]

By our assumptions on $L$ and $F$ and by Theorem 2.1, one has that

[TABLE]

for all $s\in[t-r,t]$. Hence,

[TABLE]

and so by the definition of $H$ we conclude that

[TABLE]

This completes the proof. ∎

Proposition 4.2**.**

*Let $H$ and $F$ satisfy the hypotheses $(H0)-(H3)$ and $(D1)-(D3)$, respectively. Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$ and let $(t,x)\in Q_{m}$. Then $u$ is differentiable at $(t,x)$. *

Proof.

By Theorem 4.1 one has that

[TABLE]

Since $H(x,\cdot)$ is strictly convex and $D^{+}u(t,x)$ is a convex set, the above equality implies that $D^{+}u(t,x)$ is a singleton. Then, owing to Corollary 4.1 and [10, Proposition 3.3.4], $u$ is differentiable at $(t,x)$. ∎

Let $x\in\partial\Omega$. We denote by $H^{\tau}:\partial\Omega\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ the tangential Hamiltonian

[TABLE]

where $\nu(x)$ is the outward unit normal to $\partial\Omega$ in $x$.

Theorem 4.2**.**

Let $H$ and $F$ satisfy hypotheses $(H0)$-$(H3)$ and $(D1)$-$(D3)$, respectively. Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$ and let $(t,x)\in\partial Q_{m}$. Then,

[TABLE]

The technical lemma is needed for the proof of Theorem 4.2.

Lemma 4.1**.**

Let $(t,x)\in(0,T)\times\partial\Omega$ and let $\nu(x)$ be the outward unit normal to $\partial\Omega$ in $x$. Let $v\in\mathbb{R}^{n}$ be such that $\langle v,\nu(x)\rangle=0$. Then, there exists $\widehat{\gamma}\in\Gamma_{t}[x]$ such that $\dot{\widehat{\gamma}}(t)=v$.

Proof.

Let $(t,x)\in(0,T)\times\partial\Omega$ and let $\nu(x)$ be the outward unit normal vector to $\partial\Omega$ in $x$. Let $v\in\mathbb{R}^{n}$ be such that $\langle v,\nu(x)\rangle=0$. Let $R>0$ be small enough and let $\gamma$ be the trajectory defined by

[TABLE]

for all $s$ such that $|s-t|<R$. We denote by $\widehat{\gamma}$ the projection of $\gamma$ on $\overline{\Omega}$, i.e.,

[TABLE]

for all $s$ such that $|s-t|<R$. By construction, we have that $\widehat{\gamma}\in\Gamma_{t}[x]$. We only have to prove that $\dot{\widehat{\gamma}}(t)=v$. Hence, recalling that $d_{\Omega}(\gamma(t))=0$ one has that

[TABLE]

By [8, Lemma 3.1], and by the definition of $\gamma$ we have that

[TABLE]

Since $r\longmapsto\langle Db_{\Omega}(\gamma(r)),\dot{\gamma}(r)\rangle$ is continuous and vanishes at $r=0$, one has that

[TABLE]

Hence,

[TABLE]

and so $\dot{\widehat{\gamma}}(t)=v$. This completes the proof. ∎

Proof of Theorem 4.2.

Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$. Let us take $(t,x)\in\partial Q_{m}$ and $(p_{1},p_{2})\in D^{+}u(t,x)$. Let $\nu(x)$ be the outward unit normal to $\partial\Omega$ in $x$. Let $v\in\mathbb{R}^{n}$ be such that $\langle v,\nu(x)\rangle=0$. Let $r>0$ be small enough and such that $0<t<t+r<T$. By Lemma 4.1 there exists $\gamma\in\Gamma_{t}[x]$ such that $\dot{\gamma}(t)=v$. Since $(p_{1},p_{2})\in D^{+}u(t,x)$ one has that

[TABLE]

The dynamic programming principle ensures that

[TABLE]

Moreover,

[TABLE]

Using (4.23) and (4.24) in (4.22), we deduce that

[TABLE]

By our assumptions on $L$ and $F$ and by Theorem 2.1, one has that

[TABLE]

for all $s\in[t,t+r]$. Using (4.3), dividing by $r$, and passing to the limit for $r\rightarrow 0$ we obtain

[TABLE]

By the arbitrariness of $v$ and the definition of $H^{\tau}$, (4.26) implies that

[TABLE]

Now, we prove that the converse inequality also holds. Let $\gamma:[0,T]\rightarrow\overline{\Omega}$ be an optimal trajectory such that $\gamma(t)=x$. Since $\gamma(t)\in\partial{\Omega}$, and $\gamma(s)\in\overline{\Omega}$ for all $s\in[0,T]$ one has that $\langle\dot{\gamma}(t),\nu(x)\rangle=0$. Let $r>0$ be small enough and such that $0<t-r\leq t$. Since $(p_{1},p_{2})\in D^{+}u(t,x)$, and by the dynamic programming principle one has that

[TABLE]

Hence, we obtain

[TABLE]

Arguing as above we deduce that

[TABLE]

Since $\langle\dot{\gamma}(t),\nu(x)\rangle=0$, by the definition of $H^{\tau}$ we conclude that

[TABLE]

This completes the proof. ∎

Remark 4.5*.*

Let $(t,x)\in\partial Q_{m}$. By the definition of $H^{\tau}$ for all $p\in D^{+}_{x}u(t,x)$ one has that

[TABLE]

where $p^{\tau}$ is the tangential component of $p$.

In the next result, we give a full description of $D^{+}u(t,x)$ at $(t,x)\in\partial Q_{m}$.

Proposition 4.3**.**

Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$ and let $(t,x)\in\partial Q_{m}$. The following holds true.

1. (a)

The partial derivative of $u$ with respect to $t$, denoted by $\partial_{t}u(t,x)$, does exist and

[TABLE] 2. (b)

All $p\in D_{x}^{+}u(t,x)$ have the same tangential component, which will be denoted by $D^{\tau}_{x}u(t,x)$, that is,

[TABLE] 3. (c)

For all $\theta\in\mathbb{R}^{n}$ such that $|\theta|=1$ and $\langle\theta,\nu(x)\rangle=0$ one has that

[TABLE]

Moreover,

[TABLE]

where

[TABLE] 4. (d)

$D^{+}_{x}u(t,x)=\{p\in\mathbb{R}^{n}:p=D^{\tau}_{x}u(t,x)+\lambda\nu(x),\ \lambda\in(-\infty,\lambda_{+}(t,x)]\}$.

Proof.

Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$. Let $(t,x)\in\partial Q_{m}$ and let $\nu(x)$ be the outward unit normal to $\partial\Omega$ in $x$. Recall that, by Theorem 4.2 and Remark 4.5,

[TABLE]

Let us prove $(a)$ and $(b)$ together, arguing by contradiction. Let $p=(p_{1},p_{2})$, $q=(q_{1},q_{2})\in D^{+}u(t,x)$ be such that $p^{\tau}_{2}\neq q^{\tau}_{2}$. Let $\lambda\in[0,1]$. Since $D^{+}u(t,x)$ is a convex set, we have that

[TABLE]

Moreover, observe that

[TABLE]

Since $p_{\lambda}\in D^{+}u(t,x)$, (4.30) holds true and

[TABLE]

Since $H^{\tau}$ is strictly convex on the orthogonal complement, $(\nu(x))^{\perp}$, of $\nu(x)$, recalling that $p$ and $q$ satisfy (4.30) we have that

[TABLE]

So, we conclude that $p_{1}=q_{1}$ and $p^{\tau}_{2}=q^{\tau}_{2}$. Thus, $(a)$ and $(b)$ hold true. In order to prove $(c)$, let $\theta\in\mathbb{R}^{n}$ be such that $|\theta|=1$ and $\langle\theta,\nu(x)\rangle=0$. By the local semiconcavity of $u$ in $(0,T)\times\overline{\Omega}$, Lemma 2.1, and $(b)$ we deduce that

[TABLE]

which proves (4.28). Appealing to Proposition 2.5, the local semiconcavity of $u$ implies that

[TABLE]

where

[TABLE]

Finally, Proposition 2.1 and $(c)$ yield $(d)$. This completes the proof. ∎

Theorem 4.3**.**

Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$. Then the following holds true.

1. (i)

For any $(t,x)\in(0,T)\times\Omega$ one has that

[TABLE]

In particular, for all $(t,x)\in Q_{m}$,

[TABLE] 2. (ii)

Let $(t,x)\in\partial Q_{m}$. Then,

[TABLE]

where $D^{\tau}_{x}u(t,x)$ and $\lambda_{+}(t,x)$ are given in (4.27) and (4.29), respectively. Moreover, one has that

[TABLE] 3. (iii)

Let $(t,x)\in\partial Q_{m}$. Then,

[TABLE]

where $D^{\tau}_{x}u(t,x)$ and $\lambda_{+}(t,x)$ are given in (4.27) and (4.29), respectively.

Proof.

Let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$. By Corollary 4.1, Proposition 4.2, and [10, Proposition 3.3.4] we deduce that $(i)$ holds true. Hence, we only need to analyze $(ii)$ and $(iii)$.

Step 1.

Let $(t,x)\in\partial Q_{m}$. Let $u$ be differentiable at $(s_{k},y_{k})\in(0,T)\times\Omega$ with $(s_{k},y_{k})\to(t,x)$. Since $u$ is locally semiconcave, the bounded sequence $(\partial_{t}u(s_{k},y_{k}),Du(s_{k},y_{k}))$ has a subsequence (labelled in the same way) which converges to $(p_{1},p_{2})\in D^{+}u(t,x)$. Then Proposition 4.3 implies that $p_{1}=\partial_{t}u(t,x)$ and that there exists $\overline{\lambda}\in(-\infty,\lambda_{+}(t,x)]$ such that $p_{2}=D^{\tau}_{x}u(t,x)+\overline{\lambda}\nu(x)$. To prove (4.33), it only remains to show that $\overline{\lambda}=\lambda_{+}(t,x)$. This will be achieved in Step 3. Since $u$ is a viscosity solution of the Hamilton-Jacobi equation and is differentiable at $(s_{k},y_{k})$, we have that

[TABLE]

Passing to the limit in (4.36) we obtain

[TABLE]

Step 2.

The next step consists in proving that (4.34) holds by choosing a particular sequence of points. Let $(t_{k},x_{k})\in(0,T)\times\Omega$ be a sequence such that:

$(t_{k},x_{k})\xrightarrow{k\rightarrow\infty}(t,x)$; 2. 2.

$u$ is differentiable in $(t_{k},x_{k})$; 3. 3.

$\lim_{k\rightarrow+\infty}\frac{(t_{k}-t,x_{k}-x)}{|(t_{k}-t,x_{k}-x)|}=(0,-\nu(x)).$

Arguing as above, we know that any cluster point of $(Du(t_{k},x_{k}))$ is of the form $D^{\tau}_{x}u(t,x)+\tilde{\lambda}\nu(x)$, with $\tilde{\lambda}\leq\lambda_{+}(t,x)$, and satisfies

[TABLE]

On the other hand, by the local semiconcavity of $u$ (Theorem 3.1), we also have that

[TABLE]

Therefore,

[TABLE]

Dividing this inequality by $|(t_{k}-t,x_{k}-x)|$ and passing to the limit, we obtain

[TABLE]

By (4.29) we have that

[TABLE]

This proves that $\tilde{\lambda}=\lambda_{+}(t,x)$, whereas (4.34) follows from (4.38).

Step 3.

We finally show that the limit point $\overline{\lambda}$, defined in Step 1, equals $\lambda_{+}(t,x)$. Indeed, arguing by contradiction, let us assume that $\overline{\lambda}<\lambda_{+}(t,x)$. Then, by (4.37), (4.34), and the strict convexity of $H$, we have that, for any $\lambda\in(\overline{\lambda},\lambda_{+}(t,x))$,

[TABLE]

By Theorem 4.2, we deduce that

[TABLE]

which leads to a contradiction. Therefore, we have that $\overline{\lambda}=\lambda_{+}(t,x)$, which in turn implies (4.33).

Step 4.

The proof of point $(iii)$ runs exactly along the same lines as for point $(ii)$: if $(s_{k},y_{k})$ belongs to $\partial Q_{m}$ and converges to $(t,x)$, then the bounded sequence $(\partial_{t}u(s_{k},y_{k}),D^{\tau}_{x}u(s_{k},y_{k})+\lambda_{+}(s_{k},y_{k})\nu(y_{k}))$ converges (up to a subsequence) to some $(p_{1},p_{2})\in D^{+}u(t,x)$. As in Step 1, we have that $p_{1}=\partial_{t}u(t,x)$ while $p_{2}=D^{\tau}_{x}u(t,x)+\overline{\lambda}\nu(x))$ for some $\overline{\lambda}\leq\lambda_{+}(t,x)$ and

[TABLE]

Then, as in Step 3, we conclude that $\overline{\lambda}=\lambda_{+}(t,x)$.

∎

A direct consequence of the results of this section is the following theorem.

Theorem 4.4**.**

Let $H$, $F$ and $G$ satisfy hypotheses $(H0)-(H3)$ and $(D1)-(D3)$, respectively. Then, $u$ is a constrained viscosity solution of

[TABLE]

Moreover, $u$ is differentiable at any $(t,x)\in Q_{m}$ with

[TABLE]

while, on $\partial Q_{m}$, the time-derivative $\partial_{t}u$ exists and satisfies the equation

[TABLE]

Corollary 4.2**.**

Let $H$, $F$ and $G$ satisfy hypotheses $(H0)-(H3)$ and $(D1)-(D3)$, respectively. Let $\eta\in{\mathcal{P}}_{m_{0}}(\Gamma)$ be a constrained MFG equilibrium and $(u,m)$ be the associated mild solution of the constrained MFG problem in $\overline{\Omega}$. If $(t,x)\in Q_{m}\cup\partial Q_{m}$, then there exists $y\in\overline{\Omega}$ and an optimal trajectory $\gamma\in\Gamma^{\eta}[y]$ such that $\gamma(t)=x$. Moreover, if $p$ is the dual arc associated with $\gamma$, then

[TABLE]

Proof.

The existence of $\gamma$ is an easy consequence of the definition of $m$ and the uniform Lipschitz continuity of optimal trajectories. Let us now check that (4.39) holds. In view of Remark 2.6, we have

[TABLE]

where, by Corollary 3.1, $(H(x,p(t))-F(x,m(t)),p(t))$ belongs to $D^{+}u(t,x)$. Then Proposition 4.3 implies that $p(t)=Du(t,x)$ if $(t,x)\in Q_{m}$, while $p(t)=D^{\tau}_{x}u(t,x)+\overline{\lambda}\nu(x)$ for some $\overline{\lambda}\leq\lambda_{+}(t,x)$ if $(t,x)\in\partial Q_{m}$.

It remains to check that, in this second case, $\overline{\lambda}=\lambda_{+}(t,x)$. As $\gamma$ is of class $C^{1,1}([t,T],\overline{\Omega})$ and remains in $\overline{\Omega}$ with $\gamma(t)=x\in\partial\Omega$, we have that $\langle\dot{\gamma}(t),\nu(x)\rangle=0$. In particular

[TABLE]

This proves that the strictly convex map $\lambda\mapsto H(x,D^{\tau}_{x}u(t,x)(t,x)+\lambda\nu(x))$ has a (unique) minimum at $\lambda=\overline{\lambda}$. On the other hand, by Theorem 4.3 and Theorem 4.4 we have that

[TABLE]

So, if $\hat{v}\in\mathbb{R}^{n}$, with $\langle\hat{v},\nu(x)\rangle=0$, is a maximum point for the envelope formula in (4.20) which represents $H^{\tau}(x,D^{\tau}_{x}u(t,x)+\lambda_{+}(t,x)\nu(x))$, then $\hat{v}$ is also a maximizer of (4.6), which gives $H(x,D^{\tau}_{x}u(t,x)+\lambda_{+}(t,x)\nu(x))$. By the uniform convexity of $H$, this fact yields

[TABLE]

So,

[TABLE]

which proves that $\lambda_{+}(t,x)$ also minimizes the strictly convex map $\lambda\mapsto H(x,D^{\tau}_{x}u(t,x)+\lambda\nu(t,x))$. This shows that $\overline{\lambda}=\lambda_{+}(t,x)$ thus completing the proof. ∎

Remark 4.6*.*

From the above proof it follows that, for $(t,x)\in\partial Q_{m}$, $\lambda_{+}(t,x)$ can be characterized as the unique $\lambda\in\mathbb{R}$ such that the vector $-D_{p}H(x,D^{\tau}_{x}u(t,x)+\lambda\nu(x))$ is tangent to $\Omega$ at $x$, i.e., such that

[TABLE]

4.4 The continuity equation

The main result of this section is the following theorem.

Theorem 4.5**.**

Let $\Omega$ be a bounded open subset of $\mathbb{R}^{n}$ with $C^{2}$ boundary. Let $H$ and $F$ satisfy hypotheses $(H0)-(H3)$ and $(D1)-(D3)$, respectively. Let $m_{0}\in\mathcal{P}(\overline{\Omega})$ and let $(u,m)$ be a mild solution of the constrained MFG problem in $\overline{\Omega}$. Then, there exists a bounded continuous map $V:(0,T)\times\overline{\Omega}\rightarrow\mathbb{R}^{n}$ such that $m$ is a solution in the sense of distribution of the continuity equation

[TABLE]

that is, for all $\phi\in C^{1}_{c}((0,T)\times\overline{\Omega})$ one has that

[TABLE]

Moreover, $V$ is given on $supp(m)$ by

[TABLE]

where $Q_{m}$ and $\partial Q_{m}$ are defined in (4.16), whereas $D^{\tau}_{x}u(t,x)$ and $\lambda_{+}(t,x)$ are given in (4.27) and (4.29), respectively.

Proof.

Let us define $V$ on $supp(m)$ by (4.41). By Theorem 4.3 $V$ is continuous on the set $Q_{m}\cup\partial Q_{m}$. Since $Q_{m}\cup\partial Q_{m}$ is relatively closed in $(0,T)\times\overline{\Omega}$, using the Tietze extension theorem ([20, Theorem 5.1]) we can extend $V$ continuously to $(0,T)\times\overline{\Omega}$. It remains to check that (4.40) holds. Let $\eta$ be a constrained MFG equilibrium associated with $(u,m)$. Then, by the definition of $Q_{m}$ and $\partial Q_{m}$, recalling Corollary 4.2 we have that $(t,\gamma(t))\in Q_{m}\cup\partial Q_{m}$ and $\dot{\gamma}(t)=V(t,\gamma(t))$ for any $t\in(0,T)$ and $\eta-$a.e. $\gamma\in\Gamma$. So, for any $\phi\in C^{1}_{c}((0,T)\times\overline{\Omega})$, one has that

[TABLE]

The conclusion follows by integrating the above identity over $[0,T]$. ∎

5 Appendix: proof of Lemma 2.1

5.1 Proof of Proposition 2.2

The proof of Proposition 2.2 relies on the following technical lemma.

Lemma 5.1**.**

Let $w:(0,+\infty)\rightarrow[0,+\infty)$ be an upper semicontinuous function such that $\lim_{r\rightarrow 0}w(r)=0$. Then there exists a continuous nondecreasing function $w_{1}:[0,+\infty)\rightarrow[0,+\infty)$ such that

1. (i)

$w_{1}(r)\rightarrow 0$* as $r\rightarrow 0$,* 2. (ii)

$w(r)\leq w_{1}(r)$* for any $r\geq 0$,* 3. (iii)

the function $\xi(r):=rw_{1}(r)$ is in $C^{1}([0,+\infty))$ and satisfies $\dot{\xi}(0)=0$.

Proof.

Let us first set

[TABLE]

Then $\overline{w}$ is nondecreasing, not smaller than $w$, and tends to [math] as $r\rightarrow 0$. Next, we define for $r>0$

[TABLE]

and so we set $w_{1}(0)=0$. We first observe that, since $\overline{w}$ is nondecreasing, the same holds for $w_{0}$ and $w_{1}$. Then we have that $w(r)\leq w(r_{0})\leq\overline{w}(2r)$, and so $w_{0}(r)\rightarrow 0$ as $r\rightarrow 0$. Arguing in the same way with $w_{1}$ we deduce that properties $(i)$ and $(ii)$ hold. To prove $(iii)$, let us set $\xi(r)=rw_{1}(r)$. Then $\xi\in C^{1}((0+\infty))$ with derivate $\dot{\xi}(r)=2w_{0}(2r)-w_{0}(r)$. Thus $\dot{\xi}(r)\rightarrow 0$ as $r\rightarrow 0$ and so $\xi$ in $C^{1}$ in the closed half-line $[0,+\infty)$. ∎

Proof of Proposition 2.2.

The implications $(b)\Longrightarrow(c)$ and $(c)\Longrightarrow(a)$ are obvious; so it is enough to prove that $(a)$ implies $(b)$. Given $p\in D^{+}u(x)$, let us define, for $r>0$,

[TABLE]

where $[\cdot]_{+}$ denotes the positive part. The function $w$ is continuous and tends to [math] as $r\rightarrow 0$, by the definition of $D^{+}u$. Let $w_{1}$ be the function given by the previous lemma. Then, setting

[TABLE]

we have that $\phi\in C^{1}(\mathbb{R}^{n})$ and touches $u$ from above at $x$. ∎

The idea of the proof is based on [11, Theorem 4.5]. Let $x\in\partial\Omega$ and let $\nu(x)$ be the outward unit normal to $\partial\Omega$ in $x$. Let $\theta\in\mathbb{R}^{n}$ be such that $\langle\theta,\nu(x)\rangle\leq 0$. Let us set

[TABLE]

It suffices to prove that

[TABLE]

The first inequality in (5.8) is straightforward. Indeed, for any $p\in D^{+}u(x)$,

[TABLE]

So,

[TABLE]

In order to prove the last inequality in (5.8), pick sequences $h_{k}\rightarrow 0$ and $\theta_{k}\rightarrow\theta$ such that $x+h_{k}\theta_{k}\in\overline{\Omega}$ and

[TABLE]

Let us define

[TABLE]

We observe that the interior of $\mathcal{Q}(x,\theta_{k})$ is nonempty. Since $u$ is Lipschitz there exists a sequence $x_{k}$ such that

1. (i)

$x_{k}\in\mathcal{Q}(x,\theta_{k})$, $x_{k}\rightarrow x$ as $k\rightarrow\infty$; 2. (ii)

$u$ is differentiable at $x_{k}$ and there exists $\overline{p}\in D^{+}u(x)$ such that $Du(x_{k})\rightarrow\overline{p}$ as $k\rightarrow\infty$; 3. (iii)

$|s_{k}-h_{k}|\leq h_{k}^{2}$, where $s_{k}=\langle x_{k}-x,\theta_{k}\rangle$.

By the Lipschitz continuity of $u$, we note that $(iii)$ yields

[TABLE]

So, by (5.9) we have that

[TABLE]

Moreover,

[TABLE]

Since $u$ is locally Lipschitz and $x_{k}\in\mathcal{Q}(x,\theta_{k})$, one has that

[TABLE]

Since $u$ is semiconcave we deduce that

[TABLE]

for some constant $C>0$. Therefore

[TABLE]

By the definition of $\mathcal{Q}(x,\theta_{k})$ one has that $s_{k}|\theta_{k}|\geq|x_{k}-x|-|x_{k}-x|^{2}$, so that, as $x_{k}\rightarrow x$, $|x_{k}-x|\leq 2s_{k}$ for $k$ large enough. Recalling (ii), (5.10), and the fact that $\theta_{k}\rightarrow\theta$, we conclude that

[TABLE]

This completes the proof.