The Finite Horizon impulse control Problem with arbitrary cost functions : the Viscosity Solution Approach

TL;DR

This paper addresses stochastic impulse control problems with arbitrary cost functions, demonstrating that the value function uniquely solves the associated HJB PDE using viscosity solutions, thus extending the theoretical framework for such control problems.

Contribution

It establishes the uniqueness of the viscosity solution for the HJB equation in impulse control with arbitrary costs, broadening the applicability of the viscosity solutions approach.

Findings

Proves the value function is a unique viscosity solution.

Extends the viscosity solutions framework to arbitrary cost functions.

Provides a rigorous mathematical foundation for stochastic impulse control problems.

Abstract

We consider stochastic impulse control problems when the impulses cost functions are arbitrary. We use the dynamic programming principle and viscosity solutions approach to show that the value function is a unique viscosity solution for the associated Hamilton-Jacobi-Bellman equation (HJB) partial differential equation (PDE) of stochastic impulse control problems

Equations

$$\left\{\begin{array}[]{c}min\big{\{}-\partial_{t}V(t,x)-\mathcal{L}V(t,x)-f(t,x),V(t,x)-\sup\limits_{\xi\in U}[V(t,x+\xi)-c(t,x,\xi)]\big{\}}=0\\ V(T,x)=g(x).\end{array}\right.$$

$$\mathcal{L}V=⟨b,{\nabla }_{x}V⟩+\frac{1}{2}tr[\sigma {\sigma }^{\ast }{\nabla }_x^{2}V].$$

$$\left\{\begin{array}[]{c}\min\big{\{}-\cfrac{\partial V^{n}}{\partial t}-\mathcal{L}V^{n}-f,V^{n}-\sup\limits_{\xi\in U}[V^{n-1}(t,x+\xi)-c(t,x,\xi)]\big{\}}=0\qquad[0,T)\times\mathbb{R}^{n},\\ V^{n}(T,x)=g(x)\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\forall x\in\mathbb{R}^{n}.\end{array}\right.$$

$$|\sigma (t,x)-\sigma (t,x^{\prime })|+|b(t,x)-b(t,x^{\prime })|\le C|x-x^{\prime }|.$$

$$\underset{[0,T]\times {\mathbb{R}}^n\times U}{\inf }c\ge k,$$

$$c(t,x,{\xi }_1+{\xi }_2)\le c(t,x,{\xi }_1)+c(t,x,{\xi }_2),$$

$$\underset{\xi \in U}{\sup }[g(x+\xi )-c(T,x,\xi )]\le g(x).$$

$$\left\{\begin{array}[]{c}min\big{\{}-\cfrac{\partial V}{\partial t}(t,x)-\mathcal{L}V(t,x)-f(t,x),V(t,x)-\mathcal{M}V(t,x)\big{\}}=0\qquad[0,T)\times\mathbb{R}^{n},\\ \\ V(T,x)=g(x)\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\forall x\in\mathbb{R}^{n}.\end{array}\right.$$

$$\left\{\begin{array}[]{c}-\displaystyle\cfrac{\partial V}{\partial t}(t,x)-\mathcal{L}V(t,x)-f(t,x)\geq 0\qquad\qquad\qquad\qquad\qquad\qquad\;[0,T)\times\mathbb{R}^{n}\\ V(t,x)\geq\mathcal{M}V(t,x)\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\quad[0,T)\times\mathbb{R}^{n}\\ \big{(}-\displaystyle\cfrac{\partial V}{\partial t}(t,x)-\mathcal{L}V(t,x)-f(t,x)\big{)}\big{(}V(t,x)-\mathcal{M}V(t,x)\big{)}=0\qquad[0,T)\times\mathbb{R}^{n}\\ V(T,x)=g(x)\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\forall x\in\mathbb{R}^{n}.\end{array}\right.$$

$$\mathcal{L}V=⟨b,{\nabla }_{x}V⟩+\frac{1}{2}tr[\sigma {\sigma }^{\ast }{\nabla }_x^{2}V],$$

$$\mathcal{M}V(t,x)=\underset{\xi \in U}{\sup }[V(t,x+\xi )-c(t,x,\xi )],$$

$$\begin{array}[]{ll}\min\Big{[}-\cfrac{\partial\phi}{\partial t}(t_{0},x_{0})-\mathcal{L}\phi(t_{0},x_{0})-f(t_{0},x_{0}),\\ \qquad\qquad V(t_{0},x_{0})-\mathcal{M}V(t_{0},x_{0})\Big{]}\geq 0\qquad(\text{resp.,}\leq 0).\end{array}$$

$$\begin{array}[]{ll}V(T,x)\geq g(x)\qquad(\text{resp.,}\leq 0).\end{array}$$

$$V_{\ast }(t,x):=\underset{(s,y)\to (t,x),s<T}{\mathrm{liminf}}V(s,y),V^{\ast }(t,x):=\underset{(s,y)\to (t,x),s<T}{\mathrm{limsup}}V(s,y)$$

$$\begin{array}[]{lll}V(s,y)\leq V(t,x)+p(s-t)+\langle q,y-x\rangle+\displaystyle\frac{1}{2}\langle X(y-x),y-x\rangle+o(|s-t|+|y-x|^{2})\end{array}$$

$$\begin{array}[]{lll}\big{(}\text{resp.}\\ V(s,y)\geq V(t,x)+p(s-t)+\langle q,y-x\rangle+\displaystyle\frac{1}{2}\langle X(y-x),y-x\rangle+o(|s-t|+|y-x|^{2})\big{)}.\Box\end{array}$$

$$(D_t\varphi (t,x),D_x\varphi (t,x),D_{xx}^2\varphi (t,x))\in J^{2,-}V(t,x)(\text{resp}.J^{2,+}V(t,x)).\square$$

$$\begin{array}[]{ll}min\Big{[}-p-\langle b(t,x),q\rangle-\displaystyle\frac{1}{2}Tr\big{[}(\sigma\sigma^{*})(t,x)X\big{]}-f(t,x),V(t,x)-\mathcal{M}V(t,x)\Big{]}\geq 0\qquad(\text{resp.}\leq 0).\end{array}$$

$$\begin{array}[]{ll}V(T,x)\geq g(x)\qquad(\text{resp.}\leq 0).\qquad\Box\end{array}$$

$$\begin{array}[]{ll}X_{s}=x+\displaystyle\int_{t}^{s}b(r,X_{r})dr+\displaystyle\int_{t}^{s}\sigma(r,X_{r})dW_{r}+\sum\limits_{m\geq 1}\xi_{m}1\!\!1_{[\tau_{m},T]}(s)\qquad s\geq t.\end{array}$$

$$u_{[\tau ,\sigma ]}(s)=\sum _{m\ge 1}{\xi }_{{\mu }_{t,\tau }(u)+m}11_{[{\tau }_{{\mu }_{t,\tau }(u)+m}\le s\le \sigma ]}(s),\tau \le s\le \sigma ,$$

$$\text{(Problem)}\text{maximize}J(t,x,u)\text{over all}u\in \mathcal{U}$$

$$J(t,x,u):=\mathbb{E}\bigg{[}\displaystyle\int_{t}^{T}f(s,X_{s}^{t,x,u})ds-\sum\limits_{m\geq 1}c(\tau_{m},X_{\tau_{m}},\xi_{m})1\!\!1_{[\tau_{m}\leq T]}+g(X_{T}^{t,x,u})\bigg{]}.$$

$$(\text{Value Function})V(t,x)=\underset{u\in \mathcal{U}}{\sup }J(t,x,u).$$

$$\underset{u\in \mathcal{U}}{\sup }J(t,x,u)=\underset{u\in {\mathcal{U}}_{t,T}}{\sup }J(t,x,u),(t,x)\in [0,T]\times {\mathbb{R}}^n.$$

$$J(t,x,u)\leq C-\mathbb{E}\big{[}1\!\!1_{A}\big{\{}\sum_{m\geq 1}c(\tau_{m},X_{\tau_{m}},\xi_{m})\big{\}}+1\!\!1_{A^{c}}\big{\{}\sum_{m\geq 1}c(\tau_{m},X_{\tau_{m}},\xi_{m})\big{\}}\big{]}=-\infty,$$

$$V^n(t,x)=\underset{u\in {\mathcal{U}}_{t,T}^n}{\sup }J(t,x,u).$$

$$V^n(t,x)\nearrow V(t,x)\text{as}n\nearrow +\infty$$

$$\forall (t,x)\in [0,T]\times {\mathbb{R}}^n\underset{n\to \infty }{\lim }{V}^n(t,x)\le V(t,x).$$

$$J(t,x,u^{ϵ})\ge V(t,x)-ϵ.$$

Full text

1 Introduction

Impulse control problems form an important class of stochastic control problems. Finding a stochastic impulse control policy amounts to determining the sequence of random dates at which the policy is exercised and the sequence of impulses describing the magnitude of the applied policies, which maximizes a given reward function. It seems to be impossible to give an overview over all fields of application and all different variants that have been used. We only want to mention finance, e.g. cash management and portfolio optimization, see [23] and [27], control of an exchange rate by the Central Bank, see [25] and [9], and optimal forest management, see [32], [1] and the references therein.

In the literature, one finds several different approaches to tackle stochastic impulse control problems. One approach is to focus on solving the value function for the associated (quasi-)variational inequalities or Hamilton-Jacobi-Bellman (HJB for short) integro-differential equations, and then establishing the optimality of the solution by Verification Theorem (See Øksendal and Sulem [26]). Another approach is to characterize the value function of the control problem as a (unique) viscosity solution to the associated PDEs (See Lenhart [24], Tang and Yong [31]). In dimension one other approaches, based on excessive mappings and iterated optimal stopping schemes (see [2, 3, 18, 20]).

The main objective of this paper is to study the problem of existence and uniqueness of a solution in viscosity sense $V$ of the following system of partial differential equations with obstacles which depend on the solution:

[TABLE]

Where $\mathcal{L}$ is the second-order local operator

[TABLE]

In a way, the system (1) is Bellman system of equations associated with the control impulse with utility functions $f$, terminal payoff $g$ and impulse cost given by $c$.

Amongst the papers which consider the same problem as ours, and in the framework of viscosity solutions approach, the most elaborated works are certainly the ones by Tang and Yong [31], on the one hand, and by Seydel [29]. In [29], the author restricted controls to be only Markov controls. This Markovian assumption simplifies the proof of dynamic programming principle significantly. In the case of non-Markov controls, [31] show existence and uniqueness of a solution for (1). Nevertheless the paper suffers from two facts: (i) the costs of the impulses to be decreasing in time. (ii) the costs of the impulses do not depend on the state variable. For the first note of [31] we can easily adapt the methods in [4] or [15] in order to avoid the monotonicity condition. The second issue of [31], i.e. considering the case when $c$ depending also on $x$, was right now, according to our knowledge, an open problem. However, in that latter case, we face two main difficulties. First, it is not clear how to show the regularity of the value function, together with the gain functional, therefore we can not prove the dynamic programming principle. The second one is related to the obtention of the comparison of sub- and super-solutions of system (1) which plays an important role in our study.

Therefore the main objective of our work, and this is the novelty of the paper, is to show existence and uniqueness of a solution in viscosity sense for the system when the function $c$ is continuous depending also on $x$. To derive these results, we first study $(V^{n})_{n}$, where $V^{n}$ is the value function from $t$ to $T$, when the system only at most $n$ interventions after $t$ are allowed, we give the dynamic programming principle for $V^{n}$ and we show that it is a continuous viscosity solution to

[TABLE]

Further, we obtain that the $u.s.c.$ envelope $V^{*}$ of the value function, is a viscosity sub-solution of (1), and the $l.s.c.$ envelope $V_{*}$ of the value function, is a viscosity super-solution of (1). Finally, by comparison principle we obtain that the value function of the impulse control problem is a unique continuous viscosity solution to (1).

This paper is organized as follows: In Section 2, we formulate the problem and we give the related definitions. In Section 3, we shall introduce the impulse control problem and we study the problem in which the controller only can intervene finitely many times. Further we prove the dynamic programming principle. Section 4, is devoted to the connection between impulse control problem and Hamilton-Jacobi-Bellman equation. In Section 5, we show that the solution of Hamilton-Jacobi-Bellman equation is unique in the subclass of bounded functions.

2 Assumptions and formulation of the problem

Throughout this paper $T$ (resp. $n,\,d$) is a fixed real (resp. integers) positive numbers. $|.|$ will denote the canonical Euclidian norm on $\mathbb{R}^{n}$, and $\langle.|.\rangle$ the corresponding inner product. Let us assume the following assumptions:

[H1] $b:[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R}^{n}$ and $\sigma:[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R}^{n\times d}$ be two bounded continuous functions for which there exists a constant $C>0$ such that for any $t\in[0,T]$ and $x,x^{\prime}\in\mathbb{R}^{n}$

[TABLE]

[H2] $f:[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ is uniformly continuous and bounded on $[0,T]\times\mathbb{R}^{n}$. $g:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is uniformly continuous and bounded on $\mathbb{R}^{n}$.

[H3] The cost function $c:[0,T]\times\mathbb{R}^{n}\times U\rightarrow\mathbb{R}$ is measurable and uniformly continuous. Furthermore

[TABLE]

where $k>0$. Moreover,

[TABLE]

for every $(t,x)\in[0,T]\times\mathbb{R}^{n},\xi_{1},\xi_{2}\in U.$

[H4] For any $x\in\mathbb{R}^{n}$ and $\xi\in U$

[TABLE]

Where $U$ is a compact subset of $\mathbb{R}^{n}$.

Remark 1**.**

Assumption (H3) and (H4) simply provides the classical framework for the study of impulse problems. (5) ensures that multiple impulses occurring at the same time are suboptimal. (6) ensures the nonoptimality of a impulse at maturity. $\Box$

We now consider the HJB equation:

[TABLE]

Or equivalently,

[TABLE]

Where $\mathcal{L}$ is the second-order local operator

[TABLE]

and the nonlocal operator $\mathcal{M}$ is given by

[TABLE]

for every $(t,x)\in[0,T]\times\mathbb{R}^{n}.$

The main objective of this paper is to focus on the existence and uniqueness of the solution in viscosity sense of (7) whose definition is:

Definition 1**.**

A lower (resp., upper) semicontinuous function $V:[0,T]\times\mathbb{R}^{n}\to\mathbb{R}$ is called a viscosity supersolution (resp., subsolution) to the HJB equation (7) if

• •

for every $(t_{0},x_{0})\in[0,T)\times\mathbb{R}^{n}$ and any function $\phi\in C^{1,2}([0,T)\times\mathbb{R}^{n})$, such that $(t_{0},x_{0})$ is a local minimum (resp., maximum) of $V-\phi$, we have:

[TABLE]

• •

for every $x\in\mathbb{R}^{n}$ we have

[TABLE]

A bounded function $V:[0,T)\times\mathbb{R}^{n}\to\mathbb{R}$ is a viscosity solution to the HJB equation (7) if its lower semicontinuous envelope $V_{*}$ is a viscosity supersolution and its upper semicontinuous envelope $V^{*}$ is a viscosity subsolution. $V_{*}$ and $V^{*}$ are given by

[TABLE]

for every $(t,x)\in[0,T]\times\mathbb{R}^{n}$. $\Box$

There is an equivalent formulation of this definition (see e.g. [31]) which we give because it will be useful later. So firstly, we define the notions of superjet and subjet of a function $V$.

Definition 2**.**

Let $V:[0,T]\times\mathbb{R}^{n},$ be a lower (resp. upper) semicontinuous function, $(t,x)$ an element of $(0,T)\times\mathbb{R}^{n}$ and finally $S_{n}$ the set of $n\times n$ symmetric matrices. We denote by $J^{2,+}V(t,x)$ (resp. $J^{2,-}V(t,x))$, the superjets (resp. the subjets) of $\,V$ at $(t,x)$, the set of triples $(p,q,X)\in\mathbb{R}\times\mathbb{R}^{n}\times S_{n}$ such that:

[TABLE]

[TABLE]

Note that if $\phi-V$ has a local maximum (resp. minimum) at $(t,x)$, then we obviously have:

[TABLE]

We now give an equivalent definition of a viscosity solution of HJB equation(7):

Definition 3**.**

Let $V$ be a lower (resp. upper) continuous function defined on $[0,T]\times\mathbb{R}^{n}$. Then V is a viscosity supersolution (resp., subsolution) to the HJB equation (7) if and only if

• •

for every $(t,x)\in[0,T)\times\mathbb{R}^{n}$ and $(p,q,X)\in J^{2,-}V(t,x)$  (resp. $J^{2,+}V(t,x))$,

[TABLE]

• •

for every $x\in\mathbb{R}^{n}$ we have

[TABLE]

As pointed out previously we will show that system (7) has a unique solution in viscosity sense. This system is the deterministic version of the stochastic impulse control problem will describe briefly in the next section.

3 The impulse control problem

3.1 Setting of the problem

3.1.1 Probabilistic setup

We work on a time horizon $[0,T]$, where $0<T<\infty$. Let ($\Omega,\mathcal{F},\mathbb{P})$ is a fixed probability space on which is defined a standard $d$-dimensional Brownian motion $W=(W_{t})_{t\leq T}$, whose natural filtration is $(\mathcal{F}^{0}_{t}:=\sigma\{W_{s};s\leq t\})_{0\leq t\leq T}.$ We denote by $\mathbb{F}=(\mathcal{F}_{t})_{t\leq T}$ the completed filtration of $(\mathcal{F}^{0}_{t})_{t\leq T}$ with the $\mathbb{P}$-null sets of $\mathcal{F}$. The expectation operator with respect to $\mathbb{P}$ is denoted by $\mathbb{E}$, and the indicator function of a set or event $A$ is written as $1\!\!1_{A}$. The notation $a.s.$ stands for almost-surely.

3.1.2 Impulse control definitions

The following data for the impulse control problem are given:

A spaces of control actions $U\subset\mathbb{R}^{n}$, where $U$ is a compact. 2. 2.

A reward received at time $T$, which is modelled by an $\mathcal{F}_{T}$-measurable real-valued random

variable $g$. 3. 3.

A running reward, which is represented by a real-valued adapted process $f=(f(t,x))_{(t,x)\in[0,T]\times\mathbb{R}^{n}}$. 4. 4.

A cost of the intervention $\xi\in U$, which is modelled by a real-valued adapted process

$c=(c(t,x,\xi))_{(t,x)\in[0,T]\times\mathbb{R}^{n}}$.

Define the concept of impulse control as follows:

Definition 4**.**

An impulse control $u=\sum\limits_{m\geq 1}\xi_{m}1\!\!1_{[\tau_{m},T]}$ on $[t,T]\subset\mathbb{R}^{+}=[0,+\infty)$, is such that:

• •

$(\tau_{m})_{m}$, the action times, is a sequence of $\mathbb{F}$-stopping times, valued in $[t,T]\cup\{+\infty\}$ such that $\mathbb{P}$-a.s. $\tau_{m}\leq\tau_{m+1}$.

• •

$(\xi_{m})_{m}$, the actions, is a sequence of $U$-valued random variables, where each $\xi_{m}$ is $\mathcal{F}_{\tau_{m}}$-measurable.

We denote by $\mathcal{U}$ the set of processes $u(\cdot)$.  $\Box$

Let $t\in[0,T]$ be the initial time and $x\in\mathbb{R}^{n}$ the initial state. Then, given the impulse control $u$ on $[t,T]$, a stochastic process $(X_{s})_{s\geq 0}$ follows a stochastic differential equation,

[TABLE]

Note that the assumption (H1) ensure, for any $(t,x)\in[0,T]\times\mathbb{R}^{n}$, the existence and uniqueness of a solution $X^{t,x,u}=\{X^{t,x,u}_{s},t\leq s\leq T\}$ to the SDE (13) (see [28] for more details).

Let $u=\sum_{m\geq 1}\xi_{m}1\!\!1_{[\tau_{m},T]}$ be an impulse control on $[t,T]$, and let $\tau\leq\sigma$ be two $[t,T]$-valued $\mathbb{F}$-stopping times. Then we define the restriction $u_{[\tau,\sigma]}$ of the impulse control $u$ by:

[TABLE]

$\mu_{t,\tau}$ is the number of impulses up to time $\tau$, i.e., $\mu_{t,\tau}(u):=\sum\limits_{m\geq 1}1\!\!1_{[\tau_{m}\leq\tau]}.$

The stochastic control problem is to

[TABLE]

subject to (13) with

[TABLE]

Here we denote V for the associated value function:

[TABLE]

We first observe that the optimal impulse problem over $\mathcal{U}$ can be restricted to the consideration of finite number of impulses $\mathcal{U}_{t,T}$, where $\mathcal{U}_{t,T}:=\{u=\sum_{m\geq 1}\xi_{m}1\!\!1_{[\tau_{m},T]}\in\mathcal{U}\;|\quad\mathbb{P}(\tau_{k}<T,\;\forall k\geq 1)=0\}.$

Proposition 5**.**

Under (H1), (H2) and (H3) the supremum of $J$ over $\mathcal{U}$ coincides with the one of $J$ over $\mathcal{U}_{t,T}$, that is

[TABLE]

$Proof.$ If $u$ does not belong to $\mathcal{U}_{t,T}$, then $J(t,x,u)=-\infty$. Indeed, introduce $A:=\{\omega\in\Omega\,|\,\tau_{n}(\omega)<T,\;\forall n\geq 1\}$ and $A^{c}$ be its complement. Since $u\notin\mathcal{U}_{t,T}$, then $\mathbb{P}(A)>0$. Using the boundedness of $f$ and $g$, we deduce there exists a constant $C>0$ such that

[TABLE]

since for any $(t,x)\in[0,T]\times\mathbb{R}^{n}$ and $\xi\in U,c(t,x,\xi)\geq k$, and directly deduce that $\sup_{u\in\mathcal{U}}J(t,x,u)=\sup_{u\in\mathcal{U}_{t,T}}J(t,x,u).$   $\Box$

3.2 When the controller can intervene finitely many times

For $n=0,1,...$ let $U_{t,T}^{n}=\{u\in\mathcal{U}_{t,T}\quad\text{such that}\quad\tau_{n+1}=+\infty\}$. In other words, $U_{t,T}^{n}$ is the set of all controls with at most $n$ interventions. Next let us define

[TABLE]

Lemma 6**.**

The sequence $(V^{n})_{n\in\mathbb{N}}$ is increasing and converges on $[0,T]\times\mathbb{R}^{n}$ to the function $V$:

[TABLE]

for all $(t,x)\in[0,T]\times\mathbb{R}^{n}$.

$Proof$. Since $U_{t,T}^{n}\subset U_{t,T}^{n+1}\subset U_{t,T}$, it follows that $(V^{n}(t,x))_{n\in\mathbb{N}}$ is a non-decreasing sequence and

[TABLE]

Let us fix $t$ and $x$. For a given $\epsilon>0$, there exists $u^{\epsilon}=\sum_{m\geq 1}\xi^{\epsilon}_{m}1\!\!1_{[\tau^{\epsilon}_{m},T]}\in\mathcal{U}_{t,T}$ such that

[TABLE]

Define a control $u^{n}\in\mathcal{U}_{t,T}^{n}$ from $u^{\epsilon}$ by setting $u^{n}=\sum_{m=1}^{n}\xi^{\epsilon}_{m}1\!\!1_{[\tau^{\epsilon}_{m},T]}.$ Then $X_{s}^{t,x,u^{n}}=X_{s}^{t,x,u^{\epsilon}}$ for all $s\leq\tau_{n}$ and by positivity of the impulses costs,

[TABLE]

by Cauchy-Schwarz inequality and the boundedness of $f$ and $g$. Hence letting $n\to\infty$, and since $\mu_{t,T}(u^{\epsilon})<\infty$ $a.s.$, we obtain

[TABLE]

Therefore, by (23) we get

[TABLE]

Then by arbitrariness of $\epsilon$ we get the required assertion.  $\Box$

The following lemma is a direct consequence of the positivity of $c$ and the boundedness of $f$ and $g$. As such, its proof is omitted.

Lemma 7**.**

Under the standing assumptions (H1), (H2) and (H3), the value function is bounded.

Proposition 8**.**

Under (H1), (H2) and (H3) we have

• (i)

$V^{n}$* is continuous in $[0,T]\times\mathbb{R}^{n}$*

• (ii)

$V$* is lower semicontinuous in $[0,T]\times\mathbb{R}^{n}$*

• (iii)

$J(\cdot,u)$* is continuous in $[0,T]\times\mathbb{R}^{n}$ for all $u\in\mathcal{U}_{0,T}^{n}$.*

$Proof.$ We only prove (i). The two other ones follow from similar arguments. Let $\epsilon>0$ , $t^{\prime}\in]t,T],$ and $x^{\prime}\in B(x,\epsilon),$ then there exist $u_{\epsilon}\in\mathcal{U}_{t,T}^{n}$ such that

[TABLE]

where $\bar{u}_{\epsilon}\in\mathcal{U}^{n}_{t^{\prime},T}$ will be chosen later. In particular, suppose that $u_{\epsilon}:=\sum_{m=1}^{n}\xi^{\epsilon}_{m}1\!\!1_{[\tau^{\epsilon}_{m},T]}\in\mathcal{U}^{n}_{t,T}$, then define $\bar{u}_{\epsilon}$ from $u_{\epsilon}$ by setting $\bar{u}_{\epsilon}:=\sum_{m=1}^{n}\xi^{\epsilon}_{m}1\!\!1_{[t^{\prime}\vee\tau^{\epsilon}_{m},T]}$. Therefore we have

[TABLE]

Moreover, by a standard estimate for the SDE applying Itô’s formula to $|X_{s}^{t,x,u_{\epsilon}}-X_{s}^{t^{\prime},x^{\prime},\bar{u}_{\epsilon}}|^{2}$ and using Gronwall’s lemma, we then obtain from the Lipschitz condition on $b$ and $\sigma$

[TABLE]

taking the limit as $(t,x)\to(t^{\prime},x^{\prime})$, and using the uniform continuity of $f$, $g$ and $c$ to obtain:

[TABLE]

As $\epsilon$ is arbitrary then sending $\epsilon\to 0$ to obtain:

[TABLE]

Therefore $V^{n}$ is upper semi-continuous. In a similar way we can prove that

[TABLE]

Therefore $V^{n}$ is continuous.  $\Box$

3.3 Dynamic programming principle

The rigorous connection between $V$ and HJB equation passes through the dynamic programming principle (DPP). We begin with the following two lemmas.

Lemma 9**.**

Let $u=\sum_{m=1}^{n}\xi_{m}1\!\!1_{[\tau_{m},T]}$ be a nearly optimal control of $V^{n}$, then there exist a positive constant $C$ which does not depend on $t$ and $x$ such that:

[TABLE]

*We denote by $\widehat{\mathcal{U}}^{n}_{t,T}$ the set of impulse controls which satisfies the condition (26).

$Proof.$ Let us choose a nearly optimal control $u=\sum_{m=1}^{n}\xi_{m}1\!\!1_{[\tau_{m},T]}\in\mathcal{U}_{t,T}^{n}$ such that,

[TABLE]

Since $V^{n},f$ and $g$ are bounded, then we obtain the desired result.  $\Box$

Lemma 10**.**

Under (H1), (H2) and (H3) we have

[TABLE]

for evry $(t,x)\in[0,T)\times\mathbb{R}^{n}$, where $\bar{\mathcal{U}}_{t,T}^{n}$ contain all the impulse controls in $\widehat{\mathcal{U}}^{n}_{t,T}$ which have no impulses at time t.

$Proof.$ Let $u\in\widehat{\mathcal{U}}^{n}_{t,T}\setminus\bar{\mathcal{U}}_{t,T}^{n},$ then for every $\epsilon>0$, we have to prove that there exist $\bar{u}\in\bar{\mathcal{U}}_{t,T}^{n}$ such that

[TABLE]

We only consider the case in which $u$ has a single impulse at time $t$. (In the case of multiple impulses at time $t$, we using condition (5), that we can reduce this case with only a single impulse at time $t$). Then there exist a $[t,T]-$valued $\mathbb{F}-$stopping times $\tau$, with $\mathbb{P}(\tau=t)>0$ such that

[TABLE]

where $\hat{u}=\sum_{m=2}^{n}\xi_{m}1\!\!1_{[\tau_{m},T]}\in\bar{\mathcal{U}}_{t,T}^{n}$ and $\xi$ is an $\mathcal{F}_{\tau}-$ measurable $U-$valued random variable. For every integer $k$ (large enough) we pose \tau_{k}=\big{(}\tau+\frac{1}{k}\big{)}1\!\!1_{[\tau=t]}+\tau 1\!\!1_{[\tau>t]}. Next, define the impulse control

[TABLE]

Then $\tau_{k}\to\tau$ as $k\to\infty$, $\mathbb{P}-a.s$. Moreover, for all $s\in(t,T],\;X^{t,x,u_{k}}_{s}\to X^{t,x,u}_{s}$ as $k\to\infty$, $\mathbb{P}-a.s.$

On the other hand, we have

[TABLE]

Therefore, from the dominated convergence theorem, we deduce the existence of an integer $K\geq 1$ such that

[TABLE]

Thus we get the thesis.  $\Box$

Theorem 11**.**

Given $0\leq t\leq r\leq T$, $x\in\mathbb{R}^{n}$, and a $[t,r]-$valued $\mathbb{F}$ stopping times $\tau$, we have

[TABLE]

$Proof.$ Let $\epsilon>0$, then there exists $u^{n,\epsilon}\in\bar{\mathcal{U}}_{s,T}^{n}$ and $u^{n-1,\epsilon}\in\bar{\mathcal{U}}_{s,T}^{n-1}$ such that

[TABLE]

Now, for fixed $(s,y)\in[0,T]\times\mathbb{R}^{n}$, the continuity of $V^{n}$ and $J$ established in Proposition 8 imply that there exists $r^{(\epsilon,s,y)}$ such that

[TABLE]

for all $(t^{\prime},y^{\prime})\in B(s,y;r^{(\epsilon,s,y)})$, where for $r>0$ and $(s,y)\in[0,T]\times\mathbb{R}^{n}$

[TABLE]

Therefore, the family $\{B(s,y;r):(s,y)\in[0,T]\times\mathbb{R}^{n},0<r\leq r^{(\epsilon,s,y)}\}$ forms an open covering of $(0,T]\times\mathbb{R}^{n}$. By the $Lindel\ddot{o}f$ covering Theorem ([12], Theorem 6.3 Chapter VIII), there exists a countable sequence $(t_{i},y_{i},r_{i})_{i\geq 1}$ in $[0,T]\times\mathbb{R}^{n}\times\mathbb{R}$ such that $\{B(t_{i},y_{i};r_{i})\}_{i\geq 1}$ is a countable subcover of $(0,T]\times\mathbb{R}^{n}$. We set $u_{n,i}^{\epsilon}:=u^{n,(t_{i},y_{i}),\epsilon}$ and $u^{\epsilon}_{n-1,i}:=u^{n-1,(t_{i},y_{i}),\epsilon}$.

Now set $A_{0}:=\{T\}\times\mathbb{R}^{n}$, $C_{-1}:=\emptyset$ and define for all $i\in\mathbb{N}\cup\{0\}$

[TABLE]

Under this construction, we have

[TABLE]

Next the inequalities $(\ref{progeq1})$ and $(\ref{progeq2})$ yield that

[TABLE]

For any $k\in\mathbb{N}$, set $A^{k}:=\bigcup_{0\leq i\leq k}A_{i}$. Given $u\in\widehat{\mathcal{U}}_{t,T}^{n}$, we define

[TABLE]

Then

[TABLE]

We deduce via $(\ref{progeq3})$ that

[TABLE]

for every $k\geq 1$. Letting $k\rightarrow\infty$, therefore,

[TABLE]

by dominated convergence and since $J(t,x,u)$ is bounded. Moreover, monotone convergence yields

[TABLE]

Therefore, we deduce the existence of an integer $k_{0}\geq 1$ such that

[TABLE]

The arbitrariness of $u$ and $\epsilon$ implies that

[TABLE]

On the other hand for any $u\in\widehat{\mathcal{U}}^{n}_{t,T}$, we have

[TABLE]

Taking supremum on both sides, we get

[TABLE]

Then from (36) and (37) we deduce the thesis.  $\Box$

4 Hamilton-Jacobi-Bellman equation

In the present section we study HJB equation by means of viscosity solutions.

Theorem 12**.**

$V^{n}$* is a continuous viscosity solution to*

[TABLE]

$Proof.$ We know from Proposition 8 that $V^{n}$ is continuous on $[0,T]\times\mathbb{R}^{n}$, therefore $V^{n}$ is equal to its lower semicontinuous envelope and to its upper semicontinuous envelope on $[0,T]\times\mathbb{R}^{n}$. We begin by proving that $V^{n}$ is a viscosity subsolution to (38). Suppose $V^{n}-\phi$ achieves a local maximum in $[t_{0},t_{0}+\delta)\times B(x_{0},\delta)$ with $V^{n}(t_{0},x_{0})=\phi(t_{0},x_{0})$. When $\tau=t$ in the dynamic programming principle for $V^{n}$, we have, $V^{n}(t_{0},x_{0})\geq\mathcal{M}V^{n-1}(t_{0},x_{0})$. Then if $V^{n}(t_{0},x_{0})=\mathcal{M}V^{n-1}(t_{0},x_{0})$, we already have the desired inequality. Now suppose

[TABLE]

we prove by contradiction that

[TABLE]

Suppose otherwise, i.e.,$\,-\cfrac{\partial\phi}{\partial t}(t_{0},x_{0})-\mathcal{L}\phi(t_{0},x_{0})-f(t_{0},x_{0})>0$. Then without loss of generality we can assume that $-\cfrac{\partial\phi}{\partial t}(t,x)-\mathcal{L}\phi(t,x)-f(t,x)>0$ and $V^{n}(t,x)-\mathcal{M}V^{n-1}(t,x)>\epsilon$ on $[t_{0},t_{0}+\delta)\times B(x_{0},\delta)$.

Define the stopping time $\tau$ by

[TABLE]

By Itô’s formula

[TABLE]

where $u_{0}$ is the control with no impulses.

Let $\epsilon_{1}>0$, using the dynamic programming principle between time $t_{0}$ and $\tau\wedge\tau_{1}$, we deduce the existence of a control $u^{\epsilon_{1}}\in\widehat{\mathcal{U}}^{n}_{t_{0},T}$ such that

[TABLE]

Therefore, without loss of generality, we only need to consider $u^{\epsilon_{1}}\in\widehat{\mathcal{U}}^{n}_{t_{0},T}$ such that $\tau^{\epsilon}_{1}>\tau$. Then

[TABLE]

Then from (40) and sending $\epsilon_{1}\rightarrow 0$ we get

[TABLE]

which is a contradiction. Therefore,

[TABLE]

which is the subsolution property. The supersolution property is proved analogously.  $\Box$

Now we prove that the value function satisfy, in the viscosity sense, the terminal condition.

Lemma 13**.**

*The value function is viscosity solutions to (10).

$Proof.$ First, we prove the subsolution property.

Let us prove that $V^{*}(T,x)=g(x)$. For this and to begin with, we are going to show that:

[TABLE]

By definition we know that,

[TABLE]

therefore

[TABLE]

since $V^{n}$ is continuous and at $t=T$ it is equal to $g(x)$. On the other hand we have,

[TABLE]

then

[TABLE]

wich with (42) imply that

[TABLE]

Let us now show that the left-hand side of (44) cannot be positive. Let us suppose that for some $x_{0}$, there is $\epsilon>0$ such that:

[TABLE]

Let $(t_{k},x_{k})_{k\geq 1}$ be a sequence in $[0,T]\times\mathbb{R}^{k}$ satisfying

[TABLE]

Since $V^{*}$ is bounded and $usc$ and taking into account $V^{n}\nearrow V$, we can find a sequence $(\rho^{n})_{n\geq 0}$ of functions of $\mathcal{C}^{1,2}([0,T]\times\mathbb{R}^{n})$ such that $\rho^{n}\to V^{*}$ and, on some neighborhood $B_{n}$ of $(T,x_{0})$ we have:

[TABLE]

After possibly passing to a subsequence of $(t_{k},x_{k})_{k\geq 1}$ we can assume that (46) holds on $B_{n}^{k}:=[t_{k},T]\times B(x_{k},\delta_{k}^{n})$ for some sufficiently small $\delta^{n}_{k}\in(0,1)$ such that $B_{n}^{k}\subset B_{n}$. Now since $V^{*}$ is bounded then there exists $\eta>0$ such that $|V^{*}|\leq\eta$ on $B_{n}$. We can then assume that $\rho^{n}\geq-2\eta$ on $B_{n}$. Next let us define $\tilde{\rho}^{n}_{k}$ by:

[TABLE]

Note that $\tilde{\rho}^{n}_{k}\geq\rho^{n}$ and

[TABLE]

As $\partial_{t}(\sqrt{T-t})\to-\infty$ as $t\to T$, we can choose $t_{k}$ large enough in front of $\delta^{n}_{k}$ and the derivatives of $\rho^{n}$ to ensure that

[TABLE]

Next let us consider the following stopping time $\theta_{n}^{k}:=\inf\{s\geq t_{k},(s,X_{s}^{t_{k},x_{k}})\in{B_{n}^{k}}^{c}\}\wedge T$ where ${B_{n}^{k}}^{c}$ is the complement of $B_{n}^{k}$, and $m^{k}:=\inf\{s\geq t_{k},V(s,X_{s}^{t_{k},x_{k}})=\sup\limits_{\xi\in U}[V(s,X_{s}^{t_{k},x_{k}}+\xi)-c(s,X_{s}^{t_{k},x_{k}},\xi)]\}\wedge T.$ Applying now Itô’s formula to the process $(\tilde{\rho}^{n}_{k}(s,X_{s}))_{s}$ stopped at time $\theta_{n}^{k}\wedge m^{k}$ and taking into account (46), (47) and (48) to obtain:

[TABLE]

By assumption (H2), we deduce that

[TABLE]

Therefore taking the limit in the previous inequalities yields:

[TABLE]

But this is a contradiction since $\rho^{n}\to V^{*}$ pointwisely as $n\to\infty$. Thus for any $x\in\mathbb{R}^{k}$ we have:

[TABLE]

Now, suppose that $V^{*}(T,x)>g(x)$, then from the previous equality there exists $\xi_{1}\in U$ such that:

[TABLE]

But once more we have $V^{*}(T,x+\xi_{1})>g(x+\xi_{1})$. Otherwise, i.e. if $V^{*}(T,x+\xi_{1})=g(x+\xi_{1})$ we would have from (50):

[TABLE]

which is contradictory with (H4). Therefore

[TABLE]

Then there exists $\xi_{2}\in U$ such that:

[TABLE]

Proceeding in similar fashion, after finitely many steps, boundedness of $V^{*}$ will be contradicted. Thus we have:

[TABLE]

Theorem 14**.**

*The value function $V,$ defined by (17), is a viscosity solution of the HJB (7) (with terminal condition $V(T,x)=g(x)$).

$Proof.$ Thanks to Lemma 13 we have that $V$ satisfies, in the viscosity sense, the terminal condition. As a consequence, we have only to address (9). First, we prove that $V^{*}$ is a subsolution to (9). Note that since $V^{n}\nearrow V$ and $V^{n}$ is continuous then we have

[TABLE]

Since $V^{n}(t,x)\geq\mathcal{M}V^{n-1}(t,x)$, which implies in taking the limit:

[TABLE]

Let us now $(t,x)\in[0,T[\times\mathbb{R}^{n}$ be such that:

[TABLE]

Let $(p,q,X)\in\bar{J}^{+}V^{*}(t,x)$. By (53) and Lemma 6.1 in [11], there exist sequences

[TABLE]

such that

[TABLE]

From the viscosity subsolution property for $V^{n_{j}}$ at $(t_{j},x_{j})$, for any $j\geq 0$, we have

[TABLE]

Next by (55), there exists $j_{0}\geq 0$, such that if $j\geq j_{0}$ we have

[TABLE]

Therefore for any $j\geq j_{0}$

[TABLE]

which implies that

[TABLE]

which is the subsolution property. The supersolution property is proved analogously.$\Box$

Now we give an equivalent of Hamilton-Jacobi-Bellman equation (7). We consider the new function $\Gamma$ given by the classical change of variable $\Gamma(t,x)=\exp(t)V(t,x)$, for any $(t,x)\in[0,T]\times\mathbb{R}^{n}$.

A second property is given by the:

Proposition 15**.**

$V$* is a viscosity solution of (7) if and only if $\Gamma$ is a viscosity solution to the Hamilton-Jacobi-Bellman equation in $[0,T)\times\mathbb{R}^{n}$,*

[TABLE]

where

[TABLE]

The terminal condition for $\Gamma$ is: $\Gamma(T,x)=\exp(T)g(x)$ in $\mathbb{R}^{n}.$

5 Uniqueness of the solution of Hamilton-Jacobi-Bellman equation

In this section we deal with the issue of uniqueness of the solution of system (7) and to do so, we first give the following two lemmas which is a classical one in viscosity literature (inspired by [21]):

Lemma 16**.**

(convexity of $\mathcal{M}$). Let $U,V:[0,T]\times\mathbb{R}^{n}\rightarrow\mathbb{R}$ and $\lambda\in[0,1]$. Then

[TABLE]

$Proof.$ This follows immediately since

[TABLE]

for all $(t,x)\in[0,T]\times\mathbb{R}^{n}$.  $\Box$

Lemma 17**.**

Let $V$ a supersolution of $(\ref{eq:HJB1})$. Then V_{m}:=\big{(}1-\frac{1}{m}\big{)}V+\frac{1}{m}\psi is a supersolution of

[TABLE]

where $\psi=(\exp(T)\|f\|_{\infty}+k)$.

$Proof.$ We have for $(t,x)\in[0,T)\times\mathbb{R}^{n}$

[TABLE]

Let $\phi_{m}\in C^{1,2}([0,T)\times\mathbb{R}^{n})$ and $(\bar{t},\bar{x})\in[0,T)\times\mathbb{R}^{n}$ such that $\phi_{m}(\bar{t},\bar{x})=V_{m}(\bar{t},\bar{x})$, $\phi_{m}\leq V_{m}.$ Choose \phi=\big{(}\phi_{m}-\frac{1}{m}\psi\big{)}\big{(}\frac{m}{m-1}\big{)}, then $\phi(\bar{t},\bar{x})=V(\bar{t},\bar{x})$ and $\phi\leq V$. As $V$ is a supersolution of $(\ref{eq:HJB1})$, then

[TABLE]

Which implies that

[TABLE]

and then

[TABLE]

Furthermore, by $(\ref{souslemuni})$ we obtain

[TABLE]

On the other hand, by the convexity of $\widetilde{\mathcal{M}}$(Lemma 16), we have

[TABLE]

Theorem 18**.**

Let $U$ (resp. $V$), be a bounded u.s.c. viscosity subsolutions (resp. bounded l.s.c. viscosity supersolutions) to (60). Then, $U\leq V$ in $[0,T]\times\mathbb{R}^{n}$.

$Proof.$ We will show by contradiction. Assume that $\sup_{[0,T]\times\mathbb{R}^{n}}(U(t,x)-V(t,x))=2M>0$. Furthermore, there exists $m_{0}$ such that

[TABLE]

where $V_{m_{0}}$ is as defined in Lemma 17.

Step 1.

Let $h$ be a smooth function such that

[TABLE]

Then, for a small $\epsilon,\beta,\gamma>0$, and $m\geq m_{0}$ we define:

[TABLE]

where $h_{\gamma}(x)=h(\gamma x)$, then we have the following properties:

• •

$h_{\gamma}(x)=\|U\|_{\infty}+\|V_{m}\|_{\infty}+1$ when $|x|\geq 2/\gamma$, which ensures that the supremum of $\Phi_{\epsilon,\gamma}$ is achieved therefore is a maximum.

• •

$\nabla h_{\gamma}(x),D^{2}h_{\gamma}(x)\to 0$ as $\gamma\to 0$ uniformly on $\mathbb{R}^{n}$, which allows to control the differential terms of the $h_{\gamma}$.

Now we consider any maximum points $(t_{\epsilon},x_{\epsilon},y_{\epsilon})$ of the function $\Phi_{\epsilon,\gamma}$. For $\epsilon$, $\beta$ and $\gamma$ small enough, we have

[TABLE]

Next from the inequality

[TABLE]

we get

[TABLE]

It follows that

[TABLE]

We now want to show that

[TABLE]

Define

[TABLE]

Observe that

[TABLE]

By the definition of $\Phi_{\epsilon,\gamma}$, we get (70) if we show that

[TABLE]

We proceed by contradiction. Assume that (71) does not hold, there exist $(\epsilon_{k},t_{\epsilon_{k}},x_{\epsilon_{k}},y_{\epsilon_{k}})\to(0,\bar{t},\bar{x},\bar{y})$ such that $\lim_{k\to\infty}\phi(\epsilon_{k})>D.$ But (69) implies $\bar{x}=\bar{y}.$ Thus the upper semicontinuity of $U-V_{m}$ gives

[TABLE]

which gives a contradiction.

**Step 2. ** We now claim that:

[TABLE]

Suppose that

[TABLE]

then there exist $\xi_{1}\in U$ such that:

[TABLE]

By Lemma 17 we have

[TABLE]

It follows that:

[TABLE]

where $\omega_{c}$ is the modulus of continuity of $c$.Then we have

[TABLE]

By using the mean value theorem for $h_{\gamma}$ and by choosing $\epsilon$, $\gamma$ and $m$ appropriately we get

[TABLE]

This is contradiction to the fact that $(t_{\epsilon},x_{\epsilon},y_{\epsilon})$ is the supremum point of $\Phi_{\epsilon,\gamma}$. Then the claim (72) holds.

**Step 3 ** To complete the proof it remains to show contradiction. Let us denote

[TABLE]

Then we have:

[TABLE]

Let $c,d\in\mathbb{R}$ such that

[TABLE]

Then applying the result by Crandall et al. (Theorem 8.3, [11]) to the function

[TABLE]

at the point $(t_{\epsilon},x_{\epsilon},y_{\epsilon})$, for any $\epsilon_{1}>0$, we can find $X,Y\in S_{n},$ such that:

[TABLE]

Then by definition of viscosity solution, we get:

[TABLE]

and

[TABLE]

which implies that:

[TABLE]

As

[TABLE]

It follows that:

[TABLE]

Choosing now $\epsilon_{1}=\epsilon,$ yields the relation

[TABLE]

Now, from (H1), (75) and (5) we get:

[TABLE]

Next

[TABLE]

And finally,

[TABLE]

So that by plugging into (LABEL:viscder) we obtain:

[TABLE]

By sending $\gamma\rightarrow 0$, $\epsilon\rightarrow 0$, and $\beta\rightarrow 0$, taking into account of the continuity of $f$, we obtain $\frac{M}{2}\leq 0$, which is a contradiction. Now sending $m\to\infty$, we get the required comparison between $U$ and $V$. The proof of Theorem 18 is now complete.   $\Box$

Corollary 19**.**

Assume that Assumptions (H1), (H2), (H3) and (H4) are fulfilled. Then the value function $V$ is a continuous solution of (7), unique among all bounded solutions.