Optimal control of infinite-dimensional Piecewise Deterministic Markov Processes: a BSDE approach. Application to the control of an excitable cell membrane

TL;DR

This paper develops a novel approach using BSDEs to solve optimal control problems for infinite-dimensional PDMPs, with applications to controlling excitable cell membranes modeled by Hodgkin-Huxley equations.

Contribution

It introduces a BSDE-based representation for the value function of infinite-dimensional PDMPs and proves uniqueness of solutions to the associated Hamilton-Jacobi-Bellman equation.

Findings

Established a Feynman-Kac formula for the value function.

Proved a comparison theorem ensuring uniqueness of viscosity solutions.

Applied the framework to control an excitable cell membrane model.

Abstract

In this paper we consider the optimal control of Hilbert space-valued infinite-dimensional Piecewise Deterministic Markov Processes (PDMP) and we prove that the corresponding value function can be represented via a Feynman-Kac type formula through the solution of a constrained Backward Stochastic Differential Equation. A fundamental step consists in showing that the corresponding integro-differential Hamilton-Jacobi-Bellman equation has a unique viscosity solution, by proving a suitable comparison theorem. We apply our results to the control of a PDMP Hodgkin-Huxley model with spatial component, previously studied in [22], [21] and inspired by optogenetics.

Equations

$$⟨(L^{\ast }B+c_{0}B)x,x⟩\ge 0\forall x\in E.$$

$$||x|{|}_{-1}\le ||B^{1/2}||||x||,x\in E.$$

$$\left\{\begin{array}[]{ll}||b(x,a)-b(x^{\prime},a)||\leqslant C\,||x-x^{\prime}||_{-1},\quad x,\,x^{\prime}\in E,a\in A\\ |\lambda(x,a)-\lambda(x^{\prime},a)|\leqslant C\,||x-x^{\prime}||_{-1},\quad x,\,x^{\prime}\in E,a\in A.\end{array}\right.$$

$$\left|{\int }_E\phi (s,y)Q(z,a,dy)-{\int }_E\phi (s,y)Q(z^{\prime },a,dy)\right|\le C\omega (||z-z^{\prime }|{|}_{-1}),z,z^{\prime }\in E,a\in A,$$

$$\left|{\int }_E[\phi (s,y)-\phi (s^{\prime },y)]Q(z,a,dy)\right|\le C{\sigma }_R(|s-s^{\prime }|),z\in E:||z||\le R,a\in A.$$

$$|f(x,a)-f(x^{\prime },a)|+|g(x)-g(x^{\prime })|$$

$${\alpha }_s={\alpha }_0(s-t,x)\text{mathonebb}{1}_{[t,T_1)}(s)+\sum _{n=1}^{\infty }{\alpha }_n(s-T_n,E_n)\text{mathonebb}{1}_{[T_n,T_{n+1})}(s),s\in [t,T],$$

$$X_{s}=\left\{\begin{array}[]{ll}\phi^{\alpha_{0}}(s-t,x)&\textup{if}\,\,s\in[t,\,T_{1}),\\ \phi^{\alpha_{n}}(s-T_{n},E_{n})&\textup{if}\,\,s\in[T_{n},\,T_{n+1}),\,\,n\in\mathbb{N}\setminus\{0\},\end{array}\right.$$

$$\dot{x}(s)=-Lx(s)+b(x(s),\beta (s)),x(0)=x\in E,$$

$${\varphi }^{\beta }(s,x)=S(s)x+{\int }_0^{s}S(s-r)b({\varphi }^{\beta }(r,x),\beta (r))dr.$$

$$||{\varphi }^{\alpha }(s-t,x)-{\varphi }^{\alpha }(s-t,x^{\prime })||\le C\omega (||x-x^{\prime }||),x,x^{\prime }\in E,$$

$$||{\varphi }^{\alpha }(s-t,x)-{\varphi }^{\alpha }(s^{\prime }-t,x)||\le C{\sigma }_R(s-s^{\prime }),x\in E:||x||\le R,$$

$$||{\varphi }^{\alpha }(s-t,x)||\le C(1+||x||),x\in E,$$

$$||{\varphi }^{\alpha }(s-t,x)-{\varphi }^{\alpha }(s-t,x^{\prime })|{|}_{-1}\le C\omega (||x-x^{\prime }|{|}_{-1})x,x^{\prime }\in E,$$

$$||{\varphi }^{\alpha }(s-t,x)-{\varphi }^{\alpha }(s^{\prime }-t,x)|{|}_{-1}\le C{\sigma }_R(s-s^{\prime }),x\in E:||x||\le R.$$

$${\tilde{p}}^{\alpha }(dsdy)=\sum _{n=1}^{\infty }\text{mathonebb}{1}_{[T_n,T_{n+1})}(s)\lambda (X_s,{\alpha }_n(s-T_n,E_n))Q(X_s,{\alpha }_n(s-T_n,E_n),dy)ds.$$

$$\bar{\mathcal{D}}:=\{$$

$$D\psi (s,x)\in E\text{if}x\in E,D\psi (s,x),\frac{\partial \psi }{\partial s}(s,x)\text{bounded if}x\text{bounded}\}.$$

$${\mathbb{E}}_{\alpha }^{t,x}\left[\psi (\tau ,X_{\tau })\right]=\psi (t,x)+{\mathbb{E}}_{\alpha }^{t,x}\left[{\int }_t^{\tau }\left(\frac{\partial \psi }{\partial t}(r,X_r)+⟨b(X_r,{\alpha }_r),D\psi (r,X_r)⟩\right)dr\right]$$

$$-{\mathbb{E}}_{\alpha }^{t,x}\left[{\int }_t^{\tau }⟨L{X}_r,D\psi (r,X_r)⟩dr\right]+{\mathbb{E}}_{\alpha }^{t,x}\left[{\int }_t^{\tau }{\int }_E(\psi (r,y)-\psi (r,X_r))\lambda (X_r,{\alpha }_r)Q(X_r,{\alpha }_r,dy)dr\right].$$

$$J(t,x,\alpha )={\mathbb{E}}_{\alpha }^{t,x}\left[{\int }_t^{T}f(X_s,{\alpha }_s)ds+g(X_T)\right]$$

$$V(t,x)=\underset{\alpha \in {\mathcal{A}}_{ad}^t}{\inf }J(t,x,\alpha ),$$

$$V(t,x)=\underset{\alpha \in {\mathcal{A}}_{ad}^t}{\inf }{\mathbb{E}}_{\alpha }^{t,x}\left[{\int }_t^{T_1\wedge T}f(X_s,{\alpha }_s)ds+V(T_1\wedge T,X_{T_1\wedge T})\right]t\in [0,T],x\in E.$$

$$V(t,x)=\underset{\alpha \in {\mathcal{A}}_{ad}^t}{\inf }{\mathbb{E}}_{\alpha }^{t,x}\left[{\int }_t^{\theta }f(X_s,{\alpha }_s)ds+V(\theta ,X_{\theta })\right]t\in [0,T],x\in E,$$

$$\theta :=\tau \wedge T_1\wedge T,\tau :=\inf \left\{s⩾t:(s,X_s)\notin B\left((t,x);\rho \right)\right\},$$

$$\frac{\partial v}{\partial t}(t,x)-⟨Lx,Dv(t,x)⟩+\underset{a\in A}{\inf }\{{\mathcal{L}}^{a}v(t,x)+f(x,a)\}=0,$$

$$v(T,x)=g(x),$$

$${\mathcal{L}}^a\psi (t,x):=⟨b(x,a),D\psi (t,x)⟩+\lambda (x,a){\int }_E(\psi (t,y)-\psi (t,x))Q(x,a,dy).$$

$$H^v(x,v,Dv)=0$$

$$v(T,x)=g(x),$$

$$H^{\psi }(z,v,p)$$

$$\frac{\partial \psi }{\partial t}(t,x)-⟨x,L^{\ast }D\phi (t,x)+h(||x||)L^{\ast }D\delta (t,x)⟩$$

$$+\underset{a\in A}{\inf }\left\{⟨b(x,a),D\psi (t,x)⟩+{\int }_E(\psi (t,y)-\psi (t,x))\lambda (x,a)Q(x,a,dy)+f(x,a)\right\}\ge 0.$$

$$-\frac{\partial \psi }{\partial t}(t,x)+⟨x,L^{\ast }D\phi (t,x)+h(||x||)L^{\ast }D\delta (t,x)⟩$$

$$+\underset{a\in A}{\inf }\left\{⟨b(x,a),-D\psi (t,x)⟩-{\int }_E(\psi (t,y)-\psi (t,x))\lambda (x,a)Q(x,a,dy)+f(x,a)\right\}\le 0.$$

$$G_a^{\psi }\left(s,z\right):=-\frac{\partial \psi }{\partial s}(s,z)+⟨z,L^{\ast }D\phi (s,z)+h(||z||)L^{\ast }D\delta (s,z)⟩+f(z,a)-{\mathcal{L}}^a\psi (s,z)$$

Full text

**Keywords: infinite-dimensional PDMPs, constrained backward stochastic differential equations, integro-differential Hamilton-Jacobi-Bellman equation, viscosity solutions in infinite dimensions, spatio-temporal Hodgkin-Huxley models. **

MSC 2010: 93E20, 60H10, 60J25.

1 Introduction

In this paper we consider optimal control problems for Hilbert space-valued infinite-dimensional Piecewise Deterministic Markov Processes, and we prove that the corresponding value function can be represented through a Feynman-Kac formula by means of the solution of a constrained Backward Stochastic Differential Equation (BSDE). As an intermediate step, we also show that the corresponding Hamilton-Jacobi-Bellmann (HJB) has a unique viscosity solution by providing a comparison theorem for suitable Integro Partial Differential Equations (IPDE). We apply our theoretical results to the control of a PDMP Hodgkin-Huxley model with spatial component, previously considered in [22], [21] and inspired by optogenetics.

The Feynman-Kac type representation for the value function is obtained by implementing the randomization procedure introduced in [19] for jump-diffusions, later extended in [5] and [4] respectively to the case of finite-dimensional pure jump Markov processes and of finite-dimensional PDMPs. The control randomization method is particularly useful to probabilistically represent the value function associated to stochastic control problems, where the laws of the family of controlled processes are not dominated by a common measure. Roughly speaking, the randomization principle consists in enlarging the state space by an additional independent piecewise constant component corresponding to the control, and in subsequently generating a family of dominated laws and an auxiliary control problem, where the cost is optimized with respect to the intensity of the extended pure jump component. The value function of this latter (randomized) control problem can be represented by means of the solution of a constrained BSDE, namely a backward equation driven by a random measure with a sign constraint on its martingale part. In order to be able to relate this backward equation to the HJB equation associated to the primal problem, one has to show that the randomized value function does not depend on the additional component, and that it provides a solution to the above-mentioned HJB equation. Afterwards, the Feynman-Kac representation formula for the original value function comes from the uniqueness of the viscosity solution to the corresponding HJB equation. We refer the reader to the introduction of [19] for an extended exposition of the issues involved. Note that the randomization procedure is a very general methodology which applies even if the laws of the controlled processes are dominated. The Feynman-Kac representation formula can be used to design algorithms based on the numerical approximation of the solution to the corresponding constrained BSDE, and therefore to get probabilistic numerical approximations for the value function of the addressed optimal control problem, see e.g. [20].

In our infinite-dimensional setting, we prove existence and uniqueness (in a suitable sense) of the solution of such a constrained BSDE and its independence with respect to the additional component. We also provide a randomized dynamic principle which enables us to establish that the value function of the randomized problem is a viscosity solution of the HJB-IPDE on the Hilbert space. Viscosity solutions for partial differential equations in infinite dimension with unbounded linear terms have been first studied in [10], where the notions of $B$-upper/lower-semicontinuity are introduced, and subsequently considered by many other authors, see e.g. [13] for a modern and detailed exposition on this topic. Recently the papers [26] and [27] have addressed respectively existence and uniqueness for an HJB-IPDE resulting from the control of an Hilbert space-valued SDE driven by a Lévy process. Notice that in our framework we do not ask that our PDMP is a strong solution to some SDE. Our approach is instead based on the study of the local characteristics of the PDMP in the spirit of the theory developed in [11]. We prove a comparison theorem which implies the uniqueness of the viscosity solution of our HJB-IPDE. The appropriate definition of viscosity solution, on which the comparison theorem relies, is derived suitably extending the one provided in [27].

Our theoretical results are applied to the control of a PDMP Hodgkin-Huxley model with spatial component. Hilbert space-valued PDMP models describing the spatio-temporal evolution of a neuron with a finite number of ion channels (or more general excitable membranes) have been rigorously settled in [7]. In particular it was proved in [23] that such PDMP converge to the spatio-temporal Hodgkin-Huxley model proposed in [14] when the number of channels goes to infinity, see also [2]. Inspired by optogenetics, optimal control of general infinite-dimensional PDMP has been previously considered in [22], [21]. In particular the results in [22] were applied to a tracking problem for a Hilbert space-valued Hodgkin-Huxley type PDMP. In that paper, as in the present one, piecewise open loop controls (see e.g. [28]) were considered, and the control acted on the three characteristics of the PDMP. However, the main tools were relaxed controls and the optimal control theory of Markov Decision Processes, see [6]. Moreover, even if an HJB-IPDE were written down in that paper, no study was conducted about existence or uniqueness of its solutions. We also mention the more recent paper [8], which exploits Markov Decision Processes in infinite dimension in the framework of stochastic filtering.

Many generalizations of the present work may be possible. For instance, it would be interesting to treat the general case with infinite-dimensional PDMPs on a state space with boundary, from which additional instantaneous jumps into the interior of the domain may occur (in the finite-dimensional case, this feature has been recently considered in [3]). Moreover, in our application section we have considered the classical case of a Laplacian operator, but other operators could be addressed as well. Finally, a challenging future development would consists in applying our results to the infinite-dimensional PDMP that naturally arise in filtering problems.

The paper is organized as follows. In Section 2 we construct our infinite-dimensional controlled PDMP and we define the related optimal control problem. In particular, inspired by [17], we provide a canonical construction of the PDMP state process in infinite dimension, by suitably extending the finite-dimensional construction implemented in [4], [5]. We then set the associated control problem, and we establish in Theorem 2.11 that the corresponding value function is a viscosity solution of the HJB equation (2.19)-(2.20). In Section 3 we describe the control randomization method in our setting, and we introduce the randomized optimal control problem. Then in Section 4 we define and study the related constrained BSDE, and we address the Feynman-Kac representation. As described above, the first step of the randomization approach consists in proving that the randomized value function does not depend on the additional component, and that satisfies a suitable randomized dynamic programming principle, see respectively Proposition 4.2 and Theorem 4.3. Then in Theorem 4.4 we show that also the randomized value function is a viscosity solution to the HJB equation. The last step towards the Feynman-Kac representation consists in the comparison Theorem 4.5, which provides uniqueness of the viscosity solutions to our HJB-IPDE equation. Section 5 is devoted to the application of our results to the control of a spatio-temporal Hodgkin-Huxley type model. Finally, Sections 6 and 7 are devoted to the proofs of the results provided respectively in Sections 2 and 4.

2 Optimal control of infinite-dimensional PDMPs

In the present section we are going to formulate an optimal control problem for infinite-dimensional piecewise deterministic Markov processes, and to discuss its solvability. The PDMP state space $E$ is a real separable Hilbert space, equipped with the norm $||\cdot||$ and the inner product $\langle\cdot,\cdot\rangle$, with corresponding Borel $\sigma$-field $\mathcal{E}$. In addition, we introduce a Polish space $A$, endowed with its Borel $\sigma$-field $\mathcal{A}$, called the space of control actions. The other data of the problem consist in four functions $f$, $b$, $\lambda$ on $E\times A$, $g$ on $E$, in a probability transition kernel $Q$ from $(E\times A,\mathcal{E}\otimes\mathcal{A})$ to $(E,\mathcal{E})$, and in an operator $L$, satisfying the following conditions.

(HL)

(i) $L$ is a linear, densely defined, maximal monotone operator in $E$. Moreover, there exists an operator $B$ on $E$ bounded, linear, positive (i.e., $\langle Bx,\,x\rangle>0$ for every $x\in E$, $x\neq 0$) and self-adjoint, such that $L^{\ast}B$ is bounded on $E$, and, for some $c_{0}\geq 0$,

[TABLE]

We define the space $E_{-1}$ to be the completion of $E$ under the norm $||x||_{-1}=||B^{1/2}x||.$ $E_{-1}$ is an Hilbert space equipped with the inner product $\langle x,x\rangle_{-1}=\langle B^{1/2}x,B^{1/2}x\rangle.$ Moreover,

[TABLE]

(ii) $-L$ generates a strongly continuous semigroup $(S(u))_{u\geq 0}$ such that, for any $u>0$, $S(u)$ is a contraction on $E$ with respect to $||\cdot||_{-1}$.

Remark 2.1**.**

$-L$* is the generator of a strongly continuous semigroup of contractions $(S(u))_{u\geq 0}$ on $E$, see e.g. Theorem B.45 in [13].*

Definition 2.2**.**

We say that a function $u:W\rightarrow\mathbb{R}$ is $B$-upper-semicontinuous (resp., $B$-lower-semicontinuous) on $W\subset[0,\,T]\times E$ if, whenever $t_{n}\rightarrow t$, $x_{n}\rightharpoonup x$, $B\,x_{n}\rightarrow B\,x$, $(t,x)\in W$, then $\limsup_{n\rightarrow\infty}u(t_{n},x_{n})\leq u(t,x)$ (resp. $\liminf_{n\rightarrow\infty}u(t_{n},x_{n})\geq u(t,x)$). The function $u$ is $B$-continuous on $W$ if it is $B$-upper-semicontinuous and $B$-lower-semicontinuous on $W$.

In the assumptions below $C$ is a generic constant which may vary from line to line.

(Hb$\lambda$Q)

(i) $b:E\times A\mapsto E$, $\lambda:E\times A\mapsto R_{+}$ are bounded continuous functions such that

[TABLE]

(ii) $Q$ maps $E\times A$ into the set of probability measures on $(E,\mathcal{E})$, and is a continuous stochastic kernel (see e.g. Proposition 7.30 in [6]). Moreover, for any real function $\varphi$ continuous on $(\varepsilon,\,T-\varepsilon)\times E$ for any $\varepsilon>0$ and bounded, and for every $R>0$, we have, for all $s,s^{\prime}\in(\varepsilon,\,T-\varepsilon)$,

[TABLE]

where $\omega$ is a modulus of continuity, and $\sigma_{R}(\cdot)$ is a modulus of continuity depending on $R$.

(Hfg) $f:E\times A\mapsto\mathbb{R}_{+}$, $g:E\mapsto\mathbb{R}_{+}$ are continuous and bounded functions, such that

[TABLE]

for all $x,\,x^{\prime}\in E$, where $\omega$ is a modulus of continuity.

2.1 The

optimal control problem

We construct the controlled process $X$ in a canonical way. We start by fixing $(t,x)\in[0,\,T]\times E$, and we set $\Omega=[0,\,T]\times E\times\Omega^{\prime}$, where $\omega=(t,x,\omega^{\prime})$, $\Omega^{\prime t}$ being the set of sequences $\omega^{\prime}=(t_{n},e_{n})_{n\geq 1}$ contained in $((0,\infty)\times E\cup\{(\infty,\Delta)\})$, where $\Delta\notin E$ is an isolated point adjoined to $E$, such that $t_{n}\leq t_{n+1}$, and $t_{n}<t_{n+1}$ if $t_{n}<\infty$. On the sample space $\Omega$ we define the canonical functions $T_{n}:\Omega\rightarrow(t,\,\infty]$, $E_{n}:\Omega\rightarrow E\cup\{\Delta\}$ as follows: $T_{0}(\omega)=t$, $E_{0}(\omega)=x$, and for $n\geq 1$, $T_{n}(\omega)=t_{n}$, $E_{n}(\omega)=e_{n}$, and $T_{\infty}(\omega)=\lim_{n\rightarrow\infty}t_{n}$. We also introduce the associated integer-valued counting measure on $(0,\,\infty)\times E$ given by $p(ds\,dy)=\sum_{n\in\mathbb{N}}\delta_{(T_{n},E_{n})}(ds,dy)$.

The class of admissible control law $\mathcal{A}^{t}_{ad}$ is the set of all Borel-measurable maps $\alpha:[t,\,\infty)\times E\rightarrow A$ of the form:

[TABLE]

where $(\alpha_{n})_{n}$, $\alpha_{n}:\mathbb{R}_{+}\times E\rightarrow A$, is a sequence of measurable functions, see for instance [11], [9], [1]. In other words, at each jump time $T_{n}$, we choose an open loop control $\alpha_{n}$ depending on the initial condition $E_{n}$ and on the time elapsed up to $T_{n}$, to be used until the next jump time. We define the controlled process $X:\Omega\times[t,\,\infty)\rightarrow E\cup\{\Delta\}$ setting

[TABLE]

where $\phi^{\beta}(s,x)$ is the unique mild solution to the parabolic partial differential equation

[TABLE]

with $\beta$ an $\mathcal{A}_{ad}^{0}$-measurable function, namely

[TABLE]

One can easily prove the following result, see e.g. Lemma 3.5 in [22].

Proposition 2.3**.**

Let (HL) and (Hb$\lambda$Q) hold. Then, for every $R>0$, $t\in[0,\,T]$, $t<s^{\prime}<s$, $\alpha\in\mathcal{A}_{ad}^{t}$, there exists a constant $C$, only depending on $T$, such that

[TABLE]

where $\omega$ is a modulus of continuity, and $\sigma_{R}$ is a modulus of continuity depending on $R$.

Set $\mathcal{F}_{0}=\mathcal{B}([0,\,T])\otimes\mathcal{E}\otimes\{\emptyset,\Omega^{\prime}\}$ and, for all $s\geq t$, $\mathcal{G}_{s}^{t}=\sigma(p((t,r]\times B):r\in(t,s],B\in\mathcal{E})$. For all $s\geq t$, let $\mathcal{F}_{s}^{t}$ be the $\sigma$-algebra generated by $\mathcal{F}_{0}$ and $\mathcal{G}_{s}^{t}$. In the following all the concepts of measurability for stochastic processes will refer to the right-continuous, natural filtration $\mathbb{F}^{t}=(\mathcal{F}_{s})_{s\geq t}$. By the symbol $\mathcal{P}^{t}$ we will denote the $\sigma$ algebra of $\mathbb{F}^{t}$-predictable subsets of $[t,\,\infty)\times\Omega$.

For every initial time and starting point $(t,x)\in[0,\,T]\times E$ and for each $\alpha\in\mathcal{A}^{t}_{ad}$, by Theorem 3.6 in [17] there exists a unique probability measure on $(\Omega,\mathcal{F}^{t}_{\infty})$, denoted by $\mathbb{P}^{t,x}_{\alpha}$, such that its restriction to $\mathcal{F}^{t}_{t}$ is $\delta_{x}$, and the $\mathbb{F}^{t}$-compensator under $\mathbb{P}^{t,x}_{\alpha}$ of the measure $p(ds\,dy)$ is

[TABLE]

We will denote by $\mathbb{E}^{t,x}_{\alpha}$ the expectation under $\mathbb{P}^{t,x}_{\alpha}$. The following proposition can be obtained by suitably extending the analogous finite-dimensional result, see Theorem 1.2 in [22].

Proposition 2.4**.**

Assume that Hypotheses (HL) and (Hb$\lambda$Q) hold. For any $(t,x)\in[0,\,T]\times E$ and $\alpha\in\mathcal{A}_{ad}^{t}$, let $s\mapsto\phi^{\alpha}(s,x)$ be the unique mild solution to (2.7) with $\beta=\alpha$, and $X$ be the process in (2.6) with law $\mathbb{P}^{t,x}_{\alpha}$. Then $X$ is an homogeneous strong Markov process.

Moreover, let $\mathcal{D}$ be the set of all measurable functions $\psi:\mathbb{R}_{+}\times E\rightarrow\mathbb{R}$ which are absolutely continuous on $\mathbb{R}_{+}$ as maps $s\mapsto\psi(s,\phi^{\alpha}(s-t,x))$, for all $x\in E$, and such that the map $(x,s,\omega)\mapsto\psi(s,y)-\psi(s,X_{s-})$ is a valid integrand for the random measure $Q$, and set

[TABLE]

where $D\psi$ is the unique element of $E$ such that $\frac{d\psi}{dx}[s,x](y)=\langle y,D\psi(s,x)\rangle$, $y\in E$, where $\frac{d\psi}{dx}[s,x]$ denotes the Fréchet-derivative of $\psi$ w.r.t. $x\in E$ evaluated at $(s,x)\in[0,\,T]\times E$. Let $t<\bar{T}<T$, $\hat{\tau}$ be a stopping time such that $\hat{\tau}\in[t,\,\bar{T}]$, let $\tau_{R}$ be the exit time of $X$ from $\{y:\,||y||\leq R\}$, $R>0$, and set $\tau=\hat{\tau}\wedge\tau_{R}$. Then, for every $\psi\in\bar{\mathcal{D}}$,

[TABLE]

At this point, we define for any $(t,x)\in[0,\,T]\times E$ and $\alpha\in\mathcal{A}^{t}_{ad}$, the functional cost

[TABLE]

and the value function of the control problem

[TABLE]

Proposition 2.5**.**

Assume that Hypotheses (HL), (Hb$\lambda$Q) and (Hfg) hold. Then the value function $V$ in (2.16) is bounded and uniformly continuous in the $|\cdot|\times||\cdot||_{-1}$ norm. Moreover, $V$ satisfies the following dynamic programming principle (DPP):

[TABLE]

Proof.

See Section 6.1.

One can prove that formula (2.17) also holds with $h\wedge T\wedge T_{1}$, for any deterministic time $h>t$, in place of $T\wedge T_{1}$. More generally, previous result can be extended as follows.

Proposition 2.6**.**

Under the same hypotheses of Proposition 2.5, the (DPP) (2.17) can be extended to the form

[TABLE]

with

[TABLE]

where $B((t,x);\rho):=\{(s,y)\in(t,\,T)\times E:\,\,||y-x||<\rho,|s-t|<\rho\}$, $(t,x)\in[0,\,T]\times E$, $\rho>0$.

Proof.

See Section 6.2.

2.2 The related HJB equation

Let us now consider the HJB-IPDE associated to the optimal control problem: this is the following parabolic nonlinear equation on $[0,\,T]\times E$:

[TABLE]

where $\mathcal{L}^{a}$ is the time-homogeneous operator depending on $a\in A$ defined as

[TABLE]

Remark 2.7**.**

The HJB equation (2.19)-(2.20) can be rewritten as

[TABLE]

where

[TABLE]

Definition 2.8**.**

We say that a function $\psi$ is a test function if $\psi(t,x)=\varphi(t,x)+\delta(t,x)\,h(||x||)$, where

• (i)

$\psi,\frac{\partial\varphi}{\partial t}$, $D\varphi$, $L^{\ast}D\varphi$, $\frac{\partial\delta}{\partial t}$, $D\delta$, $L^{\ast}D\delta$ are uniformly continuous on $(\varepsilon,\,T-\varepsilon)\times E$ for every $\varepsilon>0$, $\delta\geq 0$ is $B$-continuous and bounded, $\varphi$ is $B$-lower semicontinuous and bounded.

• (ii)

$h\in C^{2}(\mathbb{R})$* with $h^{\prime},h^{\prime\prime}$ uniformly continuous, $h$ is even and bounded, $h^{\prime}(r)\geq 0$ for $r\in(0,\,+\infty)$.*

Definition 2.9**.**

Viscosity solution to (2.19)-(2.20).

• (i)

A bounded $B$-upper-semicontinuous function $u:(0,\,T)\times E\rightarrow\mathbb{R}$ is a viscosity subsolution of (2.19) if, whenever $u-\psi$ has a global maximum at a point $(t,x)$ for a test function $\psi$, then

[TABLE]

• (ii)

A bounded $B$-lower-semicontinuous function $w:(0,\,T)\times E\rightarrow\mathbb{R}$ is a viscosity supersolution of (2.19) if, whenever $w+\psi$ has a global minimum at a point $(t,x)$ for a test function $\psi$, then

[TABLE]

• (iii)

A viscosity solution of (2.19)-(2.20) is a function which is both a viscosity subsolution and a viscosity supersolution.

The following lemma will play a fundamental role in the following.

Lemma 2.10**.**

Let $\psi(s,y)=\varphi(s,y)+\delta(s,y)\,h(||x||)$ be a test function of the type introduced in Definition 2.8. For any $a\in A$, define

[TABLE]

where $\mathcal{L}^{a}$ is defined in (2.21). Then, for any $t\in(\varepsilon,\,T-\varepsilon)$, $\varepsilon>0$, $x\in E$, and any measurable function $\alpha_{0}:\mathbb{R}_{+}\times E\rightarrow A$, the map

[TABLE]

is continuous on $[t,\,T-\varepsilon)$, $\varepsilon>0$, uniformly in $a$ and $\alpha_{0}$. In particular, for any $t\in(\varepsilon,\,T-\varepsilon)$, $\varepsilon>0$, $x\in E$, and any measurable function $\alpha_{0}:\mathbb{R}_{+}\times E\rightarrow A$, the map

[TABLE]

is continuous on $[t,\,T-\varepsilon)$, $\varepsilon>0$, uniformly in $\alpha_{0}$, and uniformly on $B_{R}(x):=\{x\in E:||x||\leq R\}$, $R>0$.

Proof.

See Section 6.3.

Theorem 2.11**.**

Let (HL), (Hb$\lambda$Q) and (Hfg) hold. Then the value function $V$ provides a viscosity solution to (2.19)-(2.20).

Proof.

See Section 6.4.

3 Control randomization

In this section we start to implement the control randomization method. As a first step, for an initial time $t\geq 0$ and a starting point $x\in E$, we construct an (uncontrolled) PDMP $(X,I)$ with values in $E\times A$ by specifying its local characteristics, see (3.3)-(3.4)-(3.5) below. Next we formulate an auxiliary optimal control problem where, roughly speaking, we optimize a functional cost by modifying the intensity of the process $I$ over a suitable family of probability measures.

3.1 Construction of randomized state systems

Let $E$ still denote a real separable Hilbert space Borel $\sigma$-field $\mathcal{E}$, and $A$ be a Polish space with corresponding Borel $\sigma$-field $\mathcal{A}$. Let moreover $b$, $\lambda$ and $Q$ be respectively two real functions on $E\times A$ and a probability transition from $(E\times A,\mathcal{E}\otimes\mathcal{A})$, satisfying (Hb$\lambda$Q) as before. We denote by $\phi(s,x,a)$ the unique mild solution to the parabolic partial differential equation

[TABLE]

In particular, $\phi(s,x,a)$ corresponds to the function $\phi^{\beta}(s,x)$ introduced in Section 2 when $\beta(s)\equiv a$ and, for every $x,x^{\prime}\in E$, $0<s^{\prime}<s<T$, $a\in A$, satisfies

[TABLE]

with $C$ a constant only depending on $T$, and $\omega$ some modulus of continuity by Proposition 2.3. This fact will be of great use in the sequel.

Let us now introduce another finite measure $\lambda_{0}$ on $(A,\mathcal{A})$ satisfying the following assumption:

(H$\lambda_{0}$) $\lambda_{0}$ is a finite measure on $(A,\mathcal{A})$ with full topological support.

The existence of such a measure is guaranteed by the fact that $A$ is a separable space with metrizable topology. We define

[TABLE]

We wish to construct a PDMP $(X,I)$ with enlarged state space $E\times A$ and local characteristics $(\tilde{\phi},\tilde{\lambda},\tilde{Q})$. Firstly, we need to introduce a suitable sample space to describe the jump mechanism of the process $(X,I)$ on $E\times A$. Accordingly, we set $\Omega^{\prime}$ as the set of sequences $\omega^{\prime}=(t_{n},e_{n},a_{n})_{n\geq 1}$ contained in $((t,\,\infty)\times E\times A)\cup\{(\infty,\Delta,\Delta^{\prime})\}$, where $\Delta\notin E$ (resp. $\Delta^{\prime}\notin A$) is adjoined to $E$ (resp. to $A$) as an isolated point. In the sample space $\Omega=[0,T]\times E\times A\times\Omega^{\prime}$ we define the random variables $T_{n}:\Omega\rightarrow(t,\,\infty]$, $E_{n}:\Omega\rightarrow E\cup\{\Delta\}$, $A_{n}:\Omega\rightarrow A\cup\{\Delta^{\prime}\}$, as follows: writing $\omega=(t,x,a,\omega^{\prime})$ in the form $\omega=(t,x,a,t_{1},e_{1},a_{1},t_{2},e_{2},a_{2},...)$, we set for $n\geq 1$,

[TABLE]

We define the process $(X,I)$ on $(E\times A)\cup\{\Delta,\Delta^{\prime}\}$ setting

[TABLE]

In $\Omega$ we introduce for all $s\geq t$ the $\sigma$-algebras $\mathcal{G}^{t}_{r}=\sigma(N(s,G)\,:\,s\in(t,r],G\in\mathcal{E}\otimes\mathcal{A})$ generated by the counting processes $N(s,G)=\sum_{n\in\mathbb{N}}\mathonebb{1}_{T_{n}\leq s}\mathonebb{1}_{(E_{n},A_{n})\in G}$, and the $\sigma$-algebra $\mathcal{F}^{t}_{s}$ generated by $\mathcal{F}_{0}$ and $\mathcal{G}^{t}_{s}$, where $\mathcal{F}_{0}=\mathcal{B}([0,\,T])\otimes\mathcal{E}\otimes\mathcal{A}\otimes\{\emptyset,\Omega^{\prime}\}$. We still denote by $\mathbb{F}^{t}=(\mathcal{F}^{t}_{s})_{s\geq t}$ and $\mathcal{P}^{t}$ the corresponding filtration and predictable $\sigma$-algebra. The random measure $p$ is now defined on $(t,\,\infty)\times E\times A$ as

[TABLE]

Given any starting point $(t,x,a)\in E\times A$, by Theorem 3.6 in [17], there exists a unique probability measure on $(\Omega,\mathcal{F}^{t}_{\infty})$, denoted by $\mathbb{P}^{t,x,a}$, such that its restriction to $\mathcal{F}_{0}$ is $\delta_{(x,a)}$ and the $\mathbb{F}^{t}$-compensator of the measure $p(ds\,dy\,db)$ under $\mathbb{P}^{t,x,a}$ is the random measure

[TABLE]

where

[TABLE]

We denote by $q=p-\tilde{p}$ the compensated martingale measure associated to $p$.

The sample path of a process $(X,I)$ with values in $E\times A$, starting from a fixed initial point $(x,a)\in E\times A$ at time $t$, can be defined iteratively by means of its local characteristics $(\tilde{\phi},\tilde{\lambda},\tilde{Q})$ in the following way. Set $F(t,x,a;s)=e^{-\int_{t}^{s}(\lambda(\phi(r-t,x,a),a)+\lambda_{0}(A))\,dr}$. For any $B\in\mathcal{E},\,C\in\mathcal{A}$, we have

[TABLE]

on $\{T_{1}<\infty\}$, and, for every $n\geq 1$, on $\{T_{n}<\infty\}$,

[TABLE]

We recall the following result, that is a direct consequence of Theorem 4 in [7].

Proposition 3.1**.**

For any $(t,x,a)\in[0,\,T]\times E\times A$, let $\phi(t,x,a)$ be the unique mild solution to (3.1), and $(X,I)$ be the process defined in (3.9) with law $\mathbb{P}^{t,x,a}$. Then $(X,I)$ is an homogeneous strong Markov process.

Moreover, denote by $\mathcal{D}$ the set of all measurable functions $\varphi:E\times A\rightarrow\mathbb{R}$ which are absolutely continuous on $\mathbb{R}_{+}$ as maps $s\mapsto\varphi(\phi(s,x,a),a)$, for all $(x,a)\in E\times A$, and such that the map $(x,a,s,\omega)\mapsto\varphi(y,b)-\varphi(X_{s-},I_{s-})$ is a valid integrand for the random measure (3.11), and set

[TABLE]

where $D\varphi$ is the unique element of $E$ such that $\frac{d\varphi}{dx}[x,a](y)=\langle y,D\varphi(x,a)\rangle$, $y\in E$, where $\frac{d\varphi}{dx}[x,a]$ denotes the Fréchet-derivative of $\varphi$ w.r.t. $x\in E$ evaluated at $(x,a)\in E\times A$. Then the extended generator of $(X,I)$ is given by

[TABLE]

3.2 The randomized optimal control problem

We now introduce a randomized optimal control problem associated to the process $(X,I)$, and formulated in a weak form. For fixed $(t,x,a)$, we consider a family of probability measures $\{\mathbb{P}_{\nu}^{t,x,a},\,\nu\in\mathcal{V}\}$ in the space $(\Omega,\mathcal{F}_{\infty})$, whose effect is to change the stochastic intensity of the process $(X,I)$.

Let us proceed with precise definitions. We still assume that (Hb$\lambda$Q), (H$\lambda_{0}$) and (Hfg) hold. We recall that $\mathbb{F}^{t}=(\mathcal{F}^{t}_{s})_{s\geqslant t}$ is the augmentation of the natural filtration generated by $p$ in (3.10), and that $\mathcal{P}^{t}$ denotes the $\sigma$-field of $\mathbb{F}^{t}$-predictable subsets of $[t,\,\infty)\times\Omega$. We define

[TABLE]

For every $\nu\in\mathcal{V}$, we consider the predictable random measure

[TABLE]

In particular, for any $t\in[0,\,T]$, by the Radon Nikodym theorem one can find two nonnegative functions $d_{1}$, $d_{2}$ defined on $\Omega\times[t,\,\infty)\times E\times A$, $\mathcal{P}\otimes\mathcal{E}\otimes\mathcal{A}$, such that

[TABLE]

and we have $d\tilde{p}^{\nu}=(\nu\,d_{1}+d_{2})\,d\tilde{p}$. For any $t\in[0,\,T]$, $\nu\in\mathcal{V}$, consider then the Doléans-Dade exponential local martingale $L^{t,\nu}$ defined

[TABLE]

for $s\geq t$. When $(L^{t,\nu}_{s})_{s\geq t}$ is a true martingale on $[t,\,T]$, we can define a probability measure $\mathbb{P}^{t,x,a}_{\nu}$ equivalent to $\mathbb{P}^{t,x,a}$ on $(\Omega,\,\mathcal{F}^{t}_{T})$ by

[TABLE]

By the Girsanov theorem for point processes (see Theorem 4.5 in [17]), the restriction of the random measure $p$ to $(t,T]\times E\times A$ admits $\tilde{p}^{\nu}=(\nu\,d_{1}+d_{2})\,\tilde{p}$ as compensator under $\mathbb{P}^{t,x,a}_{\nu}$. We set $q^{\nu}:=p-\tilde{p}^{\nu}$, and we denote by $\mathbb{E}_{\nu}^{t,x,a}$ the expectation operator under $\mathbb{P}_{\nu}^{t,x,a}$. Previous considerations are formalized in the following lemma, for a proof see Lemma 3.2 in [5].

Lemma 3.2**.**

Let Hypotheses (Hb$\lambda$Q) and (H$\lambda_{0}$) hold. Then, for every $(t,x,a)\in[0,\,T]\times E\times A$ and $\nu\in\mathcal{V}$, under the probability $\mathbb{P}^{t,x,a}$, the process $(L^{t,\nu}_{s})_{s\geq t}$ is a martingale. Moreover, $L_{T}^{t,\nu}$ is square integrable, and, for every $\mathcal{P}_{T}\otimes\mathcal{E}\otimes\mathcal{A}$-measurable function $H:\Omega\times[0,T]\times E\times A\rightarrow\mathbb{R}$ such that $\mathbb{E}^{t,x,a}\left[\int_{t}^{T}\int_{E\times A}|H_{s}(y,b)|^{2}\,\tilde{p}(ds\,dy\,db)\right]$ $<\infty$, the process $\int_{t}^{\cdot}\int_{E\times A}H_{r}(y,b)\,q^{\nu}(dr\,dy\,db)$ is a $\mathbb{P}^{t,x,a}_{\nu}$-martingale on $[t,T]$.

Finally, for every $(t,x)\in[0,\,T]\times E$, $a\in A$ and $\nu\in\mathcal{V}$, we introduce the dual functional cost

[TABLE]

and the dual value function

[TABLE]

4 A constrained BSDEs representation for the value function

In the present section we introduce a BSDE with a sign constraint on its martingale part for which we give existence and uniqueness of a maximal solution in an appropriate sense. This constrained BSDE will provide a probabilistic representation formula for the dual value function introduced in (3.20).

Throughout the section we still assume that (Hb$\lambda$Q), (H$\lambda_{0}$) and (Hfg) hold. The random measures $p$, $\tilde{p}$ and $q$, as well as the dual control setting $\Omega,\mathbb{F}^{t},(X,I),\mathbb{P}^{t,x,a}$, are the same as in Section 3.1. For any $(t,x,a)\in[0,\,T]\times E\times A$ we introduce the following notation.

• •

$\textbf{L}^{\textbf{2}}_{\textbf{t,x,a}}(\mathcal{F}^{t}_{\tau})$, the set of $\mathcal{F}^{t}_{\tau}$-measurable random variables $\xi$ such that $\mathbb{E}^{t,x,a}\left[|\xi|^{2}\right]<\infty$; here $\tau\geqslant 0$ is an $\mathbb{F}^{t}$-stopping time.

• •

$\textbf{S}^{\infty}$, the set of real-valued càdlàg adapted processes $Y=(Y_{t})_{t\geqslant 0}$ which are uniformly bounded.

• •

$\textbf{L}_{\textbf{t,x,a}}^{\textbf{2}}(\textup{q})$, the set of $\mathcal{P}_{T}\otimes\mathcal{B}(E)\otimes\mathcal{A}$-measurable maps $Z:\Omega\times[0,\,T]\times E\times A\rightarrow\mathbb{R}$ such that

[TABLE]

• •

$\textbf{K}^{\textbf{2}}_{\textbf{t,x,a}}$, the set of nondecreasing càdlàg predictable processes $K=(K_{s})_{t\leqslant s\leqslant T}$ such that $K_{t}=0$ and $\mathbb{E}^{t,x,a}\left[|K_{T}|^{2}\right]<\infty$.

We consider the following family of BSDEs with partially nonnegative jumps over a finite horizon $T$, parametrized by $(t,x,a)$: ${\mathbb{P}}^{t,x,a}$-a.s.,

[TABLE]

with

[TABLE]

We are interested in the maximal solution $(Y^{t,x,a},Z^{t,x,a},K^{t,x,a})\in\textbf{S}^{\infty}\times\textbf{L}^{\textbf{2}}_{\textbf{t,x,a}}(\textup{q})\times\textbf{K}^{\textbf{2}}_{\textbf{t,x,a}}$ to (4.1)-(4.2), in the sense that for any other solution $(\tilde{Y},\tilde{Z},\tilde{K})\in\textbf{S}^{\infty}\times\textbf{L}^{\textbf{2}}_{\textbf{t,x,a}}(\textup{q})\times\textbf{K}^{\textbf{2}}_{\textbf{t,x,a}}$ to (4.1)-(4.2), we have $Y_{s}^{t,x,a}\geqslant\tilde{Y}_{s}$, $\mathbb{P}^{t,x,a}$-a.s., for all $s\geqslant t$.

Let us introduce the following penalized BSDE, associated to (4.1)-(4.2), parametrized by the integer $n\geq 1$:

[TABLE]

where $K_{s}^{n,t,x,a}:=n\int_{0}^{s}\int_{A}[Z^{n,t,x,a}_{r}(X_{r},\,b)]^{-}\lambda_{0}(db)\,dr$, $s\in[t,\,T]$.

Theorem 4.1**.**

Let Hypotheses (Hb$\lambda$Q), (H$\lambda_{0}$) and (Hfg) hold. Then, for every $(t,x,a)\in[0,\,T]\times E\times A$, there exists a unique maximal solution $(Y^{t,x,a},Z^{t,x,a},K^{t,x,a})\in\textup{{S}}^{\infty}\times\textup{{L}}^{\textbf{2}}_{\textbf{t,x,a}}(\textup{q})\times\textup{{K}}^{\textbf{2}}_{\textbf{t,x,a}}$ to the BSDE with partially nonnegative jumps (4.1)-(4.2), where $Y^{t,x,a}$ is the nonincreasing limit of $(Y^{n,t,x,a})_{n}$, $Z^{t,x,a}$ is the weak limit of $(Z^{n,t,x,a})_{n}$ in $\textup{{L}}^{\textbf{2}}_{\textbf{t,x,a,{loc}}}(\textup{q})$ and $K_{s}^{t,x,a}$ is the weak limit of $(K_{s}^{n,t,x,a})_{n}$ in $\textup{{L}}^{\textbf{2}}_{\textbf{t,x,a}}(\mathcal{F}_{s})$, for any $s\geqslant 0$. Moreover, $Y^{t,x,a}$ has the explicit representation:

[TABLE]

In particular, setting $s=t$ in (4.4), we have the following representation formula for the value function of the randomized control problem:

[TABLE]

Proof.

The proof of this result is analogous to the one for the BSDE (4.1) with underlying finite-dimensional process $X$, see Theorem 4.7 in [4], and we do not report it for sake of brevity.

Our main purpose is to show how maximal solutions to BSDEs with nonnegative jumps of the form (4.1)-(4.2) provide actually a Feynman-Kac representation to the value function $V$ associated to our optimal control problem for infinite-dimensional PDMPs. Let us introduce a deterministic function $v:[0\,T]\times E\times A\rightarrow\mathbb{R}$ as

[TABLE]

Proposition 4.2**.**

Assume that Hypotheses (HL), (Hb$\lambda$Q), (H$\lambda_{0}$), and (Hfg) hold. Then the function $v$ in (4.6) does not depend on the variable $a$:

[TABLE]

By abuse of notation, we define the function $v$ on $[0,\,T]\times E$ by $v(\cdot,\cdot)=v(\cdot,\cdot,a)$, for any $a\in A$. Moreover, $v$ admits the representation formula: $\mathbb{P}^{t,x,a}$-a.s.

[TABLE]

Proof.

By Lemma 5.3 and Remark 5.5 in [4], we have that for any $(t,x,a)\in[0,\,T]\times E\times A$, $\mathbb{P}^{t,x,a}$-a.s.,

[TABLE]

Now we recall that, by (4.5) and (4.6), $v$ coincides with the value function $V^{\ast}$ of the dual control problem introduced in Section 3.2. Therefore, identity (4.7) corresponds to the fact that $V^{\ast}(t,x,a)$ does not depend on $a$. Proceeding as in the finite-dimensional case (see the proof of Proposition 5.6 in [4]), one can prove that:

[TABLE]

Property (4.10) implies that $V^{\ast}(t,x,a^{\prime})\leq J(t,x,a,\nu)$ for all $t\in[0,\,T]$, $x\in E$, $a,a^{\prime}\in A$, and by the arbitrariness of $\nu$ one can conclude that $V^{\ast}(t,x,a^{\prime})\leq V^{\ast}(t,x,a)$ for all $t\in[0,\,T]$, $x\in E$, $a,a^{\prime}\in A$. In other words $V^{\ast}(t,x,a)=v(t,x,a)$ does not depend on $a$, and (4.7) holds.

Theorem 4.3**.**

Assume that Hypotheses (HL), (Hb$\lambda$Q), (H$\lambda_{0}$), and (Hfg) hold. Then $v$ is bounded and uniformly continuous in the $|\cdot|\times||\cdot||_{-1}$ norm. Moreover, $v$ satisfies the so called randomized dynamic programming principle:

[TABLE]

Proof.

See Section 7.1.

We can now give the following important result.

Theorem 4.4**.**

Assume that Hypotheses (HL), (Hb$\lambda$Q), (H$\lambda_{0}$) and (Hfg) hold. Then the function $v$ in (4.6) is a viscosity solution to (2.19)-(2.20).

Proof.

See Section 7.2.

Finally, we provide a comparison theorem for viscosity sub and supersolutions to the first order IPDE of HJB type (2.19)-(2.20) on Hilbert spaces. To this end, we will need the following additional hypothesis on the transition measure $Q$:

(HQ’)    For any $x,x_{\varepsilon}\in E$, $S_{\varepsilon}\subset E$, such that $x_{\varepsilon}\rightarrow x$ and $\cap_{\varepsilon}S_{\varepsilon}=\emptyset$,

[TABLE]

Theorem 4.5**.**

Let (HL), (Hb$\lambda$Q), (Hfg) and (HQ’) hold. Let $u:[0,\,T]\times E\rightarrow\mathbb{R}$ (resp. $v:[0,\,T]\times E\rightarrow\mathbb{R}$) be a bounded and uniformly continuous function in the $|\cdot|\times||\cdot||_{-1}$ norm, providing a viscosity subsolution (resp. viscosity supersolution) to (2.19)-(2.20). Suppose that $u(T,x)\leq v(T,x)$ for all $x\in E$. Then $u\leq v$.

Proof.

See Section 7.3.

By means of Theorems 2.11, 4.3, 4.4, together with the comparison Theorem 4.5, we can finally obtain the following probabilistic representation formula for the value function $V$.

Theorem 4.6**.**

Let (HL), (Hb$\lambda$Q), (H$\lambda_{0}$), (Hfg), and (HQ’) hold. Then the function $v$ in (4.6) coincides with the value function $V$, and the following Feynman-Kac representation formula holds:

[TABLE]

5 Application to a Hodgkin-Huxley model of neuronal dynamics

In the present section we apply our theory to an infinite-dimensional stochastic Hodgkin-Huxley model of neuronal dynamics. The deterministic Hodgkin-Huxley system was first introduced in [14], while stochastic versions as Hilbert space valued PDMP have been studied in e.g. [2], [7],[15] and [23], [22].

We focus on the model considered in [22]. The axon is modeled by the interval $[0,1]$. We consider ion channels of type $Na$ (sodium) or $K$ (potassium), and we assume that they are located along the axon at positions in $I_{N}=\frac{1}{N}(\mathbb{Z}\cap N(0,1))$ for some fixed $N\in\mathbb{N^{*}}$, that we will denote $\frac{i}{N}$ or $z_{i}$. The set of possible states of $K$ and $Na$ channels are denoted respectively by $D_{1}$ and $D_{2}$, and are given by

[TABLE]

In the control problem new (rhodopsin) channels that are sensitive to light are inserted in the neuron. Such a rhodopsin channel (denoted by $ChR2$) can have the four possible states $O_{1},O_{2},C_{1},C_{2}$, among which $O_{1}$ and $O_{2}$ are conductive. Experimentally, the channel is illuminated and the effect of the illumination is to put the channel in one of its conductive states. We set ${\overline{D}}:=D_{1}\cup D_{2}\cup D_{ChR2}$ with $D_{ChR2}:=\{O_{1},O_{2},C_{1},C_{2}\}$, and ${\overline{D}}_{N}:={\overline{D}}^{I_{N}}$.

We consider the Hilbert space $E:=L^{2}(0,1)$ and the operator $L:=-\Delta$. The controlled PDMP consists in a set of PDEs written as ODEs in the Hilbert space $E$ indexed by $d\in\bar{D}_{N}$,

[TABLE]

where the membrane capacitance $C_{m}>0$ is constant and, for each $(v,d)\in E\times{\overline{D}}_{N}$,

[TABLE]

with

[TABLE]

where $\phi_{z_{i}}$ is a mollifier function supported on a neighborhood of $z_{i}$. For a channel of type $K$, ${\overline{g}}_{K}>0$ is the normalized conductance and ${\overline{V}}_{K}\in\mathbb{R}$ is the reversal potential; the same notation holds for $Na,l,ChR2$ ($\overline{V}_{l}$ and ${\overline{g}}_{l}$ denote respectively the leaky reversal potential and conductance). The conductance depends on the number of channels in the conductive state: for $K$ (resp. $Na$) such a state is unique, and it is $n_{4}$ (resp. $m_{3}h_{1}$). The leaky conductance ${\overline{g}}_{l}$ remains constant. Formula (5.3) models the mean value of the membrane potential on a neighborhood of $z_{i}$.

For any $x=(v,d)\in E\times{\overline{D}}_{N}$, we denote by $v_{t}=\phi^{d}_{t}(v)$ the corresponding unique mild solution to the PDE (5.1), that models the membrane potential evolution between two successive changes in the channels configuration. The transitions take place inside the discrete domain $\bar{D}_{N}$, and correspond to a continuous-time Markov chain $d_{t}$. Denoting by $(T_{n},d_{T_{n}})$ the jump times and post-jump location, the controlled PDMP starting from $x=(v,d)\in E\times\bar{D}_{N}$ is

[TABLE]

The control process $\alpha_{t}$ is proportional to the intensity of light (which is necessarily bounded), so that we take as control space $A:=[0,a_{\max}]$ with $a_{\max}>0$. Introducing a family of smooth functions $\sigma_{\zeta,\xi}:\mathbb{R}\rightarrow\mathbb{R}_{+}^{*}$ depending on $(\zeta,\xi)\in{\overline{D}}\times{\overline{D}}$ for all $x=(v,d)\in E\times{\overline{D}}_{N}$, $a\in A$, we define the jump rate function $\lambda:E\times{\overline{D}}_{N}\times A\rightarrow\mathbb{R}_{+}$ by

[TABLE]

The transition measure $Q:E\times{\overline{D}}_{N}\times A\rightarrow\mathcal{P}({\overline{D}}_{N})$ is such that, for any $x=(v,d)\in E\times{\overline{D}}_{N}$, $a\in A$, the measure $Q((v,d),a,\cdot)$ is supported by the set ${\cal S}$ of $y=(\tilde{v},\tilde{d})$ such that $\tilde{v}=v$ (the trajectories of $(v_{t})$ are continuous) and $\tilde{d}$ differs from $d$ only by one component. For $y=(\tilde{v},\tilde{d})\in{\cal S}$ such that $\tilde{d}$ differs from $d$ only by its component $i$,

[TABLE]

if $y\notin{\cal S}$, $Q(x,a;dy):=0$. The transition functions $\sigma$ from $C_{1}$ to $O_{1}$ and from $C_{2}$ to $O_{2}$ are assumed to be proportional to the control $\alpha$ while the other ones are uncontrolled functions. More precisely (see [14], [22]):

[TABLE]

and

[TABLE]

where

[TABLE]

The optimal control problem consists in mimicking a desired output reference potential $V_{ref}$, that encodes a given biological behavior while minimizing the intensity of the light applied to the neuron. This corresponds to setting, for any $x=(v,d)\in E\times\bar{D}_{N}$,

[TABLE]

so that the cost functional and the value function of the control problem are

[TABLE]

The reference signal $V_{ref}$ (that we assume not depending on time) may correspond to a healthy behavior that we want the system to recover thanks to the light stimulation. The intensity of the light is modeled by the control $\alpha_{s}=\alpha(X_{s})$. Getting the intensity minimal is crucial for the feasability of the experiment in relation to the technical characteristics of the devices that are used.

Remark 5.1**.**

The control of general infinite-dimensional PDMP is considered in [22], [21]. As in the present paper, in [22] the authors deal with piecewise open loop controls (see [28]), and the control may act on the three characteristics of the PDMP; however, the main tools were relaxed controls and the optimal control of Markov Decision Processes, see e.g. [6]. As an application, other types of models can also be considered: the PDEs in (5.1) may depend on the control variable corresponding to the case where $b_{d}$ depends on the control, $\phi_{z_{i}}$ may be replaced by $\delta_{z_{i}}$ or finally the set $D_{ChR2}$ may have three elements, in which case a $ChR2$ channel has a unique conductive state.

The rest of this section is devoted to check that the Hodgkin-Huxley stochastic model described above can be put into the framework of the theory developped in the previous sections.

Proposition 5.2**.**

• (i)

The operator $L:=-\Delta$ is densely defined, maximal monotone and self-adjoint. Moreover, $B:=(I-\Delta)^{-1}$ satisfies the strong $B$-condition with $c_{0}=1$, namely $-\Delta B+B\geq I$ which implies the weak B-condition (2.1).

• (ii)

The semigroup $(S(r))_{r\geq 0}:=(e^{-rL})_{r\geq 0}=(e^{r\Delta})_{r\geq 0}$ generated by $L:=-\Delta$ is strongly continuous, and for all $r>0$, and $S(r)$ is a contraction with respect to $||\cdot||$ and also with respect to $||\cdot||_{-1}$.

Proof.

(i) From [13], example 3.14 at page 155 (see also [21]) $B:=(I-\Delta)^{-1}$ satifies the strong $B$-condition with $c_{0}=1$ namely $-\Delta B+B\geq I$, which implies in particular the weak B-condition (2.1).

(ii) For any $k\in\mathbb{N}$, let us define

[TABLE]

$(f_{k})_{k\geq 1}$ is an orthonormal basis of $E$, $\Delta f_{k}=-k^{2}\pi^{2}f_{k}$, and, for any $v\in E$,

[TABLE]

Moreover $S(r)=e^{r\Delta}$ is such that $S(r)v\in{\cal D}(\Delta)$ for all $r>0$, $v\in E$, and satisfies

[TABLE]

We have $||S(r)v||^{2}\leq e^{-2r\pi^{2}}||v||^{2}$. Moreover, $||S(r)v||_{-1}^{2}\leq e^{-2r\pi^{2}}||v||_{-1}^{2}$.

Lemma 5.3**.**

For any $i\in I_{N}$, let $\Phi_{i}$ be the function in (5.3). Then there exists a positive constant $C_{i}$ such that, for all $v,v^{\prime}$ in $E$,

[TABLE]

Proof.

We have $\Phi_{i}(v^{\prime})-\Phi_{i}(v)=(v^{\prime}-v,\phi_{z_{i}})$, so taking the basis $(f_{k})_{k\geq 1}$ in (5.7),

[TABLE]

By the Cauchy-Schwarz inequality,

[TABLE]

It remains to prove that $((I-\Delta)\phi_{z_{i}},\phi_{z_{i}})<+\infty$, so that (5.8) holds with $C_{i}=[((I-\Delta)\phi_{z_{i}},\phi_{z_{i}})]^{\frac{1}{2}}$. To this end, we take $\phi_{z_{i}}(z):=\frac{1}{\gamma}M(\frac{z-z_{i}}{\gamma})$ with $M(z)=\mathonebb{1}_{(-1,1)}(z)\,e^{-\frac{1}{1-z^{2}}}$. We have $\phi^{\prime}_{z_{i}}(z):=\frac{1}{\gamma^{2}}M^{\prime}(\frac{z-z_{i}}{\gamma})$ and $\phi^{\prime\prime}_{z_{i}}(z):=\frac{1}{\gamma^{3}}M^{\prime\prime}(\frac{z-z_{i}}{\gamma})$. Moreover

[TABLE]

Therefore, setting $\zeta=\frac{z-z_{i}}{\gamma}$,

[TABLE]

Proposition 5.4**.**

Let $d\in\bar{D}_{N}$, $v,v^{\prime}\in E$. Then

[TABLE]

Moreover, for all $R>0$, there exists a positive constant $C_{R}$ such that, for all $a\in A$,

[TABLE]

Proof.

Let $d\in\bar{D}_{N}$ and $v,v^{\prime}\in E$. By (5.2) we have

[TABLE]

Therefore

[TABLE]

Since $||\phi_{z_{i}}||_{-1}\leq C_{i}||\phi_{z_{i}}||$ and $I_{N}$ is a finite set, (5.10) follows from (5.13) and Lemma 5.3.

Let us finally prove (5.11). We assume that $||v||\vee||v^{\prime}||\leq R$. By definition (5.4), it is sufficient to check that, for any $i\in I_{N}$,

[TABLE]

which in turn corresponds to prove the same property for the functions $\alpha_{q}(\Phi_{i}(v))$, $\beta_{q}(\Phi_{i}(v))$, $q=n,m,h$. Recalling (5.3) and applying the Cauchy-Scwartz inequality, we see that $\Phi_{i}(v)$, $\Phi_{i}(v^{\prime})$ belong to a bounded interval $J_{R}$ depending on $R$. Then, denoting by $K_{q,R}$ the Lipschitz constant of $\alpha_{q}$ on $J_{R}$, from Lemma 5.3

[TABLE]

where $C_{i}$ is the positive constant in (5.8). The conclusion follows recalling that $I_{N}$ is a finite set.

Lemma 5.5**.**

For any $d\in\bar{D}_{N}$, and $s,s^{\prime}\in[t,\,T]$,

• (i)

$||\phi^{d}_{s-t}(v)||\leq C(1+||v||)$,  $v\in E$,

• (ii)

$||\phi^{d}_{s-t}(v)-\phi^{d}_{s^{\prime}-t}(v)||\leq C\,\sigma_{R}(|s-s^{\prime}|)$, $v\in E:||v||\leq R$,

• (iii)

$||\phi^{d}_{s-t}(v)-\phi^{d}_{s-t}(v^{\prime})||\leq C\,\omega(||v-v^{\prime}||)$,  $v,v^{\prime}\in E$,

• (iv)

$||\phi^{d}_{s-t}(v)-\phi^{d}_{s^{\prime}-t}(v)||_{-1}\leq C\,\sigma_{R}(|s-s^{\prime}|)$, $v\in E:||v||\leq R$,

• (v)

$||\phi^{d}_{s-t}(v)-\phi^{d}_{s-t}(v^{\prime})||_{-1}\leq C\,\omega(||v-v^{\prime}||_{-1})$,  $v,v^{\prime}\in E$.

Proof.

We first prove (i) and (iii). Setting $S(r)=e^{-rL}$, the equation for the mild solution to (5.1) starting from $x=(v,d)\in E\times\bar{D}_{N}$ reads

[TABLE]

Concerning (i), using the contraction property of $S(u)$ with respect to $||\cdot||$ given in Proposition 5.2-(ii), we obtain

[TABLE]

On the other hand, recalling (5.12),

[TABLE]

Using Lemma 5.3 we get

[TABLE]

and item (i) follows by Gronwall’s Lemma.

Let us now turn to (iii). For any $d\in\bar{D}_{N}$, $v,v^{\prime}\in E$, we have

[TABLE]

Taking the norm $||\cdot||$, and applying Proposition 5.2-(ii) together with (5.10), we obtain

[TABLE]

The conclusion follows again from the Gronwall Lemma.

Properties (iv) and (v) can be proved analogously, using the contraction property of $S(u)$ with respect to $||\cdot||_{-1}$ given in Proposition 5.2-(ii).

Additional results on $V=H_{0}^{1}(I)$.

The space $V=H_{0}^{1}(I)$ is continuously embedded in the set of continuous functions on $I$. For any $k\in\mathbb{N}$, let us set

[TABLE]

Then $(e_{k})_{k\geq 1}$ is an orthonormal basis of $V=H_{0}^{1}(I)$, and $\Delta e_{k}=-k^{2}\pi^{2}e_{k}$. For all $v\in V$, we set $(v,e_{k})_{V}:=\int_{0}^{1}v(z)e_{k}(z)dz+\int_{0}^{1}v^{\prime}(z)e^{\prime}_{k}(z)dz$. We have

[TABLE]

Remark 5.6**.**

Lemma 5.3 and Propositions 5.4. hold true with $V=H_{0}^{1}(I)$ in place of $E=L^{2}(0,1)$.

The following result for the PDEs (5.1), given in Lemma 4.1 in [22], plays a fundamental role.

Lemma 5.7**.**

Set $V_{-}:=\min({\overline{V}}_{Na},{\overline{V}}_{K},{\overline{V}}_{L},{\overline{V}}_{ChR2})$, $V_{+}:=\max({\overline{V}}_{Na},{\overline{V}}_{K},{\overline{V}}_{L},{\overline{V}}_{ChR2})$, and let $d\in\bar{D}_{N}$. If $v\in H_{0}^{1}(I)$ is continuous in $I=[0,1]$, and $v(z)\in[V_{-},V_{+}]$ for all $z\in I$, then, for every $d\in\bar{D}_{N}$,

[TABLE]

Physiologically speaking, we are only interested in the domain $[V_{-},V_{+}]$. Since Lemma 5.7 shows that this domain is invariant for the controlled PDMP, we can modify the local characteristics of the PDMP outside the domain $[V_{-},V_{+}]$ without changing its dynamics inside of $[V_{-},V_{+}]$. We will do so for the rate functions $\sigma_{d_{i},\xi}$. From now on, consider a compact set $K$ containing the closed ball of $E$, centered in [math] with radius $\max\{V_{-},V_{+}\}$. We will rewrite $\sigma_{d_{i},\xi}$ outside $K$ such that they all become Lipschitz and bounded functions. We also take $V_{\rm ref}$ taking values in $K$ and $\tilde{f}$ bounded and globally Lipschitz such that

[TABLE]

Since the control set $A=[0,a_{\rm max}]$ is bounded, the corresponding value function and cost

[TABLE]

are bounded as well.

The next two results show that the case of the stochastic controlled infinite-dimensional Hodgkin-Huxley model can be actually covered by the theory on controlled infinite-dimensional PDMPs developed in the present paper.

Proposition 5.8**.**

Let $v,v^{\prime}\in V$ such that $v(z)$ and $v^{\prime}(z)$ belong to $[V_{-},V_{+}]$ for all $z\in[0,1]$. The following hold.

• (i)

There exist a positive constants $C_{1}$ such that, for all $d\in\bar{D}_{N}$, and $a\in A$,

[TABLE]

• (ii)

If in addition $||(I-\Delta)V_{ref}||<+\infty$, there exists a positive constant $C_{2}$ such that, for all $d\in\bar{D}_{N}$, $a\in A$, the function $f$ in (5.6) satisfies

[TABLE]

Proof.

Let us prove item (i). Recalling (5.3) and using the Cauchy-Schwarz inequality we have

[TABLE]

Since

[TABLE]

and the same inequalities hold for $\phi_{s}^{d}(v^{\prime})$, we have

[TABLE]

with $R=\max\{|V_{-}|,|V_{+}|\}$ and $K_{q,R}$ the Lipschitz constant of $\alpha_{q}$ depending on $R$. Taking into account Remark 5.6, we conclude by applying the $V$-versions of Lemmas 5.3 and 5.4.

Let us now consider item (ii). Using the basis $(e_{k})$ introduced in (5.14), and applying the Cauchy-Schwarz inequality,

[TABLE]

where

[TABLE]

By Proposition 5.4 and Remark 5.6, it remains to study the boundedness properties of $\cal T$. Since by assumption $||(I-\Delta)V_{ref}||<+\infty$,

[TABLE]

Let us thus consider the term $||(I-\Delta)\phi_{d}(s,v)||_{V}$. Being $(I-\Delta)$ linear, we can write

[TABLE]

Moreover, since $(I-\Delta)$ and $S(r)$ commute,

[TABLE]

where we have used that (recall formula (5.12))

[TABLE]

Recalling (5.19) and (5.20), (5.24) yields

[TABLE]

Recalling (5.9) we see that, for any $i\in I_{N}$, $||(I-\Delta)\phi_{z_{i}}||_{V}\leq C_{i}$. Since $I_{N}$ is finite, we conclude from the above inequality that there exists some constant $\Gamma$ such that

[TABLE]

analogous inequalities holds true for $\phi^{d}_{s}(v^{\prime})$ and $v^{\prime}$. Then (5.23), together with (5.25), yields

[TABLE]

and the conclusion follows.

Proposition 5.9**.**

Let $v_{0}\in V$ such that $v_{0}(z)\in[V_{-},V_{+}]$ for all $z\in[0,1]$. Then there exist two positive constants $C_{1},C_{2}$, only depending on $T,N,\max\{|V_{-}|,|V_{+}|\})$, such that, for all $d\in D$, $a\in A$,

[TABLE]

Proof.

Estimate (5.27) is obtained arguing as in Lemma 5.5-(i). The boundedness of $f(\phi^{d}_{s}(v_{0}),a)$ follows from (5.27), recalling that

[TABLE]

On the other hand, recalling (5.12) and (5.19),

[TABLE]

and we obtain the bound from Lemma 5.5-(i) and the fact that $||\cdot||\leq||\cdot||_{V}$.

The boundedness of $\lambda_{d}(\phi^{d}_{s}(v_{0}),a)$ follows from the form of the functions $\alpha_{q}$, $\beta_{q}$, together with (5.19) and the fact that $||\cdot||\leq||\cdot||_{V}$.

6 Proofs of the results in Section 2

6.1 Proof of Proposition 2.5

The boundedness of $V$ directly comes from (2.16) and the boundedness of $f$ and $g$. Let $B([0,\,T]\times E)$ be the set of all bounded functions on $[0,\,T]\times E$, and define the map $\mathcal{T}:B([0,\,T]\times E)\rightarrow B([0,\,T]\times E)$ as

[TABLE]

Set $\mathcal{U}=\{u:[0,\,+\infty)\rightarrow A\,\,\textup{measurable}\}$. One can show that

[TABLE]

where $\chi^{u}(s,x)=e^{-\int_{0}^{s}\lambda(\phi^{u}(r,x),u_{r})\,dr}$, $f^{u}(s,x)=f(\phi^{u}(s,x),u_{s})$, and

[TABLE]

It can be proved that $\mathcal{T}$ is a contracting map in $B([0,\,T]\times E)$ and $V$ is its unique fixed point, see e.g. Theorem 4.6 in [8] or Theorem 3.3 and Lemma 3.4 in [22]. In particular, $V$ satisfies the DPP (2.17).

Denote by $C_{b}([0,T]\times E)$ the set of bounded functions, continuous on $[0,\,T]\times E$ with the $|\cdot|\times||\cdot||_{-1}$ norm. In order to prove that $V\in C_{b}([0,T]\times E)$, it suffices to show that for any function $\psi\in C_{b}([0,T]\times E)$ one has $\mathcal{T}\psi\in C_{b}([0,T]\times E)$. As a matter of fact, we know that $\mathcal{T}$ is a contracting map in $B([0,\,T]\times E)$ and that $V$ is its unique fixed point, namely $V=\mathcal{T}V$. Assume now that, for any function $\psi\in C_{b}([0,T]\times E)$, one has $\mathcal{T}\psi\in C_{b}([0,T]\times E)$. Then $\mathcal{T}$ is a contracting map in $C_{b}([0,T]$ and has a unique fixed point in $C_{b}([0,T]$, that we denote $w^{\ast}$. We have

[TABLE]

so that $||w^{\ast}-V||_{\infty}=0$.

In the following $C$ will denote a generic constant, that may vary from line to line, and that may depend on $T$. We start by noticing that, by (6.1), $\mathcal{T}\psi(t,x)=\inf_{u\in\mathcal{U}}\bar{J}(t,x,u),$ with

[TABLE]

Let $t,t^{\prime},s\in[0,\,T]$, $t^{\prime}\leq t\leq s$, $x,x^{\prime}\in E$, $u\in\mathcal{U}$. Recalling hypotheses (Hb$\lambda$Q)-(i), (Hfg) and (LABEL:contrflowestimate), we have $|\chi^{u}(s,x)|\leq 1$, $|f^{u}(s,x)|\leq C$, and, for any $s^{\prime}\leq s$,

[TABLE]

On the other hand, by (Hb$\lambda$Q)-(i)-(ii), together with the boundedness and continuity of $\psi$, we have $|L_{\psi}^{u}(s,x)\leq C$ and, for $s<T-t$,

[TABLE]

where the latter inequality follows from (2.13). Then, for any $t,t^{\prime}\in[0,\,T]$, $x,x^{\prime}\in E$, $u\in\mathcal{U}$,

[TABLE]

for some modulus of continuity $\omega$, $\omega^{\prime}$, where the latter inequality follows from (6.2), (6.3), (6.4), (6.5). This shows in particular that $V$ is uniformly continuous in the $|\cdot|\times||\cdot||_{-1}$ norm.

6.2 Proof of Proposition 2.6

We first show that the left-hand side of (2.18) is smaller than the right-hand side. To this end, let us fix $\alpha\in\mathcal{A}_{ad}^{t}$. By (2.5), we have that, under $\mathbb{P}^{t,x}_{\alpha}$, $\theta=\tau_{d}\wedge T\wedge T_{1},$ with

[TABLE]

From the selection theorem (see e.g. Proposition 7.50 in [6]), for any $\varepsilon>0$, there exists a Borel-measurable map $\gamma^{\varepsilon}\in\mathcal{A}_{ad}^{\theta}$ of the form

[TABLE]

such that $\gamma^{\varepsilon}(\theta)$ is an $\varepsilon$-optimal control for $V(\theta,X_{\theta})$, namely

[TABLE]

Set

[TABLE]

and define (by a slight abuse of notation), for any real $\eta\geq\tau_{d}$ and $\chi\in E$,

[TABLE]

We have $\bar{\alpha}^{\varepsilon}_{s}=\alpha^{0}_{s}\mathonebb{1}_{s\in(t,\,\tau_{d}]}+\gamma_{s}^{\varepsilon}(\tau_{d},X_{\tau_{d}})\mathonebb{1}_{s\in(\tau_{d},\,T]}$, if $\theta=\tau_{d}$, and $\bar{\alpha}^{\varepsilon}_{s}=\alpha^{0}_{s}\mathonebb{1}_{s\in(t,\,T_{1}]}+\gamma_{s}^{\varepsilon}(T_{1},E_{1})\mathonebb{1}_{s\in(T_{1},\,T]}$ if $\theta=T\wedge T_{1}$. Therefore, $\bar{\alpha}^{\varepsilon}\in\mathcal{A}^{t}_{ad}$. In particular,

[TABLE]

At this point, we aim at proving that

[TABLE]

We notice that

[TABLE]

Moreover, setting for any $(\eta,\chi)\in[0,\,\infty)\times E$

[TABLE]

we see that

[TABLE]

Since

[TABLE]

it follows from (6.10) and (6.11) that (6.8) and (6.9) hold true if and only if, for any $n>1$,

[TABLE]

We show for simplicity the case $n=2$, the other cases are obtained analogously. Concerning (6.13), we have

[TABLE]

On the other hand, (6.14) with $n=2$ reads

[TABLE]

Thus (6.8) and (6.9) hold true, and (6.2) gives

[TABLE]

where we have also used the fact that $\mathbb{E}^{t,x}_{\bar{\alpha}^{\varepsilon}}\left[\varphi(\theta,X_{\theta})\right]=\mathbb{E}^{t,x}_{\alpha}\left[\varphi(\theta,X_{\theta})\right]$ for any measurable function $\varphi$, since $\bar{\alpha}^{\varepsilon}_{s}$ and $\alpha_{s}$ coincide on $(t,\theta]\subset(t,T_{1}]$. The result follows from the arbitrariness of $\varepsilon>0$ and $\alpha\in\mathcal{A}_{ad}^{t}$.

It remains to prove that the left-hand side of (2.18) is greater than the left-hand side. To this end, let $\alpha\in\mathcal{A}_{ad}^{t}$. By using the same argument as in the previous step, with $\gamma_{s}^{\varepsilon}(\eta,\chi)$ replaced by

[TABLE]

we have

[TABLE]

and the result follows by taking the infimum over $\alpha\in\mathcal{A}_{ad}^{t}$ in the left-hand side term.

6.3 Proof of Lemma 2.10

Let us fix $t\in(\varepsilon,\,T-\varepsilon)$, $\varepsilon>0$. We first prove that the map

[TABLE]

is continuous on $[t,\,T-\varepsilon)$, uniformly in $\alpha_{0}$, and on $B_{R}(x):=\{x\in E:||x||\leq R\}$, $R>0$. To this end, let $r,r^{\prime}\in[t,\,T-\varepsilon)$. Since $\psi$ satisfies Definition 2.8, in particular $\frac{\partial\psi}{\partial s}$, $L^{\ast}\,D\varphi$, $L^{\ast}\,D\delta$ are bounded on bounded sets of $E$. In the following $C$ will denote a generic constant that may depend on $T$ and $x$, and that may vary from line to line. For any $x\in E:||x||\leq R$, $R>0$, we have

[TABLE]

where in the latter inequality we have used (2.10). Using again the properties of the test functions, together with (2.10)-(2.11), we get

[TABLE]

Analogously,

[TABLE]

Moreover, for any $x\in E$, $a\in A$, and any measurable function $\alpha_{0}:\mathbb{R}_{+}\times E\rightarrow A$, the map

[TABLE]

is continuous on $[t,\,T-\varepsilon)$, uniformly in $\alpha_{0}$ and in $a$. Indeed, from (Hfg) and (2.13), for any $r\in[t,\,T-\varepsilon)$, $x\in E:||x||\leq R$, $R>0$,

[TABLE]

Let us finally study the continuity of the maps

[TABLE]

Since by Definition 2.8 $D\psi$ is bounded on bounded sets of $E$, and using assumption (Hb$\lambda$Q) for $b$, for any $x\in E:||x||\leq R$, $R>0$ we get

[TABLE]

On the other hand, by assumption (Hb$\lambda$Q) for $\lambda$ and $Q$, recalling that $\psi$ is uniformly continuous on $(\varepsilon,T-\varepsilon)\times E$, for any $x\in E:||x||\leq R$, $R>0$ we have

[TABLE]

where the latter inequality follows from (2.10)-(2.11)-(2.12).

6.4 Proof of Theorem 2.11

We start by giving the following preliminary result.

Lemma 6.1**.**

Assume that Hypotheses (HL) and (Hb$\lambda$Q) hold. Let $0<t<\bar{T}<T$, $\hat{\tau}$ be a stopping time such that $\hat{\tau}\in[t,\,\bar{T}]$, $x\in E$, $\alpha\in\mathcal{A}_{ad}^{t}$, and $X$ be the process in (2.6) under $\mathbb{P}^{t,x}_{\alpha}$. For $R>0$, let $\tau_{R}$ be the exit time of $X$ from $\{y:\,||y||\leq R\}$, and set $\tau=\hat{\tau}\wedge\tau_{R}$. Let $\psi=\varphi+h(||\cdot||)\,\delta$ be a test function. Then,

[TABLE]

Proof of Lemma 6.1. The result follows from the Dynkin formula (2.14) and the properties of the test functions $\psi$ in Definition 2.8. In particular, $D\psi(r,X_{r})=D\varphi(r,X_{r})+h(||X_{r}||)\,D\delta(r,X_{r})+\delta(r,X_{r})\,\frac{h^{\prime}(||X_{r}||)}{||X_{r}||}X_{r}$, and $\langle LX_{r},\delta(r,X_{r})\,\frac{h^{\prime}(||X_{r}||)}{||X_{r}||}X_{r}\rangle\geq 0$, being $L$ monotone.

Proof of Theorem 2.11.

Viscosity subsolution property. Let $\psi(s,y)=\varphi(s,y)+\delta(s,y)\,h(||y||)$ be a test function of the type introduced in Definition 2.8, such that $V-\psi$ has a global maximum at $(t,x)\in]0,\,T[\times E$. We also assume that

[TABLE]

and consequently

[TABLE]

Remember that $T_{1}$ denotes the first jump time of $X$. From the dynamic programming principle (2.18) applied to $\theta:=(t+\eta)\wedge T_{1}$ where $\eta>0$ is such that $(t+\eta)<T$ otherwise arbitrary for the moment, and since by (6.17), $V(s,y)\leq\psi(s,y)$ for all $(s,y)$, we have

[TABLE]

All elements of $\mathcal{A}_{ad}^{t}$ have the form (2.5). Let us fix $a\in A$, and let us take $\alpha\in\mathcal{A}_{ad}^{t}$ such that $\alpha_{0}\equiv a$. Notice that, $\mathbb{P}^{t,x}_{\alpha}$-a.s., $X_{r}=\phi^{a}(r-t,x)$ for $r\in[t,\,\theta)$. In particular, by (2.11),

[TABLE]

Denoting by $\tau_{R}$ the exit time of $X$ from $\{y:\,||y||\leq R\}$, it follows that $\theta=(t+h)\wedge T_{1}\wedge\tau_{R_{x}}$. As a matter of fact, if $(t+h)\leq T_{1}$, then $(t+h)\wedge T_{1}\wedge\tau_{R_{x}}=t+h$. On the other hand, if $(t+h)>T_{1}$, we have two cases: if $X_{T_{1}}\notin B_{R_{x}}$, then $(t+h)\wedge T_{1}\wedge\tau_{R_{x}}=\tau_{R_{x}}=T_{1}$, if $X_{T_{1}}\in B_{R_{x}}$, then $(t+h)\wedge T_{1}\wedge\tau_{R_{x}}=T_{1}$. Then (6.18) for such an $\alpha$ and Lemma 6.1 imply that

[TABLE]

with $X_{r}=\phi^{a}(r-t,x)$. Moreover, by Lemma 2.10, the (deterministic) map

[TABLE]

is continuous at $t$, uniformly in $a$. Therefore, for any $\epsilon>0$ there exists $\eta>0$ such that (6.4) with $\theta$ associated to $\eta$ becomes

[TABLE]

valid for any $a\in A$. Now we observe that $\mathbb{E}^{t,x}_{\alpha}\left[\theta-t\right]\geq 0$ by definition of $\theta$. Then (6.20) implies

[TABLE]

for any $\varepsilon>0$ and $a\in A$. The conclusion follows by the arbitrariness of $\varepsilon$ and $a$.

Viscosity supersolution property. Let $\psi(s,y)=\varphi(s,y)+\delta(s,y)\,h(||x||)$ be a test function of the type introduced in Definition 2.8, such that $V+\psi$ has a global minimum at $(t,x)\in]0,\,T[\times E$. We also assume that

[TABLE]

so we have

[TABLE]

We will show that $V$ is a viscosity supersolution by contradiction. Let us use the notations of Lemma 2.10. Assume that

[TABLE]

By Lemma 2.10, there exists $\eta>0$, independent from $\alpha_{0}$, such that

[TABLE]

Let us now set $\theta:=(t+\eta)\wedge T_{1}$ where $\eta$ satisfies $(t+\eta)<T$. By the dynamic programming principle (2.18), for all $\gamma>0$ there exists $\alpha\in\mathcal{A}_{ad}^{t}$ such that

[TABLE]

and therefore, recalling (6.22) and (6.23),

[TABLE]

As in the proof of the viscosity subsolution property, we set $R_{x}$ to the the bound in (2.11), and we notice that $\theta=(t+h)\wedge T_{1}\wedge\tau_{R_{x}}$, where $\tau_{R}$ denotes the exit time of $X$ from $\{y:\,||y||\leq R\}$. Applying Lemma 6.1 to $\psi$ between $t$ and $\theta$, we get

[TABLE]

Then

[TABLE]

By the definition of $\theta$, together with (2.5) and (2.6), for all $r\in[t,\theta)$, $\alpha_{r}=\alpha_{0}(r-t,x)$, with $\alpha_{0}$ as in (2.5) and $X_{r}=\phi^{\alpha_{0}}(r-t,x)$. Thus (6.25) yields

[TABLE]

Now we notice that

[TABLE]

where in the latter inequality we have used that by assumption $\lambda$ is bounded by some constant $M$. By letting $\gamma$ go to zero we obtain the contradiction.

7 Proofs of the results in Section 4

7.1 Proof of Theorem 4.3

The boundedness of $v$ follows from (4.9), (4.5), together with the definition of $V^{\ast}$ in (3.20) and the assumption (Hfg).

Let us now turn to the continuity properties. We argue as in the proof of Proposition 2.5. We denote by $B([0,\,T]\times E)$ the set of all bounded functions on $[0,\,T]\times E$, and, for any $a\in A$, we define the map $\mathcal{T}_{a}:B([0,\,T]\times E)\rightarrow B([0,\,T]\times E)$ as

[TABLE]

We have

[TABLE]

where $\chi^{\nu}(s,x,a)=e^{-\int_{0}^{s}\left(\lambda(\phi(r,x,a),a)+\int_{A}\nu_{s}(b)\lambda_{0}(db)\right)\,dr}$, $f(s,x,a)=f(\phi(s,x,a),a)$, and

[TABLE]

As in the case of the map in (6.1), for any $a\in A$, $\mathcal{T}_{a}$ is a contracting map in $B([0,\,T]\times E)$ and $v$ is its unique fixed point. In particular, $v$ satisfies the randomized DPP (4.11).

Then, we denote by $C_{b}([0,T]\times E)$ the set of bounded functions, continuous on $[0,\,T]\times E$ with the $|\cdot|\times||\cdot||_{-1}$ norm. We aim at showing that, for any function $\psi\in C_{b}([0,T]\times E)$, for any $a\in A$ one has $\mathcal{T}_{a}\psi\in C_{b}([0,T]\times E)$. This would prove that $v\in C_{b}([0,T]\times E)$.

In what follows $C$ will denote a generic constant, that may vary from line to line, and that may depend on $T$. Let $t,t^{\prime},s\in[0,\,T]$, $t^{\prime}\leq t\leq s$, $x,x^{\prime}\in E$, $a\in A$, $\nu\in\mathcal{V}$. Recalling hypotheses (Hb$\lambda$Q)-(i), (Hfg) and (3.2), we have $|\chi^{\nu}(s,x,a)|\leq 1$, $|f(s,x,a)|\leq C$, and, for any $s^{\prime}\leq s$,

[TABLE]

On the other hand, by (Hb$\lambda$Q)-(i)-(ii), together with the boundedness and continuity of $\psi$, we have $|L_{\psi}(s,x,a)\leq C$ and, for $s<T-t$,

[TABLE]

where the latter inequality follows from (3.2). Then, for any $t,t^{\prime}\in[0,\,T]$, $x,x^{\prime}\in E$, $a\in A$, $\nu\in\mathcal{V}$,

[TABLE]

for some modulus of continuity $\omega$, $\omega^{\prime}$, where the latter inequality follows from (7.1), (7.2), (7.3), (7.4). This shows in particular that $v$ is uniformly continuous in the $|\cdot|\times||\cdot||_{-1}$ norm.

7.2 Proof of Theorem 4.4

We first give the following preliminary result.

Lemma 7.1**.**

Let $0<t<\bar{T}<T$, $\hat{\tau}$ be a stopping time such that $\hat{\tau}\in[t,\,\bar{T}]$, $x\in E$, $a\in A$, $\nu\in\mathcal{V}$, and $(X,I)$ be the PDMP constructed in Section 3.1 under the probability $\mathbb{P}^{t,x,a}_{\nu}$. For $R>0$, let $\tau_{R}$ be the exit time of $X$ from $\{y:\,||y||\leq R\}$, and set $\tau=\hat{\tau}\wedge\tau_{R}$. Let $\psi=\varphi+h(||\cdot||)\,\delta$ be a test function. Then,

[TABLE]

Proof of Lemma 7.1. By Proposition 3.1, applying the Dynkin formula to $\psi(s,X_{s})$ between $t$ and $\tau$ and taking the expectation under $\mathbb{P}^{t,x,a}_{\nu}$, we get

[TABLE]

We conclude noticing that $D\psi(r,X_{r})=D\varphi(r,X_{r})+h(||X_{r}||)\,D\delta(r,X_{r})+\delta(r,X_{r})\,\frac{h^{\prime}(||X_{r}||)}{||X_{r}||}X_{r}$, and that $\langle LX_{r},\delta(r,X_{r})\,\frac{h^{\prime}(||X_{r}||)}{||X_{r}||}X_{r}\rangle\geq 0$, being $L$ is monotone.

Proof of Theorem 4.4.

Viscosity subsolution property. Let $\psi(s,y)=\varphi(s,y)+\delta(s,y)\,h(||x||)$ be a test function of the type introduced in Definition 2.8, such that $v-\psi$ has a global maximum at $(t,x)\in[0,\,T]\times E$. We also assume that

[TABLE]

so we have

[TABLE]

Fix $(t,x,a)$ and $\nu\in\mathcal{V}$. Let $\eta>0$ and define $\theta=(t+\eta)\wedge T_{1}$, where $T_{1}$ denotes the first jump time of $(X,I)$. Using the identification property (4.9), from the randomized dynamic programming principle (4.11), together with (7.7), we get

[TABLE]

Applying Lemma 7.1, we obtain

[TABLE]

where

[TABLE]

Now we notice that $\mathbb{P}^{t,x,a}$-a.s., for all $r\in(t,\theta)$, $(X_{r},I_{r})=(\phi(r-t,x,a),a)$. Moreover, by Lemma 2.10, the map

[TABLE]

is continuous, uniformly with respect to $a\in A$. We can proceed as in the proof of Theorem 2.11. By the latter continuity property, for any $\epsilon>0$, we can find $\eta>0$ independent of $a$ such that (7.2) holds true for $\theta$ corresponding to $\eta$. Since $\mathbb{E}^{t,x}_{\alpha}\left[\theta-t\right]\geq 0$ by definition of $\theta$, then identity (7.2) implies

[TABLE]

for any $\epsilon>0$ and $a\in A$. As in the proof of Theorem 2.11, we conclude by the arbitrariness of $\varepsilon$ and $a$.

Viscosity supersolution property. Let $\psi(s,y)=\varphi(s,y)+\delta(s,y)\,h(||x||)$ be a test function of the type introduced in Definition 2.8, such that $v+\psi$ has a global minimum at $(t,x)\in[0,\,T]\times E$. We also assume that

[TABLE]

so we have

[TABLE]

We will show that $v$ is a viscosity supersolution by contradiction. Let us use the notations of Lemma 2.10. Assume that

[TABLE]

By Lemma 2.10 that we apply for $\alpha_{0}\equiv a$, $a\in A$ arbitrary, there exists $\eta>0$, independent from $a$, such that

[TABLE]

Let us set $\theta=(t+\eta)\wedge T_{1}$ and fix $a\in A$. By the dynamic programming principle (4.11) toghether with the identification property (4.9), we see that, for all $\gamma>0$, it exists a strictly positive, predictable and bounded function $\nu$ such that

[TABLE]

Recalling (7.11) and (7.12), we get

[TABLE]

Applying Lemma 7.1, inequality (7.15) yields

[TABLE]

Noticing that

[TABLE]

with $\mathcal{L}^{I}$ is the operator in (7.9), previous inequality gives

[TABLE]

By the definition of $\theta$, together with (3.9), for all $r\in[t,\theta)$, $X_{r}=\phi(r-t,x,a)$. Thus, (7.2) together with (7.14) yields

[TABLE]

We conclude as in the proof of Theorem 2.11 using that

[TABLE]

where $M$ is an upper bound of $\lambda$. We obtain the contradiction by letting $\gamma$ go to zero.

7.3 Proof of the comparison Theorem 4.5

We begin recalling the following result concerning an equivalent definition of viscosity super and subsolution to (2.19)-(2.20).

Definition 7.2**.**

Let assumptions (HL), (Hb$\lambda$Q) and (Hfg) be satisfied. We will say that a function $\psi$ is a test function in the sense of Definition 7.2 if $\psi(s,y)=\varphi(s,y)+h(||y||)$, where $\varphi$, $h$ are as in Definition 2.8 without being bounded, however $\varphi$ is bounded on every set $(\varepsilon,T-\varepsilon)\times\{x\in E:||x||\leq R\}$, $\varepsilon\in(0,T)$, $R>0$.

• (i)

A bounded $B$-upper-semicontinuous function $u:(0,\,T]\times E\rightarrow\mathbb{R}$ is a viscosity subsolution in the sense of Definition 7.2 of (2.19)-(2.20) if $u(T,x)\leq g(x)$ on $E$, and, whenever $u-\psi$ has a global maximum at a point $(t,x)$ for a test function $\psi(s,y)=\varphi(s,y)+h(||y||)$, then

[TABLE]

• (ii)

A bounded $B$-lower-semicontinuous function $w:(0,\,T]\times E\rightarrow\mathbb{R}$ is a viscosity supersolution in the sense of Definition 7.2 of (2.19)-(2.20) if $w(T,x)\geq g(x)$ on $E$, and, whenever $w+\psi$ has a global minimum at a point $(t,x)$ for a test function $\psi(s,y)=\varphi(s,y)+h(||y||)$, then

[TABLE]

• (iii)

A viscosity solution of (2.19)-(2.20) in the sense of Definition 7.2 is a function which is both a viscosity subsolution and a viscosity supersolution.

Lemma 7.3**.**

Let assumptions (HL), (Hb$\lambda$Q) and (Hfg) be satisfied. If a function $u:(0,\,T)\times E\rightarrow\mathbb{R}$ (resp. $w:(0,\,T)\times E\rightarrow\mathbb{R}$) is bounded and uniformly continuous in the $|\cdot|\times||\cdot||_{-1}$ norm, and is a viscosity subsolution (resp. supersolution) of equation (2.19)-(2.20), then it is a viscosity subsolution (resp. supersolution) of equation (2.19)-(2.20) in the sense of Definition 7.2.

Proof of Lemma 7.3. This lemma extends to the infinite-dimensional framework a well known result in the finite-dimensional case, see e.g. Lemma 2.1 in [25].

We consider the subsolution case, the supersolution case can be proved analogously. Let thus $u:(0,\,T)\times E\rightarrow\mathbb{R}$ be bounded and uniformly continuous function in the $|\cdot|\times||\cdot||_{-1}$ norm, providing a viscosity subsolution to (2.19)-(2.20). Let $u-\psi$ has a global maximum at $(t,x)$ for a test function $\psi(s,y)=\varphi(s,y)+h(||y||)$, where without loss of generality we can assume that $\varphi$ and $h(||\cdot||)$ are bounded and that $u(t,x)=\psi(t,x)$. By assumption, it exists a modulus $\sigma_{u}$ such that

[TABLE]

For any $\varepsilon>0$, let $\bar{u}^{\varepsilon}$ be the sup-inf convolution of $u$ (see e.g. Definition D.24 in [13]), namely

[TABLE]

Then, according to Proposition D.26 in [13], $\bar{u}^{\varepsilon}$, $\frac{\partial\bar{u}^{\varepsilon}}{\partial_{t}}$, $D\bar{u}^{\varepsilon}$ are uniformly continuous in the $|\cdot|\times||\cdot||_{-1}$ norm and bounded, and for any $s\in[0,\,T]$, $y\in E$,

[TABLE]

where $\frac{t_{\varepsilon}}{\sqrt{\varepsilon}}\rightarrow 0$ as $\varepsilon\rightarrow 0$. This implies in particular that $\bar{u}^{\varepsilon}$, $\frac{\partial\bar{u}^{\varepsilon}}{\partial_{t}}$, $D\bar{u}^{\varepsilon}$ and $A^{\ast}D\bar{u}^{\varepsilon}$ are uniformly continuous with respect in the $|\cdot|\times||\cdot||$ norm.

Let $\eta$ be a smooth function, such that $\eta(\tau)=1$ for $\tau<1$, $\eta(\tau)=0$ for $\tau>2$, and which is strictly decreasing on $[1,2]$. We define

[TABLE]

By definition $u(t,x)-\psi^{\varepsilon}(t,x)=0$. Moreover

[TABLE]

For all $(s,y)\in[0,\,T]\times E$, $u(s,y)\leq\bar{u}^{\varepsilon}(s,y)$ by (7.20), and $u(s,y)-\psi(s,y)\leq 0$ by assumption.

It follows that $u-\psi^{\varepsilon}$ has a global maximum at $(t,x)$. Therefore, we apply Definition 2.9 with $\psi^{\varepsilon}(s,y)=\varphi^{\varepsilon}(s,y)+h(||y||)\,\delta^{\varepsilon}(s,y)$, where

[TABLE]

Notice that $\psi^{\varepsilon}(t,x)=\psi(t,x)=u(t,x)$, $\frac{\partial\psi^{\varepsilon}}{\partial t}(t,x)=\frac{\partial\psi}{\partial t}(t,x)$, $D\psi^{\varepsilon}(t,x)=D\psi(t,x)$. We get

[TABLE]

At this point we notice that

[TABLE]

where in the latter inequality we have used (7.21). The conclusion follows by the Lebesgue dominated convergence theorem.

Proof of Theorem 4.5.

We will show the result by contradiction. Assume therefore that $u\nleq v$.

Step 1. Set $u^{\eta}(t,x)=u(t,x)-\frac{\eta}{t}$, $v^{\eta}(s,y)=v(s,y)+\frac{\eta}{s}$, $\eta>0$, and, for $\varepsilon,\delta,\beta>0$, define the function

[TABLE]

By perturbed optimization (see, e.g. [10], page 430) there exist sequences $a_{n},b_{n}\in\mathbb{R}$, $p_{n},q_{n}\in E$ such that

[TABLE]

and

[TABLE]

attains a strict maximum at some point $(\bar{t},\bar{s},\bar{x},\bar{y})\in(0,\,T]\times(0,\,T]\times E\times E$. Standard considerations yield (see e.g. [13], page 209)

[TABLE]

Then, recalling that by assumption $u\nleq v$, it follows from (7.23)-(7.24)-(7.25) and the uniform continuity of $u$, $v$, that for sufficiently small $\varepsilon,\eta,\delta,\beta>0$ and $n$ large enough, $\bar{t},\bar{s}<T$.

*Step 2. * From Step 1 we deduce that

[TABLE]

where $h(||z||):=\delta||z||^{2}$, and

[TABLE]

In particular, $\nabla h(||z||)=2\delta z$, and

[TABLE]

Step 3. Viscosity inequalities. By Lemma 7.3, $u$ is a viscosity subsolution of equation (2.19)-(2.20) in the sense of Definition 7.2. Therefore, using Step 2, we have

[TABLE]

Similarly, being $v$ is a viscosity supersolution of equation (2.19)-(2.20) in the sense of Definition 7.2 by Lemma 7.3, proceeding as before one gets

[TABLE]

Subtracting (7.3) from (7.3) we obtain

[TABLE]

where we have used that $\inf A_{1}-\inf A_{2}\leq\sup(A_{1}-A_{2})$). Using condition (2.1), together with the assumptions on the functions $b$ and $f$, (7.28) yields

[TABLE]

At this point, by Hypothesis (Hb$\lambda$Q)-(i)-(ii), we get

[TABLE]

Therefore it remains to prove that

[TABLE]

converges to [math] when the parameters go to their respective limits.

Step 4. Proof of the convergence of (7.30) to [math]. Set $m:=2(||u||_{\infty}\vee||v||_{\infty})$ and

[TABLE]

By Step 1, we know that $M$ is a strict maximum on $(0,\,T]\times(0,\,T]\times E\times E$ of the function

[TABLE]

The definition of $\Phi^{\varepsilon,\delta,\beta}$ implies that

[TABLE]

which in turn implies that

[TABLE]

Moreover $u(\bar{t},\bar{x})-v(\bar{s},\bar{y})\leq m$ and $a_{n}\bar{t}+b_{n}\bar{s}\leq T$ for all $n\geq 2$, since $|a_{n}|+|b_{n}|\leq\frac{1}{2}$ for $n\geq 2$. Therefore

[TABLE]

Let us take $K\in\mathbb{N}$ satisfying

[TABLE]

and define the set

[TABLE]

Notice that from (7.32) we have $(\bar{x},\bar{y})\in\Gamma_{1,d}$. Let also $\alpha>0$ be such that

[TABLE]

and $D$ be a smooth function on $E\times E$ satisfying

[TABLE]

Then the function

[TABLE]

admits a strict maximum at $(\bar{t},\bar{s},\bar{x},\bar{y})$. Indeed, if $(x,y)\in\Gamma_{1,d}$ the expression (7.38) coincides with $\Phi^{\varepsilon,\delta,\beta}(t,s,x,y)+a_{n}t+b_{n}s+\langle Bp_{n},x\rangle+\langle Bq_{n},y\rangle$, and if $(x,y)\notin\Gamma_{1,d}$, by the definition of $D(x,y)$ the expression (7.38) is smaller or equal to

[TABLE]

the latter being strictly smaller than $M$ by the choice of $\alpha$ (cf. (7.36)). Using Step 1 with $x=y$ we obtain that, for all $y\in E$,

[TABLE]

which implies

[TABLE]

Let us set

[TABLE]

For any $y\in{\Sigma_{1}}$ we obtain by (7.39) that

[TABLE]

Let us now set (since we are interested in $\delta\in(0,1)$ we have $\frac{2KM}{\sqrt{\delta}}<\frac{2KM}{\delta}$)

[TABLE]

Inequality (7.39) yields

[TABLE]

Finally set

[TABLE]

Let us now take $y\in\Sigma_{3}$. Then $(y,y)$ belongs to $\Gamma_{1,d}^{c}$. From the previous arguments

[TABLE]

which together with (7.3) implies

[TABLE]

Thus we obtain

[TABLE]

At this point, let us go back to (7.30). Using the partitioning $E=\Sigma_{1}\cup\Sigma_{2}\cup\Sigma_{3}$, in (7.30) we split the integral on $E$ in the integrals over the sets $\Sigma_{i}$. From (7.40) together with (Hb$\lambda$Q), we get

[TABLE]

which obviously converges to zero. On the other hand, by (7.41) and (7.42), we obtain

[TABLE]

We have chosen the parameters according to (7.22). Then in particular $\frac{||Bp_{n}||}{\delta}\leq\frac{1}{n\delta}$ converges to [math]. This completes the proof recalling assumption (HQ’) (see Section 4) and the respective definitions of $\Sigma_{2},\Sigma_{3}$.

Acknowledgements.

The first author would like to thank Prof. Fausto Gozzi for his helpful discussions and valuable suggestions to improve this paper. The first author has been financed by ”Progetto di Ricerca GNAMPA - INdAM 2018”, and partially benefited from the support of the Italian MIUR-PRIN 2015-16 ”Deterministic and stochastic evolution equations”. The financial support of the Laboratoire de Probabilités, Statistique et Modélisation (LPSM, UMR 8001) of Sorbonne Université is also greatly acknowledged.