Pathwise Stochastic Control with Applications to Robust Filtering

TL;DR

This paper introduces a pathwise stochastic control framework using rough differential equations to develop robust filters that handle parameter uncertainty, combining stochastic control and rough path theory.

Contribution

It proposes a novel approach to pathwise stochastic control via rough differential equations and applies it to create robust filtering methods resilient to parameter uncertainty.

Findings

Resolved degeneracy in control of noise coefficients.

Developed robust filters using pathwise control and rough path theory.

Demonstrated effectiveness in stochastic filtering applications.

Abstract

We study the problem of pathwise stochastic optimal control, where the optimization is performed for each fixed realisation of the driving noise, by phrasing the problem in terms of the optimal control of rough differential equations. We investigate the degeneracy phenomenon induced by directly controlling the coefficient of the noise term, and propose a simple procedure to resolve this degeneracy whilst retaining dynamic programming. As an application, we use pathwise stochastic control in the context of stochastic filtering to construct filters which are robust to parameter uncertainty, demonstrating an original application of rough path theory to statistics.

Equations

$$\mathrm{d}{X}_s=b(X_s,{\gamma }_s)\mathrm{d}s+\sigma (X_s,{\gamma }_s)\mathrm{d}{W}_s,s\in [t,T],$$

$$J(t,x;\gamma)=\mathbb{E}\bigg{[}\int_{t}^{T}f(X_{s},\gamma_{s})\,\mathrm{d}s+g(X_{T})\bigg{]}$$

$$-\mathrm{d}v-\inf_{\gamma}\big{\{}b\cdot\nabla v+f\big{\}}\,\mathrm{d}t-\sigma\cdot\nabla v\,\mathrm{d}W=0,$$

$$\mathrm{d}{X}_s=b(X_s,{\gamma }_s)\mathrm{d}s+\lambda (X_s)\mathrm{d}{\mathbf{\zeta }}_s,$$

$$\mathrm{d}{X}_s^{t,x,\gamma }={\gamma }_s(\sigma \mathrm{d}{W}_s+\mathrm{d}{\zeta }_s),s\in [t,T],$$

$$v(t,x)=\sup_{\gamma}\mathbb{E}\big{[}X^{t,x,\gamma}_{T}\big{]},$$

$$\mathrm{d}{X}_s^{t,x,\gamma ,\eta }={\gamma }_s(\sigma \mathrm{d}{W}_s+{\dot{\eta }}_s\mathrm{d}s),$$

$$\frac{\partial v^{\eta}}{\partial t}+\sup_{\gamma\in[-\varepsilon,\varepsilon]}\bigg{\{}\frac{1}{2}\gamma^{2}\sigma^{2}\frac{\partial^{2}v^{\eta}}{\partial x^{2}}+\gamma\dot{\eta}_{t}\frac{\partial v^{\eta}}{\partial x}\bigg{\}}=0,$$

$$v^{\eta }(t,x)=x+\epsilon {\int }_t^T|{\dot{\eta }}_s|\mathrm{d}s,$$

$${\gamma }_t^{\ast }=\epsilon \mathrm{sgn}({\dot{\eta }}_t).$$

$$\mathrm{d}{X}_s=b(X_s,{\gamma }_s)\mathrm{d}s+\lambda (X_s,{\gamma }_s)\mathrm{d}{\zeta }_s,s\in [0,T],$$

$$\|\gamma\|_{p}:=\bigg{(}\sup_{\mathcal{P}}\sum_{[s,t]\in\mathcal{P}}|\gamma_{s,t}|^{p}\bigg{)}^{\frac{1}{p}}<\infty,$$

$${\zeta }_{s,t}^{(2)}={\zeta }_{s,r}^{(2)}+{\zeta }_{r,t}^{(2)}+{\zeta }_{s,r}\otimes {\zeta }_{r,t},$$

$$\displaystyle{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{\frac{1}{p}\textrm{{-H\"{o}l}}}:=\|\zeta\|_{\frac{1}{p}\textrm{{-H\"{o}l}}}+\big{\|}\zeta^{(2)}\big{\|}_{\frac{2}{p}\textrm{{-H\"{o}l}}}<\infty,$$

$$\displaystyle\text{where}\qquad\|\zeta\|_{\frac{1}{p}\textrm{{-H\"{o}l}}}:=\sup_{s\neq t\in[0,T]}\frac{|\zeta_{s,t}|}{|t-s|^{\frac{1}{p}}}\qquad\text{and}\qquad\big{\|}\zeta^{(2)}\big{\|}_{\frac{2}{p}\textrm{{-H\"{o}l}}}:=\sup_{s\neq t\in[0,T]}\frac{\big{|}\zeta^{(2)}_{s,t}\big{|}}{|t-s|^{\frac{2}{p}}}.$$

$$\displaystyle\big{\|}\zeta^{(2)}\big{\|}_{\frac{p}{2}}$$

$${\left|\left|\left|\mathbf{\zeta }\right|\right|\right|}_p$$

$${\varrho }_{\frac{1}{p}\text{-Ho¨l}}(\mathbf{\eta },\mathbf{\zeta })$$

$${\varrho }_p(\mathbf{\eta },\mathbf{\zeta })$$

$${\zeta }_{s,t}^{(2)}={\int }_s^t{\zeta }_{s,r}\otimes \mathrm{d}{\zeta }_r.$$

$$(X,X^{\prime })\in {\mathcal{C}}^{p\text{-var}}([0,T];{\mathbb{R}}^m)\times {\mathcal{C}}^{p\text{-var}}([0,T];\mathcal{L}({\mathbb{R}}^d;{\mathbb{R}}^m))$$

$$R_{s,t}^X:=X_{s,t}-X_s^{\prime }{\zeta }_{s,t},$$

$${\int }_0^T{X}_r\mathrm{d}{\mathbf{\zeta }}_r:=\underset{|\mathcal{P}|\to 0}{\lim }\sum _{[s,t]\in \mathcal{P}}{X}_s{\zeta }_{s,t}+X_s^{\prime }{\zeta }_{s,t}^{(2)}$$

$$\displaystyle\bigg{|}\int_{s}^{t}X_{r}\,\mathrm{d}\bm{\zeta}_{r}$$

$$\displaystyle\leq C_{p}\Big{(}\big{\|}R^{X}\big{\|}_{\frac{p}{2};[s,t]}\|\zeta\|_{p;[s,t]}+\|X^{\prime}\|_{p;[s,t]}\big{\|}\zeta^{(2)}\big{\|}_{\frac{p}{2};[s,t]}\Big{)},$$

$$\mathrm{d}{X}_s=b(X_s,{\gamma }_s)\mathrm{d}s+\lambda (X_s,{\gamma }_s)\mathrm{d}{\mathbf{\zeta }}_s,s\in [0,T],$$

$$\lambda (X,\gamma {)}^{\prime }=D_x\lambda (X,\gamma )X^{\prime },$$

$$\|X\|_{p;[0,T]}\leq n\bigg{(}\sum_{i=1}^{n}\|X\|_{p;[t_{i-1},t_{i}]}^{p}\bigg{)}^{\frac{1}{p}}.$$

$$s_{j-1}=t_0^j<t_1^j<\dots <t_{n_j}^j=s_j$$

$$\text{or}{t}_{i-1}=s_0^i<s_1^i<\dots <s_{N_i}^i=t_i$$

$$\displaystyle\sum_{j=1}^{N}\big{|}X_{s_{j}}-X_{s_{j-1}}\big{|}^{p}$$

$$\displaystyle=n^{p}\sum_{i=1}^{n}\sum_{j=1}^{N_{i}}\big{|}X_{s^{i}_{j}}-X_{s^{i}_{j-1}}\big{|}^{p}\leq n^{p}\sum_{i=1}^{n}\|X\|_{p;[t_{i-1},t_{i}]}^{p}.$$

$$\displaystyle\big{|}R^{X}_{s,t}\big{|}$$

Full text

Keywords: stochastic control, rough paths, rough HJB equation, stochastic filtering, parameter uncertainty.

MSC 2010: 60H99, 93E20, 93E11.

1 Introduction

Stochastic optimal control is a classical optimization problem with numerous applications, from optimal liquidation and portfolio selection in mathematical finance to various problems in production planning, engineering and biology. Here one typically has a stochastic differential equation (SDE) of the form

[TABLE]

with an initial condition $X_{t}=x$, where $W$ is a Brownian motion and $\gamma$ is an adapted control process, and the goal is to minimize (resp. maximize) a cost (resp. reward) functional of the form

[TABLE]

over all possible choices of the control $\gamma$. The resolution of this problem is by now well understood—two primary approaches being that of the Pontryagin stochastic maximum principle and Bellman’s principle of optimality (or dynamic programming principle), which allows one to characterise the value function of the control problem, defined by $v(t,x)=\inf_{\gamma}J(t,x;\gamma)$, as the unique solution of a Hamilton–Jacobi–Bellman (HJB) partial differential equation (PDE).

In their 1998 paper [30], Lions and Souganidis considered a variant of this problem, known as ‘pathwise stochastic control’, where the optimization is performed pathwise. In other words, one considers controlling the solution of an equation of the form (1.1) for each individual realisation of the Brownian motion $W$. Moreover, they suggest that in this case the value function should satisty a ‘stochastic HJB equation’. Indeed, at least in the case when $\sigma$ does not depend on the control $\gamma$, if we pretend for the moment that the paths of $W$ were smooth, then, at least formally, the classical theory leads one to derive a stochastic PDE of the form

[TABLE]

with $v(T,\cdot)=g$.

The notion of pathwise stochastic control actually goes back at least as far as the work of Davis and Burstein [16, 17], who note that pathwise control is actually equivalent to the classical stochastic control setting if one allows for anticipative controls, leading to the conclusion that the difference between classical and pathwise control boils down to nonanticipativity of the controls. In this view, pathwise control can be thought of as performing optimal control with the benefit of complete knowledge of both the past and future realisations of the stochastic noise. On the other hand, as shown for instance by Rogers [33], pathwise control can also be used to obtain duality results for classical (nonanticipative) stochastic control, thus providing an alternative approach for numerical computations.

Since pathwise control entails the optimization of a stochastic system path by path, it is natural to fix such an (arbitrary) path and proceed to analyse the resulting deterministic problem. This invites a pathwise interpretation of the stochastic integral appearing in the controlled dynamics. The strategy followed by Buckdahn and Ma [8] circumnavigates such a technical requirement, by instead employing a Doss–Sussmann-style transformation to convert the problem into a more standard setting of ‘wider-sense control problem’, allowing them to establish their value function as the unique stochastic viscosity solution of the associated HJB equation. A more direct approach, avoiding such an ad hoc change of variables, requires one to utilize a pathwise approach to stochastic integration.

One such deterministic approach to integration against paths of low regularity is provided by rough path theory, introduced by Lyons [31]. The basic idea here is that the notion of integration can be extended in a consistent way to paths of lower regularity such that strong stability results concerning continuity of the integration map with respect to the driving ‘rough path’ hold, but one requires extra information about the driving signal than is expressed in the path alone. Such paths, $\zeta$ say, must therefore be ‘enhanced’ by a suitable ‘second order’ process $\zeta^{(2)}$ which captures this missing information. The addition of the process $\zeta^{(2)}$ is equivalent to considering the Lévy area of the path $\zeta$, and corresponds to the addition of the iterated integral $\int_{0}^{\cdot}\int_{0}^{r}\mathrm{d}\zeta_{s}\otimes\mathrm{d}\zeta_{r}$, but since this integral does not exist in the classical sense, its value must be postulated, rather than being uniquely determined by the original path $\zeta$.

We note in particular the more recent work of Diehl, Friz and Gassiat [18], which appears to be the first attempt to apply rough path theory to optimal control, in which the authors consider controlled dynamics of the form

[TABLE]

driven by a geometric rough path $\bm{\zeta}$. They proceed to both obtain a version of Pontryagin’s maximum principle and establish their value function as the unique solution of a ‘rough HJB equation’, and moreover obtain a duality result for the corresponding nonanticipative stochastic control problem. Their results suggest that rough path theory is an ideal tool for the study of pathwise stochastic control—a notion that we will echo in the present work.

Notably however, the existing literature on pathwise control invariably focuses only on the case where the control process appears in the drift term, but does not appear in the coefficient of the noise term (or rough path). That is, the controlled dynamics considered are typically of the general form in (1.3), where $\lambda$ is not allowed to depend on the control $\gamma$. As observed in Diehl et al. [18], the pathwise control problem with full dynamic control (particularly control in the coefficient of the noise term), when stated in the obvious way, turns out to be degenerate, which explains the lack of results in this direction. An indication of this arises when one reruns the formal derivation that led to (1.2) in the case where $\lambda$ depends on $\gamma$. In this case the resulting equation exhibits the Brownian motion $W$ inside the infimum. At least heuristically, this corresponds to the ability to perfectly optimize over the path of the noise term, which, as we will see, is the source of the degeneracy.

Example 1.1** (Insider trading).**

Let us suppose that an agent is trading a stock with price composed of a diffusive term with volatility $\sigma$, representing usual market uncertainty, and a deterministic path $\zeta$ which represents some additional information known only to the agent. We suppose that $\zeta\colon[0,T]\to\mathbb{R}$ is a continuous path which has infinite variation on any interval. Denoting the size of the agent’s investment by $\gamma$ and their wealth process by $X$, we have the controlled dynamics111Let us suppose for the moment that we have employed a suitable notion of integration such that the integral against $\zeta$ is well-posed.

[TABLE]

with $X^{t,x,\gamma}_{t}=x$, where $W$ is a standard Brownian motion and $x$ is the agent’s initial wealth. We assume that at each time the agent can only hold a finite amount of stock. More precisely, we impose that $\gamma$ takes values in the finite interval $[-\varepsilon,\varepsilon]$, for some $\varepsilon>0$. The agent’s expected terminal wealth is given by the value function

[TABLE]

where the supremum is taken over the collection of progressively measurable $[-\varepsilon,\varepsilon]$-valued processes. In order to study the nature of this problem, let us approximate $\zeta$ by a smooth function $\eta$. The dynamics (1.4) are then approximated by

[TABLE]

where $\dot{\eta}$ denotes the derivative of $\eta$. The resulting control problem is classical. The associated HJB equation is given by

[TABLE]

with terminal condition $v^{\eta}(T,x)=x$. The solution $v^{\eta}$ is seen to be

[TABLE]

and we infer that the optimal control $\gamma^{\ast}$ is given by

[TABLE]

Suppose now that we were to repeatedly refine the approximation $\eta$ so that it better captures the fast fluctuations of $\zeta$. Even without giving a precise definition of what we might mean by the limit as $\eta\to\zeta$, it is clear that the solution (1.6) should diverge to infinity in this limit. In other words, the original value function, as defined in (1.5), should be simply given by $v(t,x)=\infty$ whenever $t<T$. Thus, we infer that our original control problem, with the infinite variation signal $\zeta$, is degenerate.

The phenomenon exhibited here is typical for such control problems, where we attempt to control the coefficient of the infinite variation term in the controlled dynamics. The problem in the previous example is that, in contrast to a classical stochastic setting, since the controller can ‘see’ the path $\zeta$ in advance, they can choose controls $\gamma$ with very small, but extremely quick, fluctuations, which allow the solution $X$ to take full advantage of the infinite variation of $\zeta$. Indeed, notice that the sign of the optimal control in (1.7) changes at the same rate as that of $\dot{\eta}$, which varies ‘infinitely quickly’ in the limit as $\eta\to\zeta$.

To paraphrase Diehl et al. [18], if the coefficient of the driving signal has enough dependence on the control, and this signal has unbounded variation on any interval, then the controller can drive the solution to reach any point instantly whilst incurring an arbitrarily low cost.

In the current work we investigate this degeneracy phenomenon in more detail, and see how it may be resolved by introducing an artificial cost to penalise the variation of the controls. We will see how this cost may be chosen to ensure that a dynamic programming principle is retained, thus allowing one to recover a setting comparable to that of [18]. As an extension of that paper, we proceed to consider pathwise control with unbounded cost functions and, by obtaining locally uniform bounds on the controls, establish the value function as the unique solution of a rough HJB equation.

Stochastic filtering concerns the problem of estimating the current state of a hidden process from noisy observations, and itself has widespread and important applications, from finance and biology to engineering, defence and aerospace. In their paper [15], Crisan, Diehl, Friz and Oberhauser used rough path theory to resolve an existing open problem in the theory of ‘robust’ stochastic filtering, by establishing continuity of a large class of stochastic filters with respect to the observation path, by first enhancing it by its Lévy area. The second contribution of the current work is to consider an application of pathwise control to another kind of ‘robust’ filtering, namely robustness of the filter with respect to model uncertainty.

Although classical filters are generally known to perform well under perfect knowledge of the system dynamics, they are typically very sensitive to modelling errors. Thus, the problem of robust filtering, in this sense, has attracted a great deal of interest; see the discussion at the beginning of Section 4.2. In [2] the authors constructed such a robust filter, the calculation of which involves the derivation of a pathwise stochastic control problem. In that setting the control terms did not appear (in any crucial way) in the coefficient of the driving noise in the controlled dynamics. Similarly to the approach in Buckdahn and Ma [8], a change of variables could therefore be used to ‘hide’ the rough noise term in the drift coefficient, thus recovering a more classical optimal control setting. In the current work we aim to significantly extend the theoretical results of [2], which will require us to be able to handle full control of the dynamics. As in Crisan et al. [15], it will be useful to consider the observation process as a rough path, by first enhancing it by its Lévy area.

It is our hope that the following exposition will be of interest to readers familiar with rough path analysis, but also accessible to those without a working knowledge of the subject. Accordingly, we begin Section 2 with a brief recall of the necessary technical preliminaries, and then present some new results for rough differential equations in the setting of optimal control. In Section 3 we discuss some alternative reformulations of the pathwise control problem with the aim to resolve the degeneracy issue. We provide a rigorous treatment of the resulting unbounded control problem, and illustrate the ideas with some simple examples. In Section 4 we turn our attention to robust stochastic filtering. Our approach leads naturally to a pathwise optimal control problem and, despite the nonlinearities inherited from the classical filtering equations, we will proceed to characterise the associated value function as the solution of a rough HJB equation.

2 Rough path preliminaries

We would like to consider an $\mathbb{R}^{m}$-valued process $X$ which, for each choice of control $\gamma\colon[0,T]\to\mathbb{R}^{k}$, satisfies an equation of the form

[TABLE]

where $\zeta$ is a continuous (deterministic) $\mathbb{R}^{d}$-valued path of infinite variation.

Suppose that $b$ and $\lambda$ are Lipschitz continuous. In the case when $\gamma$ is of finite variation and $\lambda$ does not depend on the solution $X$, the integral against $\zeta$ then exists in the classical Riemann–Stieltjes sense by integration by parts (see e.g. Theorem 1.2.3 in Stroock [36]). The equation (2.1) then has a unique solution, and moreover the solution map from the driver $\zeta$ to the corresponding solution $X^{\zeta}$ is continuous with respect to the supremum norm. In fact, in this case all the results of the next section can be reproduced without any reference to rough path theory, or any other such sophisticated machinery.

On the other hand, in the general case when $\lambda$ depends on $X$, the integral against $\zeta$ in (2.1) does not even exist in the Riemann–Stieltjes sense. Moreover, even if $\zeta$ were smooth, the solution map $\zeta\mapsto X^{\zeta}$ is known to lack continuity, which later would be fatal to the derivation of our HJB equation.

A deterministic approach to integration against very general classes of signals is provided by rough path theory, which moreover allows the continuity property mentioned above to be recovered. As mentioned in the introduction, the key here is, rather than to simply integrate against the path $\zeta$, to first enhance $\zeta$ by a suitable ‘second order’ process $\zeta^{(2)}$, which contains the missing information required to construct the so-called ‘rough integral’ against the enhanced path $\bm{\zeta}:=(\zeta,\zeta^{(2)})$. There are by now a number of monographs on this subject, such as Friz and Hairer [22] and Friz and Victoir [24].

The language of rough path theory is typically written either in terms of the $\frac{1}{p}$-Hölder regularity of paths, or in terms of their $p$-variation. When working only with continuous paths (as we shall), these two notions of regularity are more or less equivalent (see Chapter 5 in [24] for precise details), and the theory may be built up in an almost identical fashion using either notion. In the current work it will turn out to be necessary to work primarily with $p$-variation norms. On the other hand, in the proof of Proposition 2.4 below we will make use of the marginally better control on the regularity of paths over small time intervals provided by restricting to $\frac{1}{p}$-Hölder rough paths. We shall therefore make use of both these notions of regularity.

2.1 Notation

Throughout, we will consider a finite time interval $[0,T]$, and write $\Delta_{[0,T]}:=\{(s,t):0\leq s\leq t\leq T\}$ for the standard 2-simplex. For any path $\zeta$ on $[0,T]$ we define the path increment $\zeta_{s,t}:=\zeta_{t}-\zeta_{s}$, and write $\|\zeta\|_{\infty}:=\sup_{s\in[0,T]}|\zeta_{s}|$ for the supremum norm. We will also make use of the following function spaces. We write

• •

$\mathcal{L}(\mathbb{R}^{d};\mathbb{R}^{m})$ for the space of linear maps from $\mathbb{R}^{d}$ to $\mathbb{R}^{m}$,

• •

$\textrm{{Lip}}_{b}$ for the space of bounded Lipschitz functions $b\colon\mathbb{R}^{m}\times\mathbb{R}^{k}\to\mathbb{R}^{m}$,

• •

$C^{n}_{b}$ ($n\in\mathbb{N}$) for the space of $n$ times continuously differentiable (in the Fréchet sense) functions $\lambda\colon\mathbb{R}^{m}\times\mathbb{R}^{k}\to\mathcal{L}(\mathbb{R}^{d};\mathbb{R}^{m})$ such that $\lambda$ and all its derivatives up to order $n$ are uniformly bounded,

• •

$\mathcal{C}^{p\textrm{{-var}}}=\mathcal{C}^{p\textrm{{-var}}}([0,T];\mathbb{R}^{k})$ for the space of $\mathbb{R}^{k}$-valued continuous paths of finite $p$-variation, that is, continuous paths $\gamma$ such that the seminorm

[TABLE]

where the supremum is taken over all partitions $\mathcal{P}$ of the interval $[0,T]$,

• •

$\mathcal{C}^{0\textrm{{,}}p\textrm{{-var}}}=\mathcal{C}^{0\textrm{{,}}p\textrm{{-var}}}([0,T];\mathbb{R}^{k})$ for the closure of smooth paths from $[0,T]\to\mathbb{R}^{k}$ with respect to the $p$-variation seminorm.

For $p\in[2,3)$ we write $\mathscr{C}^{p}=\mathscr{C}^{p}([0,T];\mathbb{R}^{d})$ for the space of $\mathbb{R}^{d}$-valued $\frac{1}{p}$-Hölder rough paths, that is, pairs $\bm{\zeta}=(\zeta,\zeta^{(2)})$, where the path $\zeta\colon[0,T]\to\mathbb{R}^{d}$ and its ‘enhancement’ $\zeta^{(2)}\colon\Delta_{[0,T]}\to\mathbb{R}^{d}\otimes\mathbb{R}^{d}$ satisfy certain algebraic and analytical constraints, namely Chen’s relation222Here $\otimes$ is just the standard tensor product from $\mathbb{R}^{d}\times\mathbb{R}^{d}$ to $\mathbb{R}^{d}\otimes\mathbb{R}^{d}\simeq\mathbb{R}^{d\times d}$.,

[TABLE]

which is assumed to hold for all times $s\leq r\leq t$, as well as the condition that

[TABLE]

The enhanced path $\bm{\zeta}$ is sometimes referred to as the ‘lift’ of $\zeta$. We also define

[TABLE]

We will sometimes write e.g. $\|\zeta\|_{p;[s,t]}$ for the $p$-variation of $\zeta$ over the subinterval $[s,t]$.

As we are working on the time interval $[0,T]$, it is straightforward to see that any rough path $\bm{\zeta}=(\zeta,\zeta^{(2)})\in\mathscr{C}^{p}$ satisfies $\|\zeta\|_{p}\leq\|\zeta\|_{\frac{1}{p}\textrm{{-H\"{o}l}}}T^{\frac{1}{p}}$ and $\|\zeta^{(2)}\|_{\frac{p}{2}}\leq\|\zeta^{(2)}\|_{\frac{2}{p}\textrm{{-H\"{o}l}}}T^{\frac{2}{p}}$, which in particular implies that ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{p}<\infty$ for any $\bm{\zeta}\in\mathscr{C}^{p}$.

We introduce the induced rough path metrics333The ‘metrics’ $\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}$, $\varrho_{p}$ do not distinguish between constants, but $\mathscr{C}^{p}$ does become a complete metric space when endowed with the metric $(\bm{\eta},\bm{\zeta})\mapsto|\eta_{0}-\zeta_{0}|+\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}(\bm{\eta},\bm{\zeta})$. given, for rough paths $\bm{\eta}=(\eta,\eta^{(2)})$ and $\bm{\zeta}=(\zeta,\zeta^{(2)})$, by

[TABLE]

As can be readily checked, any smooth path $\zeta\colon[0,T]\to\mathbb{R}^{d}$ can be ‘lifted’ in a canonical way to a rough path $\bm{\zeta}=(\zeta,\zeta^{(2)})$ by enhancing it with the integral

[TABLE]

On the other hand, for a general $\frac{1}{p}$-Hölder continuous path $\zeta$, the integral in (2.2) does not exist in the classical sense. In this case the value of this integral is postulated by the enhancement $\zeta^{(2)}$, which in practice is often constructed using stochastic integration.

Later we will also consider the space of geometric rough paths $\mathscr{C}_{g}^{0,p}\subset\mathscr{C}^{p}$, defined as the closure of canonical lifts of smooth paths with respect to $\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}$. For example, when $\zeta$ is a semimartingale and the integral in (2.2) is defined using Stratonovich integration, the resulting lift turns out to be a (random) geometric rough path. This property of being well approximated by smooth paths allows one to make sense of solutions to a wide class of rough ODEs and PDEs—we will see an example of this in Definition 3.13 below.

2.2 Rough integration

We now define a suitable class of integrands for rough integration. Given a rough path $\bm{\zeta}\in\mathscr{C}^{p}$, we define the space of controlled rough paths (in the sense of Gubinelli [26]), which we denote by $\mathscr{D}_{\zeta}^{p}=\mathscr{D}_{\zeta}^{p}([0,T];\mathbb{R}^{m})$, consisting of pairs of paths

[TABLE]

such that the remainder term $R^{X}$, given by

[TABLE]

satisfies $\|R^{X}\|_{\frac{p}{2}}<\infty$. Here $X^{\prime}$ is called the Gubinelli derivative of $X$ (with respect to $\zeta$). Equipped with the norm $(X,X^{\prime})\mapsto|X_{0}|+|X^{\prime}_{0}|+\|X^{\prime}\|_{p}+\|R^{X}\|_{\frac{p}{2}}$, the space $\mathscr{D}_{\zeta}^{p}$ is a Banach space.

Remark 2.1*.*

As our main interest is in the optimal control of the solution $X$ to (2.1), the notion that $X$ is ‘controlled’ by $\zeta$ introduces a possible source of confusion, but our use of the term should always be clear from the context.

Proposition 2.2** (Proposition 2.6 in [25]).**

Let $\bm{\zeta}=(\zeta,\zeta^{(2)})\in\mathscr{C}^{p}([0,T];\mathbb{R}^{d})$, and let $(X,X^{\prime})\in\mathscr{D}^{p}_{\zeta}([0,T];\mathcal{L}(\mathbb{R}^{d};\mathbb{R}^{m}))$ be a controlled rough path. Then the limit

[TABLE]

exists444Strictly speaking, in making precise sense of the product $X^{\prime}_{s}\zeta^{(2)}_{s,t}$, we use the natural identification of $\mathcal{L}(\mathbb{R}^{d};\mathcal{L}(\mathbb{R}^{d};\mathbb{R}^{m}))$ with $\mathcal{L}(\mathbb{R}^{d}\otimes\mathbb{R}^{d};\mathbb{R}^{m})$., where the limit is taken over any sequence of partitions $\mathcal{P}$ of the interval $[0,T]$ such that the mesh size $|\mathcal{P}|\to 0$. This limit (which does not depend on the choice of sequence of partitions) is called the rough integral of $X$ against $\bm{\zeta}$.

Moreover, for any $0\leq s<t\leq T$, we have the estimate

[TABLE]

where the constant $C_{p}$ depends only on $p$.

2.3 Rough differential equations with controls

For a given $p\in[2,3)$, rough path $\bm{\zeta}=(\zeta,\zeta^{(2)})\in\mathscr{C}^{p}([0,T];\mathbb{R}^{d})$ and control function $\gamma\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}([0,T];\mathbb{R}^{k})$, we consider the rough differential equation (RDE)

[TABLE]

controlled (in the sense of optimal control) by $\gamma$, with $X_{0}=x\in\mathbb{R}^{m}$, where the second term on the right-hand side is interpreted as a rough integral against $\bm{\zeta}$.

The main element that takes us outside the standard RDE setting is the appearance of the control $\gamma$ in the coefficients. Note however that, since $\gamma\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$, it is immediately controlled by $\zeta$ with $\gamma^{\prime}=0$, so that $(\gamma,0)\in\mathscr{D}_{\zeta}^{p}([0,T];\mathbb{R}^{k})$. Then, provided that $\lambda\in C^{2}_{b}$, for any $(X,X^{\prime})\in\mathscr{D}_{\zeta}^{p}([0,T];\mathbb{R}^{m})$, the composition $\lambda(X,\gamma)$ can also be interpreted as being controlled by $\zeta$, with Gubinelli derivative given by

[TABLE]

where $D_{x}\lambda$ is the Fréchet derivative of $\lambda$ in its first argument.

Lemma 2.3**.**

For some $n\geq 1$, let $0=t_{0}<t_{1}<\ldots<t_{n-1}<t_{n}=T$, be a partition of the interval $[0,T]$. Then, for any path $X$, one has that

[TABLE]

Proof.

Let $0=s_{0}<s_{1}<\ldots<s_{N-1}<s_{N}=T$ be another partition of the interval $[0,T]$. We can label the union of these two partitions in two different ways as follows. We can either write

[TABLE]

where, crucially, $n_{j}\leq n$ for every $j$. We have

[TABLE]

The result then follows from taking the supremum over all possible partitions $s_{0}<s_{1}<\ldots<s_{N}$ of the interval $[0,T]$. ∎

Proposition 2.4**.**

Let $b\in\text{Lip}_{b}$ and $\lambda,\psi\in C^{2}_{b}$. For some $p\in[2,3)$ and $L>0$, let $\bm{\zeta}\in\mathscr{C}^{p}$ such that ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{\frac{1}{p}\textrm{{-H\"{o}l}}}\leq L$, and suppose that $X$ satisfies the RDE (2.4) with $X^{\prime}=\lambda(X,\gamma)$, for some $\gamma\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$. We have the following estimates:

1. (i)

\|\psi(X,\gamma)^{\prime}\|_{p}\leq C_{\lambda,\psi,p}\big{(}\|X\|_{p}+\|\gamma\|_{\frac{p}{2}}\big{)}, 2. (ii)

\big{\|}R^{\psi(X,\gamma)}\big{\|}_{\frac{p}{2}}\leq C_{\psi,p}\Big{(}\|X\|_{p}^{2}+\big{\|}R^{X}\big{\|}_{\frac{p}{2}}+\|\gamma\|_{\frac{p}{2}}\Big{)}, 3. (iii)

\|X\|_{p}\leq C_{b,\lambda,p,T,L}\big{(}1+\|\gamma\|_{\frac{p}{2}}^{1+p}\big{)}, 4. (iv)

\big{\|}R^{X}\big{\|}_{\frac{p}{2}}\leq C_{b,\lambda,p,T,L}\big{(}1+\|\gamma\|_{\frac{p}{2}}^{2+p}\big{)},

where in each case the constant $C$ depends only on the variables indicated.

Proof.

The first two estimates follow from standard arguments, noting that the Gubinelli derivative of $\psi(X,\gamma)$ is given by $\psi(X,\gamma)^{\prime}=D_{x}\psi(X,\gamma)\lambda(X,\gamma)$. Let us therefore turn our attention to the proof of (iii). In the following the symbol $\lesssim$ shall denote inequality up to a multiplicative constant depending only on $b,\lambda,p,T$ and $L$.

Let $[s,t]\subseteq[0,T]$. We then have

[TABLE]

where we applied (2.3) to obtain the last line. It follows that for a given interval $I\subseteq[0,T]$ of length $|I|$,

[TABLE]

Applying the estimates in (i) and (ii) with $\psi=\lambda$, we obtain

[TABLE]

for some constant $C_{1}$ (which only depends on $b,\lambda$ and $p$).

Since $\|\zeta\|_{p;I}\leq\|\zeta\|_{\frac{1}{p}\textrm{{-H\"{o}l}}}|I|^{\frac{1}{p}}\leq L|I|^{\frac{1}{p}}$, there exists some $r>0$ (depending only on $p,L$ and $C_{1}$) sufficiently small such that

[TABLE]

whenever $|I|\leq r$. It is enough to prove the result for $T\leq r$, since one can then extend the result to any larger $T$ using Lemma 2.3. We will therefore assume that $T\leq r$, so that (2.6) holds for all intervals $I$ under consideration. We then deduce that

[TABLE]

From the basic estimate

[TABLE]

we then have that

[TABLE]

for some constant $C_{2}$ (depending on $b,\lambda,p$ and $L$). From here, we aim to infer an estimate which holds on small subintervals, and then use Lemma 2.3 to paste such subintervals together to obtain an estimate which holds on the entire interval $[0,T]$.

It follows from above that, if $C_{2}\|X\|_{p;I}\leq\frac{1}{2}$, then

[TABLE]

Let $t^{\ast}=\sup\{t\in[0,T]:C_{2}\|X\|_{p;I}\leq\frac{1}{2}\text{ whenever }|I|\leq t\}$. If $t^{\ast}=T$ then we are done. Otherwise, let $I$ be an interval such that $|I|=t^{\ast}$ and $C_{2}\|X\|_{p;I}=\frac{1}{2}$. Then

[TABLE]

and we deduce that

[TABLE]

The interval $[0,T]$ can be partitioned into $n:=\lceil T/t^{\ast}\rceil$ subintervals $I$ of length at most $t^{\ast}$, on each of which we have $\|X\|_{p;I}\leq\frac{1}{2C_{2}}$. From Lemma 2.3, we obtain the bound $\|X\|_{p;[0,T]}\lesssim n^{1+\frac{1}{p}}$, where $n\leq 1+T/t^{\ast}\lesssim 1+\|\gamma\|_{\frac{p}{2};[0,T]}^{p}$, and the estimate in (iii) follows.

Substituting (2.8) into (2.7), we have

[TABLE]

for some new constant $C_{3}$. This equation is of the same form as (2.9). We can thus apply exactly the same argument as above to deduce the estimate in (iv). ∎

The results of Theorem 2.5 and Proposition 2.6 below are new in this setting due to the inclusion of the control function $\gamma$, particularly in the controlled path setting of Gubinelli with path regularity measured in $p$-variation, but they are based upon standard results, so we shall postpone their proofs to the Appendix.

Theorem 2.5**.**

Let $b\in\textrm{{Lip}}_{b}$, $\lambda\in C^{3}_{b}$ and $\bm{\zeta}\in\mathscr{C}^{p}$. For any $x\in\mathbb{R}^{m}$ and any $\gamma\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$, there exists a unique solution $(X,X^{\prime})\in\mathscr{D}_{\zeta}^{p}$ to the RDE

[TABLE]

such that $X^{\prime}=\lambda(X,\gamma)$, where $\lambda(X,\gamma)$ is interpreted as a controlled rough path with Gubinelli derivative given by (2.5).

Proposition 2.6**.**

Let $b\in\textrm{{Lip}}_{b}$, $\lambda\in C^{3}_{b}$, $\gamma,\vartheta\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$ and $\bm{\eta},\bm{\zeta}\in\mathscr{C}^{p}$ with ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\eta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{\frac{1}{p}\textrm{{-H\"{o}l}}}\leq L$, ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{\frac{1}{p}\textrm{{-H\"{o}l}}}\leq L$. Let $(X,X^{\prime})=(X,\lambda(X,\gamma))\in\mathscr{D}^{p}_{\eta}$ (resp. $(Y,Y^{\prime})=(Y,\lambda(Y,\vartheta))\in\mathscr{D}^{p}_{\zeta}$) be the unique solution of the RDE (2.10) controlled by $\gamma$ (resp. $\vartheta$) and driven by $\bm{\eta}$ (resp. $\bm{\zeta}$) with the initial condition $x$ (resp. $y$). Suppose that $\|\gamma\|_{\frac{p}{2}},\|\vartheta\|_{\frac{p}{2}}\leq M$ for some $M>0$. Then

[TABLE]

Moreover, given $\psi\in C^{3}_{b}$, we have

[TABLE]

Here the constants $C,C^{\prime}$ depend on $b,\lambda,p,T,L$ and $M$, and $C^{\prime}$ also depends on $\psi$.

3 Pathwise optimal control

3.1 Avoiding degeneracy

Our set-up is the following. We fix a geometric rough path $\bm{\zeta}=(\zeta,\zeta^{(2)})\in\mathscr{C}_{g}^{0,p}([0,T];\mathbb{R}^{d})$ such that ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{\frac{1}{p}\textrm{{-H\"{o}l}}}\leq L$ for some $p\in[2,3)$ and $L>0$. We consider, for each $\gamma\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}([0,T];\mathbb{R}^{k})$, the controlled dynamics

[TABLE]

driven by $\bm{\zeta}$. We then consider the control problem with value function given by

[TABLE]

for $(t,x)\in[0,T]\times\mathbb{R}^{m}$, where the cost functional $J$ is defined as

[TABLE]

Here $f\colon\mathbb{R}^{m}\times\mathbb{R}^{k}\to\mathbb{R}$, $\psi\colon\mathbb{R}^{m}\times\mathbb{R}^{k}\to\mathcal{L}(\mathbb{R}^{d};\mathbb{R})$ and $g\colon\mathbb{R}^{m}\to\mathbb{R}$.

Lemma 3.1**.**

Suppose that $b\in\textrm{{Lip}}_{b}$, $\lambda\in C^{3}_{b}$ and $\psi\in C^{2}_{b}$. Then, for any $t,x$ and $\gamma$, we have that

[TABLE]

where the constant $C$ depends only on $b,\lambda,\psi,p,T$ and $L$.

Proof.

By Theorem 2.5, the RDE (3.1) has a unique solution $(X^{t,x,\gamma},\lambda(X^{t,x,\gamma},\gamma))\in\mathscr{D}^{p}_{\zeta}$ for any $\gamma\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$, and the integral $\int_{t}^{\cdot}\psi(X^{t,x,\gamma}_{s},\gamma_{s})\,\mathrm{d}\bm{\zeta}_{s}$ is then well-defined. By (2.3), we have that

[TABLE]

Applying the estimates in Proposition 2.4, we deduce (3.4). ∎

Remark 3.2*.*

The choice to measure the regularity of the controls using $q$-variation for $q=\frac{p}{2}$ ensures that $\frac{1}{p}+\frac{1}{q}>1$, so that the corresponding Young integral of $\gamma$ against $\zeta$ always exists. It may be tempting to wonder whether the result of Lemma 3.1 could still hold using a bound based on the $q$-variation of controls for a larger value of $q$. However, this is not true in general. Indeed, if $\frac{1}{p}+\frac{1}{q}<1$, then one can construct a sequence $\{(\zeta^{n},\gamma^{n})\}_{n\geq 1}$ of pairs of bounded variation paths such that $\|\zeta^{n}\|_{p}=1=\|\gamma^{n}\|_{q}$ for all $n\geq 1$, but such that $\int_{0}^{T}\gamma^{n}_{s}\,\mathrm{d}\zeta^{n}_{s}\to\infty$ as $n\to\infty$, which would contradict (3.4).

Preventing degeneracy of this control problem can essentially be thought of as preventing the size of the rough integral above from becoming arbitrarily large. Lemma 3.1 shows that one can control the size of this integral by the $\frac{p}{2}$-variation of the controls. However, since controls can exhibit arbitrarily large $\frac{p}{2}$-variation whilst remaining uniformly bounded, the cost functional in (3.3) is not able to adequately penalise this variation. In view of Example 1.1, for a typical choice of $\psi$, one should expect the value function in (3.2) to be simply given by

[TABLE]

We also point out that merely restricting the class of controls $\gamma$ to, say, smooth functions does nothing to resolve this problem.

The estimate in (3.4) implies that one could prevent degeneracy by imposing a uniform bound on the $\frac{p}{2}$-variation of the controls but, as appreciated in Diehl et al. [18], this would not be a very natural condition. Instead, we first propose to introduce an artificial cost in order to penalise this variation.

Definition 3.3**.**

Let $\mathcal{S}\subseteq\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$ be a Banach space of functions from $[0,T]\to\mathbb{R}^{k}$ (with a possibly stronger topology). We shall call a function $\beta\colon\Delta_{[0,T]}\times\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}\to\mathbb{R}\cup\{+\infty\}$ a regularising cost on $\mathcal{S}$, if it is bounded below, takes the value $+\infty$ on $\Delta_{[0,T]}\times(\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}\setminus\mathcal{S})$, and, for every $0\leq r<t\leq T$, the map $\beta_{r,t}\colon\mathcal{S}\to\mathbb{R}$ is continuous, and satisfies

[TABLE]

An example of such a cost on $\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$ is given by

[TABLE]

for any $\varepsilon>0$ and $q>2(1+p)$.

Remark 3.4*.*

We point out that the power in the denominator in (3.5) is sufficient but by no means necessary. This choice is a result of the estimate in (3.4), which we do not expect to be sharp.

Instead of the naive value function in (3.2), we consider the modified function given by555The restriction to controls $\gamma\in\mathcal{C}^{0\textrm{{,}}\frac{p}{2}\textrm{{-var}}}\subset\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$ is negligible. Indeed, we recall that $\mathcal{C}^{\frac{q}{2}\textrm{{-var}}}\subset\mathcal{C}^{0\textrm{{,}}\frac{p}{2}\textrm{{-var}}}$ for any $q\in[2,p)$.

[TABLE]

for some regularising cost $\beta$. In practice, the justification of the introduction of this ‘artificial cost’ depends on the application one has in mind; we will see examples of this later in Sections 3.6 and 4.5.

The following proposition demonstrates the nondegeneracy of this modified control problem.

Proposition 3.5**.**

Under the natural assumption that $f$ and $g$ are bounded below, the same is true of the value function $V$.

Proof.

It follows from Lemma 3.1 and (3.5) that

[TABLE]

for some new constant $C$, and hence that

[TABLE]

Since the cost functions $f,g$ and $\beta$ are all bounded below, the result follows. ∎

3.2 Recovering dynamic programming

We have seen that one can resolve the degeneracy of the optimal control problem by introducing an artificial cost to penalise the variation of the controls. In Definition 3.3 we introduced a rather general class of cost functions which provide a sufficient penalisation. The problem with such cost functions, such as the one in (3.6), is that typically they are not additive, in the sense that $\beta_{r,s}+\beta_{s,t}\neq\beta_{r,t}$. A consequence of this is that the corresponding control problem is no longer dynamic. That is, the value function in (3.7) is not generally amenable to dynamic programming, and thus one cannot necessarily write down a PDE associated with the control problem. Our next aim will be to demonstrate the existence of an additive regularising cost on a more regular space of controls, which allows dynamic programming to be recovered.

Lemma 3.6**.**

Let $\beta$ be a regularising cost on $\mathcal{C}^{0\textrm{{,}}\frac{p}{2}\textrm{{-var}}}$. Let $\mathcal{S}\subseteq\mathcal{C}^{0\textrm{{,}}\frac{p}{2}\textrm{{-var}}}$ be a subset which contains all smooth functions from $[0,T]\to\mathbb{R}^{k}$. Then the value function defined in (3.7) satisfies

[TABLE]

Proof.

By definition, for any $\gamma\in\mathcal{C}^{0\textrm{{,}}\frac{p}{2}\textrm{{-var}}}$, there exists a sequence of smooth controls $\{\gamma^{n}\}_{n\geq 1}$ such that $\|\gamma^{n}-\gamma\|_{\infty}+\|\gamma^{n}-\gamma\|_{\frac{p}{2}}\to 0$ as $n\to\infty$. The result then follows from the continuity of $\beta_{t,T}$ and the stability estimates in Proposition 2.6. ∎

In particular, (3.8) holds with $\mathcal{S}=W^{1,q}$ for any $q\geq 1$, where $W^{1,q}=W^{1,q}([0,T];\mathbb{R}^{k})$ denotes the usual Sobolev space. We recall the continuous embeddings $W^{1,q}\hookrightarrow\mathcal{C}^{1\text{-var}}\hookrightarrow\mathcal{C}^{0\textrm{{,}}\frac{p}{2}\textrm{{-var}}}$, exhibited by the inequalities

[TABLE]

where we write $\dot{\gamma}$ for the unique element $\dot{\gamma}\in L^{q}([0,T];\mathbb{R}^{k})$ such that $\mathrm{d}\gamma_{s}=\dot{\gamma}_{s}\,\mathrm{d}s$. It follows that, for any $\varepsilon>0$ and $q>2(1+p)$, the choice

[TABLE]

for $\gamma\in W^{1,q}$ (and $\beta(\gamma)\equiv\infty$ otherwise), defines a regularising cost on $W^{1,q}$. Moreover, $\beta$ is additive, in the sense that $\beta_{r,s}+\beta_{s,t}=\beta_{r,t}$ for all $r\leq s\leq t$; in other words, for each $\gamma\in W^{1,q}$, the two-parameter functional $\beta(\gamma)\colon\Delta_{[0,T]}\to\mathbb{R}$ is uniquely characterised by the path $t\mapsto\beta_{0,t}(\gamma)$. With this choice of $\beta$, we can now write

[TABLE]

where, for $(t,x,a)\in[0,T]\times\mathbb{R}^{m}\times\mathbb{R}^{k}$,

[TABLE]

with $\gamma^{t,a,u}_{r}:=a+\int_{t}^{r}u_{s}\,\mathrm{d}s$ for $r\in[t,T]$. The function $v$ is both nondegenerate, and satisfies the following:

Proposition 3.7** (Dynamic programming principle).**

Let us write $X^{t,x,a,u}:=X^{t,x,\gamma^{t,a,u}}$. Then, for any $t,x,a$ and $r\in[t,T]$, with $v$ as in (3.9), we have

[TABLE]

This result follows the same proof as that of Theorem 2.1 in [40, Chapter 4]. In particular, the rough integrals appearing in the controlled dynamics and value function do not cause any additional difficulty.

3.3 A generalised dynamic control problem

To summarise the previous subsections, we propose to reformulate the naive control problem, given originally by (3.1)–(3.3), to resolve the degeneracy problem whilst retaining enough dynamic structure to retain dynamic programming, by restricting to a sufficiently regular space of controls, and introducing an additive artificial cost function, written in terms of the derivative of the controls. Rather than merely (3.1), by including $\gamma$ as part of the state trajectory, we instead consider the controlled dynamics

[TABLE]

For generality, we have introduced the function $h\colon\mathbb{R}^{k}\times U\to\mathbb{R}^{k}$, where here $(U,\|\cdot\|_{U})$ is a finite dimensional Banach space, and the control $u$ belongs to the space $\mathcal{U}$ of bounded measurable functions $u\colon[0,T]\to U$.

We shall henceforth consider the cost functional

[TABLE]

and the value function

[TABLE]

where we have absorbed a regularising cost into the function $f\colon\mathbb{R}^{m}\times\mathbb{R}^{k}\times U\to\mathbb{R}$, which crucially is now also allowed to depend on $u$. There is also no harm in allowing the terminal cost $g\colon\mathbb{R}^{m}\times\mathbb{R}^{k}\to\mathbb{R}$ to depend on the terminal value of $\gamma$.

Remark 3.8*.*

If the function $h=h(a,u)$ were bounded in $u$, then setting $\tilde{X}=(X,\gamma)$ would now put us into a comparable setting to Diehl et al. [18]. However, it is more natural here to allow $h$ to be unbounded in $u$, meaning that [18, Theorem 5] does not directly apply666This boundedness condition is not stated explicitly in [18], but is necessary for the application of [4, Corollary III.3.6] in the proof of [18, Theorem 5]; see Assumption (A1) in [4, Chapter III].. Moreover, in [18] the cost functions $f$ and $g$ are assumed to be bounded, but we will relax this assumption in the current work.

The inclusion of the integral $\int\psi(X,\gamma)\,\mathrm{d}\bm{\zeta}$ in the value function also takes us outside the setting of [18]. This term could be included in the terminal cost by setting $\tilde{X}=(X,\gamma,Z)$ and $\tilde{g}(x,a,z)=g(x,a)+z$ with $Z_{r}=z+\int_{t}^{r}\psi(X,\gamma)\,\mathrm{d}\bm{\zeta}$, albeit with the additional complication that the terminal cost $\tilde{g}$ would then be neither bounded from above nor below.

Assumption 3.9**.**

We assume that

• •

$b\in\textrm{{Lip}}_{b}$ and $\lambda,\psi\in C^{3}_{b}$,

• •

$f=f(x,a,u)$ and $g=g(x,a)$ are continuous, bounded below, and Lipschitz continuous in $(x,a)$, uniformly in $u$,

• •

$h=h(a,u)$ is continuous, Lipschitz in $a$, uniformly in $u$, and is bounded in $a$, locally uniformly in $u$, and moreover, for some $\delta\geq 1$, satisfies

[TABLE]

• •

with the same $\delta$ as in (3.13), the running cost $f$ satisfies

[TABLE]

Remark 3.10*.*

One could in principle also allow the drift coefficient $b$ to depend on the control $u$. In this case it is less straightforward to obtain solutions to the RDE (3.10), but the necessary technical results have already been established in [18].

The following lemma demonstrates the nondegeneracy of our newly formulated control problem.

Lemma 3.11**.**

For any $t,x,a$ and $u$, we have that

[TABLE]

where the constant $C$ depends only on $b,\lambda,\psi,h,p,T$ and $L$.

Proof.

By (3.4), (3.11) and Hölder’s inequality, we have that

[TABLE]

where $p^{\prime}$ is the Hölder conjugate of $2(1+p)$. Then, by (3.13) and (3.14) (noting that, since $U$ is finite dimensional, $h$ is uniformly bounded on bounded subsets of $U$), we can ensure that (3.15) holds for a new constant $C$. ∎

Corollary 3.12**.**

Let $K$ be a compact subset of $\mathbb{R}^{m}\times\mathbb{R}^{k}$. There exists an $M>0$ such that, when taking the infimum over $u\in\mathcal{U}$ in (3.12) for $(t,x,a)\in[0,T]\times K$, one may restrict to controls $u$ satisfying $\|\gamma^{t,a,u}\|_{\frac{p}{2}}\leq M$.

Proof.

By Lemma 3.11 and the assumption that $g$ is bounded below, we have that

[TABLE]

for some possibly new constant $C$. Let $u^{\ast}\in\mathcal{U}$ be an arbitrary control. By the above, we may ignore all controls $u$ such that

[TABLE]

This gives an upper bound on $\int_{t}^{T}f(X^{t,x,a,u}_{s},\gamma^{t,a,u}_{s},u_{s})\,\mathrm{d}s$, which we observe, by the proof of Lemma 3.11, also implies an upper bound on $\|\gamma^{t,a,u}\|_{\frac{p}{2}}$. ∎

3.4 A smooth noise approximation

Although a dynamic programming principle of the form in Proposition 3.7 holds for the value function $v$ in (3.12), the appearance of the rough integrals makes it less straightforward to derive a PDE directly from this result. As in Example 1.1, we will therefore proceed by first approximating $\zeta$ by a smooth function $\eta$. We then define the corresponding approximate control problem, with dynamics

[TABLE]

where $\gamma^{t,a,u}$ satisfies (3.11). Naturally equation (3.16) has a unique $C^{1}$ solution. However, in the following it will be useful to also embed this solution in rough path space. As $\eta$ is smooth, we can simply enhance it with its iterated integrals in the classical Lebesgue–Stieltjes sense,

[TABLE]

so that $\bm{\eta}=(\eta,\eta^{(2)})$ is itself a rough path. Any continuous path with finite $p$-variation would make a valid candidate for the Gubinelli derivative of $X^{\eta}$ (with respect to $\eta$), but to be consistent with the genuinely rough case above we insist on the choice $(X^{\eta})^{\prime}=\lambda(X^{\eta},\gamma)$. We can then consider $(X^{\eta},\lambda(X^{\eta},\gamma))$ as the solution of (3.16) in the sense of Theorem 2.5.

We also define the corresponding approximate value function $v^{\eta}$ as

[TABLE]

Writing $\dot{\eta}$ for the derivative of $\eta$, under Assumption 3.9, we can apply Theorem 3.2 in Bardi and Da Lio [5], to obtain that $v^{\eta}$ is the unique viscosity solution of the HJB equation

[TABLE]

with the terminal condition

[TABLE]

Moreover, by Theorem 2.2 in [5], this solution is locally Lipschitz continuous.

3.5 A rough HJB equation

Replacing $\eta$ in (3.18) with $\bm{\zeta}$, we formally derive the rough PDE given by

[TABLE]

with

[TABLE]

We point out that as written equation (3.20) is only formal, and is given a precise meaning in Definition 3.13 below.

The following definition exhibits a standard notion of solution for rough PDEs, used in [18], as well as for instance by Caruana, Friz and Oberhauser [9, 10, 23] (see also Chapter 12 in [22]).

Definition 3.13**.**

For any smooth function $\eta\colon[0,T]\to\mathbb{R}^{d}$, write $v^{\eta}$ for the unique viscosity solution of (3.18) and (3.19). Moreover, write $\bm{\eta}$ for the rough path obtained by enhancing $\eta$ with its iterated integrals in the Lebesgue–Stieltjes sense, as in (3.17). We say that a continuous function $v$ solves (3.20) and (3.21) if

[TABLE]

locally uniformly on $[0,T]\times\mathbb{R}^{m}\times\mathbb{R}^{k}$, whenever $(\eta^{n})_{n\geq 1}$ is a sequence of smooth paths such that $\bm{\eta}^{n}\to\bm{\zeta}$ with respect to the $\frac{1}{p}$-Hölder rough path metric, i.e. $\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}(\bm{\eta}^{n},\bm{\zeta})\to 0$ as $n\to\infty$.

Note that uniqueness of such a solution is built into the definition. Moreover, note that since we assumed that $\bm{\zeta}$ is a geometric rough path, there certainly exists such a sequence of smooth paths $(\eta^{n})_{n\geq 1}$.

Theorem 3.14**.**

Under Assumption 3.9, the value function $v$ defined in (3.12) solves (3.20) and (3.21) in the sense of Definition 3.13. Moreover, writing $v=v^{\zeta}$, the map from $\mathscr{C}^{0,p}_{g}([0,T];\mathbb{R}^{d})\to\mathbb{R}$ given by $\bm{\zeta}\mapsto v^{\zeta}(t,x,a)$ is locally uniformly continuous with respect to each of the rough path metrics $\varrho_{p}$ and $\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}$, locally uniformly in $(t,x,a)$.

Proof.

Let $K$ be a compact subset of $\mathbb{R}^{m}\times\mathbb{R}^{k}$ and let $\bm{\eta}\in\mathscr{C}^{p}$ be another rough path such that $\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}(\bm{\eta},\bm{\zeta})\leq 1$. By possibly replacing $L$ by $L+1$, we may assume that ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\eta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{\frac{1}{p}\textrm{{-H\"{o}l}}}\leq L$. Let us write $X^{\eta}=X^{t,x,a,u,\eta}$ (resp. $X^{\zeta}=X^{t,x,a,u,\zeta}$) for the solution of the RDE (3.10) driven by $\bm{\eta}$ (resp. $\bm{\zeta}$), and write $v^{\eta}$ (resp. $v^{\zeta}$) for the corresponding value function, as defined in (3.12).

By Corollary 3.12, there exists an $M>0$ such that, for $(t,x,a)\in[0,T]\times K$, we may restrict to controls $u\in\mathcal{U}^{M}\subseteq\mathcal{U}$ satisfying $\|\gamma^{t,a,u}\|_{\frac{p}{2}}\leq M$, so that in particular the hypotheses of Proposition 2.6 are satisfied.

In the following we shall use $\lesssim$ to denote inequality up to a multiplicative constant which may depend on $b,\lambda,\psi,f,g,h,p,T,L$ and $M$. It follows from Proposition 2.6 that

[TABLE]

and

[TABLE]

By the Lipschitz assumptions on $f$ and $g$, for any $(t,x,a)\in[0,T]\times K$, we have

[TABLE]

Taking a sequence of smooth paths $(\eta^{n})_{n\geq 1}$ such that $\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}(\bm{\eta}^{n},\bm{\zeta})\to 0$, the required convergence follows by taking $\bm{\eta}=\bm{\eta}^{n}$ in the above. Since the approximate value functions $v^{\eta^{n}}$ are continuous, continuity of the function $v$ with respect to $(t,x,a)$ also follows from this convergence. The stated continuity of the value function with respect to the driving rough path is also immediate from the above. ∎

Remark 3.15*.*

One could also introduce another Brownian motion $W$ and consider as controlled dynamics the hybrid Itô-rough differential equation $\mathrm{d}X=b(X,\gamma)\,\mathrm{d}s+\sigma(X,\gamma)\,\mathrm{d}W+\lambda(X,\gamma)\,\mathrm{d}\bm{\zeta}$. Just as in the classical case, the value function is then defined as the infimum (or supremum) over adapted controls of an expected cost function, and the associated HJB equation is then of second order; see Example 3.17 below.

3.6 Examples

Example 3.16**.**

When a Brownian motion $W$ is enhanced with its iterated integrals in the sense of Stratonovich integration, i.e.

[TABLE]

then, almost surely, $\mathbf{W}=(W,W^{(2)})$ defines a $\frac{1}{p}$-Hölder geometric rough path for any $p\in(2,3)$. The choice $\bm{\zeta}=\mathbf{W}$ thus leads to the stochastic PDE

[TABLE]

Example 3.17** (Insider trading revisited).**

Let us return to the setting of Example 1.1, where we recall that an agent is trading a stock with the benefit of some extra information not available to the rest of the market. We denote the agent’s initial investment by $a$, and the rate at which they purchase new stock by $u$. The dynamics of the agent’s wealth process $X$ and investment $\gamma$ are given by777When $\sigma\neq 0$ the inclusion of the Brownian motion takes us outside the class of problems considered above, but there is no conceptual change and we expect all of the analysis to follow with appropriate technical adjustments.

[TABLE]

where $W$ is a Brownian motion and $\zeta$ is an arbitrary continuous path. Note that, since $\zeta$ is continuous and $\gamma^{t,a,u}$ is of finite variation, the integral $\int_{t}^{\cdot}\gamma^{t,a,u}_{s}\,\mathrm{d}\zeta_{s}$ exists in the Riemann–Stieltjes sense, so there is no need here to lift $\zeta$ into rough path space.

Let us suppose that the agent must pay a transaction cost of $\varepsilon u^{2}$. The agent’s expected terminal wealth is then given by the value function

[TABLE]

where $\mathcal{U}$ is the space of progressively measurable $\mathbb{R}$-valued processes. In this case the HJB equation (3.20) takes the form

[TABLE]

with

[TABLE]

Approximating $\zeta$ by a smooth function $\eta$, we obtain the classical HJB equation

[TABLE]

The solution of this equation along with the terminal condition (3.24) is given by

[TABLE]

Recalling Definition 3.13, we obtain the solution of (3.23) and (3.24) as

[TABLE]

Note that this quantity remains finite even when $\zeta$ is of infinite variation. Thus, an agent, even with perfect knowledge of the future stock price, subject to sufficient transaction costs, can only make a finite profit.

4 Robust filtering

4.1 The Kalman–Bucy filter

In this section we turn our attention to the problem of stochastic filtering under model uncertainty. Let us take an underlying filtered space $(\Omega,\mathcal{F},(\mathcal{F}_{t})_{t\geq 0})$. We suppose that an $\mathbb{R}^{m}$-valued signal process $S$ and an $\mathbb{R}^{d}$-valued observation process $Y$ satisfy the following pair of linear equations

[TABLE]

with the initial conditions $Y_{0}=0$ and $S_{0}\sim N(\mu_{0},\Sigma_{0})$ for some $\mu_{0}\in\mathbb{R}^{m}$ and $\Sigma_{0}\in\mathcal{S}_{+}^{m}$, where $\mathcal{S}_{+}^{m}$ denotes the set of symmetric, positive definite $m\times m$-matrices. Here $B^{1}$ (resp. $B^{2}$) is a standard $\mathbb{R}^{l}$(resp. $\mathbb{R}^{d}$)-valued Brownian motion, and $\alpha\colon[0,T]\to\mathbb{R}^{m\times m}$, $\sigma\colon[0,T]\to\mathbb{R}^{m\times l}$ and $c\colon[0,T]\to\mathbb{R}^{d\times m}$ are parameters. Here we include the case when the signal noise and observation noise are correlated; we suppose that their quadratic covariation is given by

[TABLE]

for some correlation matrix $\rho\colon[0,T]\to\mathbb{R}^{l\times d}$. In the scalar case, the correlation should naturally satisfy $\rho^{2}\leq 1$. The analogous assumption here is that the matrix $I-\rho\rho^{\top}$ be positive semi-definite, where $I$ denotes the $l\times l$ identity matrix.

We shall denote by $(\mathcal{Y}_{t})_{t\geq 0}$ the (completed) natural filtration generated by $Y$. In short, the filtering problem is concerned with, at each time $t$, determining the best estimate for $S_{t}$ given $\mathcal{Y}_{t}$, that is, finding the best estimate for the current value of $S$, given our past observations of $Y$. The mathematical theory underpinning the filtering of stochastic systems is by now well understood; a particularly good exposition is given in Bain and Crisan [3]. As observed by Kalman and Bucy [27, 28], and subsequently studied by numerous authors in various contexts, in this setting where, crucially, the underlying dynamics are linear, the conditional distribution of $S_{t}$ given $\mathcal{Y}_{t}$ is Gaussian. Moreover, the conditional mean $q_{t}=\mathbb{E}[S_{t}\,|\,\mathcal{Y}_{t}]$ of this distribution satisfies the SDE

[TABLE]

and the conditional variance $R_{t}=\mathbb{E}[(S_{t}-q_{t})(S_{t}-q_{t})^{\top}|\,\mathcal{Y}_{t}]$ satisfies the deterministic matrix Riccati equation

[TABLE]

The filtering equations above allow one to fully characterise the conditional distribution of the signal. However, this procedure assumes that we know a priori the exact values of the parameters $\alpha,\sigma,c$ and $\rho$. In practice these parameters must be estimated from data, and in adopting these estimates one concedes an additional source of statistical uncertainty. In the present work we are interested in incorporating this uncertainty directly into the construction of the filter. That is, we are interested in stochastic filtering for linear systems which is robust with respect to model uncertainty.

4.2 Robust filtering via nonlinear expectations

Robust filtering has been studied in various papers, predominantly in the engineering literature. A typical approach is to construct an optimization procedure based on a minimax estimator for the hidden state, whereby one attempts to minimize a maximum expected loss over the space of possible models. See for instance the work of Borisov [6, 7], Miller and Pankov [32], Siemenikhin, Lebedev and Platonov [34, 35] or Verdú and Poor [37]. By design, such estimators take into account a generally large set of models, even though many of them should be considered to be very implausible, thus often sacrificing filter performance under the most statistically reasonable model. Another approach is that of $H_{\infty}$, as well as hybrid $H_{2}/H_{\infty}$ filtering, which examines the energy gain from the noise input to the filtering error and attempts to minimize this energy transfer subject to suitable constraints; see Aliyu and Boukas [1], Chen and Zhou [11], Khargonekar, Rotea and Baeyens [29], Xie, de Souza and Fu [38] or Yang and Ye [39].

A new approach to filtering in the presence of uncertainty was introduced in [12], which utilises a nonlinear expectation described in terms of a penalty function, which describes how our uncertainty evolves through time. This penalty can be calculated recursively, and can be used to construct robust estimates for any number of nonlinear functionals of the signal process, as well as robust interval estimates analogous to classical confidence/credible intervals.

The first application of this approach in a continuous time setting was presented in [2], which studies a similar setting the one described above. In that paper however, the parameter $c$ was assumed to be known, and the signal and observation noises were assumed to be uncorrelated. In the current work we shall relax these assumptions, and also allow a more general penalty, which in particular takes into account the statistical likelihood for different parameter choices. As we will see, this approach will lead to the derivation of a pathwise stochastic control problem, and thus require the central ideas of the previous sections in order to proceed.

We consider convex expectations, that is maps $\mathcal{E}(\,\cdot\,|\,\mathcal{Y}_{t})\colon L^{\infty}(\mathcal{F})\to L^{\infty}(\mathcal{Y}_{t})$ satisfying the properties of monotonicity, translation equivariance, normalization and convexity, which additionally satisfy the Fatou property. Equivalently, and more explicitly, we consider maps which admit a representation of the form

[TABLE]

where $\mathcal{Q}_{t}$ is a collection of equivalent probability measures, and $\beta(\hskip 1.0pt\cdot\,|\,\mathcal{Y}_{t})$ is a nonnegative $\mathcal{Y}_{t}$-measurable penalty function. See e.g. Föllmer and Schied [20, 21] for a proper exposition of the theory of nonlinear expectations.

As can be inferred from (4.3), in the context of model uncertainty, i.e. uncertainty in the underlying probability measure, nonlinear expectations provide an evaluation of random variables which takes into account every admissible measure. In other words, they consider every plausible view of the world, and envisage the worst case scenario. However, in contrast with sublinear expectations, the inclusion of the penalty term $\beta(\hskip 1.0pt\cdot\,|\,\mathcal{Y}_{t})$ means that we can penalise different measures according to how unreasonable we consider them to be, thus restricting our attention to only those measures which we consider to be realistic. Convex expectations are in this sense less pessimistic than their sublinear counterparts.

In our setting, the class of admissible measures simply corresponds to the family of possible parameters $\alpha,\sigma,c,\rho,\mu_{0}$ and $\Sigma_{0}$ of the dynamics of the signal and observation processes. For notational brevity, we shall denote888One is not obliged to consider all of these parameters as being uncertain, but we will focus on this, the most general case.

[TABLE]

and write

[TABLE]

for the space in which $\gamma$ takes values, where $\Upsilon$ denotes the space of valid correlation matrices:

[TABLE]

where $\lambda_{\mathrm{max}}(\hskip 0.5pt\cdot\hskip 0.5pt)$ denotes the largest eigenvalue. We write $\mathbb{P}^{\gamma,\mu_{0},\Sigma_{0}}$ for the measure associated with the parameters $\gamma,\mu_{0}$ and $\Sigma_{0}$, and write $\mathbb{E}^{\gamma,\mu_{0},\Sigma_{0}}$ for the corresponding expectation.

For a given uncertainty aversion parameter $k_{1}>0$ and exponent $k_{2}\geq 1$, we define, for any real-valued bounded measurable function $\varphi$, the convex expectation with the representation

[TABLE]

Here the essential supremum is taken over all possible parameters $(\mu_{0},\Sigma_{0})\in\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}$ for the initial distribution of the signal, and over all choices of parameters $\gamma$ governing the dynamics of $S$ and $Y$.

In view of the insights of the previous section, we anticipate the eventual need to restrict to a sufficiently regular space of parameters $\gamma$ (which we will later refer to as controls). We consequently make the following assumption.

Assumption 4.1**.**

We shall take the space of possible parameters $\gamma$ to be the family of all absolutely continuous functions $\gamma\colon[0,T]\to\Gamma$ with bounded derivative.

The penalty function $\beta$ represents our opinion of how unreasonable different values of the parameters are. We shall discuss this term further in the next subsection. The uncertainty aversion parameters $k_{1},k_{2}$ are included for generality, but will play no significant role in our analysis.

The nonlinear expectation defined above can be used to construct a ‘robust’ point estimate of $\varphi(S_{t})$, as

[TABLE]

Moreover, the nonlinear expectation $\mathcal{E}(\varphi(S_{t})\,|\,\mathcal{Y}_{t})$ will typically overestimate the true value of $\varphi(S_{t})$, so one may therefore think of $\mathcal{E}(\varphi(S_{t})\,|\,\mathcal{Y}_{t})$ as an ‘upper’ expectation. Defining the corresponding ‘lower’ expectation by $-\mathcal{E}(-\varphi(S_{t})\,|\,\mathcal{Y}_{t})$, one can then construct a robust interval estimate for $\varphi(S_{t})$ via

[TABLE]

4.3 The penalty function

In [2] the penalty $\beta$ was assumed to be fixed a priori, i.e. it only took our prior beliefs into account. Although the parameters of the underlying system are unknown, as we make new observations we may wish to use these observations to update our opinion of how reasonable different parameter choices are. We shall therefore suppose that this penalty takes the form of a negative log-posterior density. That is, we take

[TABLE]

where $\pi$ and $L(\hskip 1.0pt\cdot\,|\,\mathcal{Y}_{t})$ denote the prior and likelihood respectively.

The penalty function in (4.6) is built from the log-likelihood function, a familiar object from classical statistics. Penalties based on log-likelihoods form the basis of the data-driven robust (DR) expectation of [13], which allows the level of penalisation of different parameter choices to be recursively updated through time as we collect new observations. We refer to [13] for further discussion. Here we add to this an additional penalty based on our prior beliefs, which may be calibrated accordingly.

We shall assume that the prior takes the form

[TABLE]

where the functions $\mathfrak{f}$ and $g$ may be calibrated to represent our prior beliefs about the plausibility of different parameter choices. Here $q$ and $R$ are the conditional mean and variance corresponding to the parameters $\gamma,\mu_{0}$ and $\Sigma_{0}$, given by the solutions of (4.1) and (4.2).

Note that the measures $\mathbb{P}^{\gamma,\mu_{0},\Sigma_{0}}$ for different choices of $\gamma,\mu_{0}$ and $\Sigma_{0}$ are all equivalent on $\mathcal{Y}_{t}$. A natural choice for $L_{t}(\hskip 1.0pt\cdot\,|\,\mathcal{Y}_{t})$ is thus the Radon–Nikodym derivative

[TABLE]

which is precisely the likelihood ratio of the (arbitrary) parameter choice $\gamma,\mu_{0},\Sigma_{0}$, with respect to a (fixed) choice of reference parameters $\gamma^{\ast},\mu_{0}^{\ast},\Sigma_{0}^{\ast}$. We will now derive an explicit expression for this likelihood.

Recall (from e.g. Bain and Crisan [3, Chapter 2]) that for a given choice of parameters $\gamma,\mu_{0},\Sigma_{0}$, the innovation process $V$, given in this setting by

[TABLE]

is a $\mathcal{Y}_{t}$-adapted Brownian motion under $\mathbb{P}^{\gamma,\mu_{0},\Sigma_{0}}$. Writing $q^{\ast}$ (resp. $V^{\ast}$) for the conditional mean (resp. innovation process) under the reference measure $\mathbb{P}^{\gamma^{\ast},\mu_{0}^{\ast},\Sigma_{0}^{\ast}}$, we have

[TABLE]

Thus, by Girsanov’s theorem (see e.g. [14, Chapter 15]), as $V$ and $V^{\ast}$ have the predictable representation property under their respective measures, we can represent the likelihood as a stochastic exponential, namely

[TABLE]

Substituting $\mathrm{d}V^{\ast}_{s}=\mathrm{d}Y_{s}-c^{\ast}_{s}q^{\ast}_{s}\,\mathrm{d}s$, a short calculation yields

[TABLE]

Since the reference parameters are taken to be fixed, they simply amount to an additive constant in the above expression. That is,

[TABLE]

For simplicity we will henceforth omit this constant from our analysis, conceding that our penalty function is correct up to an additive constant. This constant may be reintroduced upon numerical computation of the nonlinear expectation, chosen to ensure that the penalty function always takes the value zero at its minimum.

It will be useful later to interpret the stochastic integral in (4.8) in the sense of Stratonovich, rather than that of Itô. We therefore make the transformation

[TABLE]

Recalling (4.1), and using the fact that $c$ is absolutely continuous and in particular of bounded variation, after some calculation we deduce that the quadratic covariation term is given by

[TABLE]

where $\textrm{{tr}}(\hskip 1.0pt\cdot\hskip 1.0pt)$ denotes the trace. Note that $c_{s}R_{s}c_{s}^{\top}$ is positive semi-definite and therefore has nonnegative trace. Substituting back into (4.8), we obtain

[TABLE]

For notational consistency with Section 3, we introduce the functions $f$ and $\psi$, given by

[TABLE]

where we recall $\gamma=(\alpha,\sigma,c,\rho)$. Combining (4.6), (4.7) and (4.9), and substituting into (4.5), we then obtain the following representation.

[TABLE]

4.4 Fixing an observation path

Since the parameters $\alpha,\sigma,c$ and $\rho$ are assumed to be absolutely continuous, $R$ is then the $C^{1}$ solution of (4.2), and it follows from integration by parts (see e.g. Theorem 1.2.3 in [36]) that the Itô integral against $Y$ in (4.1) can also be interpreted pathwise as a Riemann–Stieltjes integral. Moreover, these two notions of integral coincide almost surely. This can be seen by noting that the corresponding Riemann sums converge almost surely to the Riemann–Stieltjes integral, but also in $L^{2}$ to the Itô integral, so these two notions of integral must agree by the uniqueness of limits in probability.

In filtering we make inference based on observations of the process $Y$. Thus, it is natural to restrict our attention to a particular path of $Y$, which we denote by $\zeta$. That is, we define $\zeta\colon[0,T]\to\mathbb{R}^{d}$ by

[TABLE]

for some fixed $\omega\in\Omega$. By the previous paragraph, we can then consider the filter dynamics (4.1)–(4.2) with $Y$ replaced by $\zeta$, namely

[TABLE]

Remark 4.2*.*

Strictly speaking, we a priori only have that the solution $(q,R)$ of (4.1)–(4.2) exists almost surely for each choice of parameters $\alpha,\sigma,c,\rho,\mu_{0},\Sigma_{0}$. Here we actually wish to consider this solution for every choice of parameters, for almost every fixed $\omega\in\Omega$. This can be justified by first considering a countable dense collection of parameters, and then appealing to the stability of solutions to Lipschitz SDEs (see e.g. Chapter 16 in [14]). Alternatively, having fixed an (arbitrary continuous) path $\zeta$, one can establish existence and uniqueness of solutions of $\eqref{eq:qdynamics}$ directly for any choice of parameters by a classical Picard iterative argument.

Recall that the representation in (4.10) for the nonlinear expectation involves the stochastic integral of $\psi(q,\gamma)$ against $Y$. Unlike the stochastic integral in (4.1), since the paths of $q$ and $Y$ both have Brownian-type regularity, in general this integral does not exist in the pathwise Riemann–Stieltjes sense. As in the previous section, we instead aim to interpret it as a rough integral. Similarly to the setting of Crisan et al. [15], this requires us to lift the observation process $Y$ into rough path space.

In the previous section we were able to solve optimal control problems where the driving noise was a geometric rough path. However, since Itô integration does not satisfy first order calculus—that is, it does not satisfy the classical integration by parts/chain rule—when enhancements are defined using iterated Itô integrals the resulting rough paths are in general not geometric. It was for this reason that we insisted on transforming the Itô integral in (4.8) into the Stratonovich integral in (4.9). Similarly to Example 3.16, by setting

[TABLE]

we have that, almost surely, $\mathbf{Y}=(Y,Y^{(2)})$ defines a $\frac{1}{p}$-Hölder geometric rough path for any $p\in(2,3)$. Recalling that we defined $\zeta=Y(\omega)$ for a given $\omega\in\Omega$, we can now consider $\zeta$ as a rough path by defining its lift as

[TABLE]

for the same $\omega$.

It remains to establish $\psi(q,\gamma)$ as being controlled (in the sense of Gubinelli) by $\zeta$. The Gubinelli derivative of $q$ with respect to $\zeta$ can be inferred by simply inspecting (4.11). Indeed, recalling the notation $\zeta_{s,t}:=\zeta_{t}-\zeta_{s}$, we have that

[TABLE]

Since $c$ is of bounded variation, it is trivially controlled by $\zeta$ with derivative zero, and we conclude (from e.g. Corollary 7.4 in [22]) that $\psi(q,\gamma)=-cq$ is indeed controlled by $\zeta$ with Gubinelli derivative $\psi(q,\gamma)^{\prime}=-c(Rc^{\top}+\sigma\rho)$. Thus, almost surely,

[TABLE]

exists as a rough integral and, moreover, coincides with the Stratonovich integral in (4.10).

4.5 Reformulation as an optimal control problem

Writing $\gamma=(\alpha,\sigma,c,\rho)$ as usual, consider the functional $\kappa_{t}\colon\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}\to\mathbb{R}$ defined by

[TABLE]

where $q$ and $R$ satisfy (4.11)–(4.12) with the terminal condition $(q_{t},R_{t})=(\mu,\Sigma)$. The function $\kappa_{t}$ is related to the nonlinear expectation (4.10) by the following lemma.

Lemma 4.3**.**

Denote by $\Phi(\hskip 1.0pt\cdot\,;\mu,\Sigma)$ the distribution function of a $N(\mu,\Sigma)$ distribution. For any rough path $\bm{\zeta}=(\zeta,\zeta^{(2)})=\mathbf{Y}(\omega)\in\mathscr{C}^{0,p}_{g}$ as defined above, and any bounded measurable function $\varphi$, we have the equality

[TABLE]

where the expectation on the left-hand side is evaluated on the realisation $Y=\zeta$.

The proof of Lemma 4.3 is the same as that of Proposition 2.1 in [2].

The expression for $\kappa$ in (4.13) looks very much like that of the value function of an optimal control problem with state trajectories governed by (4.11)–(4.12). To make this exact we should write $\kappa$ as an infimum over the ‘control’ $\gamma$ alone. This is easy, but one should note that, for certain choices of control $\gamma$ and terminal condition $(\mu,\Sigma)\in\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}$, there will not actually exist a corresponding initial value $(q_{0},R_{0})\in\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}$. This can happen for one of two reasons. First, due to the term $\sigma_{s}\sigma^{\top}_{s}$, the solution to (4.12) may no longer be positive semi-definite, so that $R_{0}$ does not correspond to a covariance matrix. Second, the solution to (4.12) may ‘blow up’ in finite time, due to the quadratic term (in $R$) in the final term on the right-hand side of (4.12). An example of such behaviour is exhibited in [2, Section 3].

Heuristically, the Kalman–Bucy filter is well behaved when run forwards in time from an initial condition, but here we instead fix a terminal condition and run the filtering equations backwards in time, which introduces the abnormalities described above. To prevent this unphysical behaviour we simply prescribe the value $g(\mu_{0},\Sigma_{0})=\infty$ for any initial value $(\mu_{0},\Sigma_{0})\notin\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}$ and, although we don’t actually obtain a physical initial value for solutions which ‘blow up’ in a finite time, we assign an infinite ‘initial’ cost to all such trajectories.

We can now write

[TABLE]

where $q^{t,\mu,\Sigma,\gamma},R^{t,\Sigma,\gamma}$ satisfy (4.11)–(4.12) with the terminal condition

[TABLE]

noting that trajectories with the undesired behaviour described above will never be considered when taking the infimum in (4.15).

We have derived an optimal control problem, with the controlled dynamics (4.11)–(4.12), and the value function defined in (4.15). Moreover, the appearance of the ‘Brownian-like’ path $\zeta$ in (4.11), and indeed the rough path $\bm{\zeta}$ in (4.15), puts us back into the setting of pathwise stochastic control. In the case where the parameter $c$ is known, the signal and observation noises are uncorrelated (so that $\rho\equiv 0$), and if we omit the likelihood term in the penalty of our nonlinear expectation, then we are not directly controlling the coefficient of the rough term $\zeta$. This was the case in the setting of [2], where a change of variables was then used to completely isolate the observation path from the controlled terms.

In the current setting however we cannot escape the need to control the coefficient of $\zeta$. As described in Section 3, if the variation of the controls $\gamma$ is not sufficiently penalised then the control problem degenerates. The physical interpretation here is the following: even if we suppose that the parameters $\alpha,\sigma,c$ and $\rho$ are able to fluctuate at the same rate as the observation path $\zeta$, it is not reasonable to suppose that we should be able to calibrate these parameters over time scales that are so small that our observations are dominated by measurement noise.

Accordingly, we employ the strategy introduced in the previous section of introducing a regularising cost, and rewriting the problem in terms of an abstract control process $u$. We consider the dynamics (4.11)–(4.12) along with

[TABLE]

for some function $h\colon\Gamma\times U\to U$, where $u$ belongs to the class $\mathcal{U}$ of bounded measurable functions $u\colon[0,T]\to U:=\mathbb{R}^{m\times m}\times\mathbb{R}^{m\times l}\times\mathbb{R}^{d\times m}\times\mathbb{R}^{l\times d}$. The terminal condition is now given by

[TABLE]

Allowing $f$ to depend on $u$, and $g$ to depend on $\gamma_{0}$ (which makes no difference to the proof of Lemma 4.3), and writing $\tilde{\kappa}$ for the regularised version of $\kappa$, we can write

[TABLE]

where

[TABLE]

is the value function of our new control problem. As before, to avoid unphysical trajectories, we assign an infinite cost to any controls $u$ such that $(\mu_{0},\Sigma_{0})\notin\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}$ or such that the solution to (4.12) ‘blows up’ in a finite time. Moreover, we assign an infinite cost to those controls which lead to $\rho$ leaving the space of valid correlation matrices $\Upsilon$ (as defined in (4.4)).

Our uncertainty is thus represented by the function $v$. Once the value of this function has been determined, one can use (4.16) and then (4.14) to evaluate arbitrary functions of the signal process $S$ under the nonlinear expectation $\mathcal{E}(\hskip 1.0pt\cdot\hskip 2.0pt|\mathcal{Y}_{t})$.

4.6 A nonlinear backward control problem

It remains to characterise the value function $v$ defined in (4.17) as the unique solution of a rough HJB equation. For convenience, we rewrite the controlled dynamics in full as

[TABLE]

where $\gamma=(\alpha,\sigma,c,\rho)$, and we define

[TABLE]

We note that this is a ‘backward’ control problem in the sense that, in contrast to the classical setting of optimal control, here we prescribe a terminal condition for the state trajectories, and consider a cost associated with their initial value. More significantly, we note that $b_{\mu},b_{\Sigma}\notin\textrm{{Lip}}_{b}$ and $\lambda,\psi\notin C^{3}_{b}$, so we cannot immediately apply the results of the previous section. Nevertheless, as we will see, the desired results can be recovered with some modifications.

Notation 4.4**.**

In the following we write $|\cdot|$ for the usual Euclidean norm, and $\|A\|$ for the Frobenius norm of a given matrix $A$, i.e. $\|A\|^{2}=\textrm{{tr}}(A^{\top}A)$. Given an element $\gamma=(\alpha,\sigma,c,\rho)$ of $U=\mathbb{R}^{m\times m}\times\mathbb{R}^{m\times l}\times\mathbb{R}^{d\times m}\times\mathbb{R}^{l\times d}$, we write \|\gamma\|=\max\big{\{}\|\alpha\|,\|\sigma\|,\|c\|,\|\rho\|\big{\}}. We point out however that, since the space $\Upsilon$ of correlation matrices is uniformly bounded999Indeed, one can show that $\|\rho\|\leq\sqrt{l}$ for every $\rho\in\Upsilon$., the dependence on $\|\rho\|$ is not particularly crucial.

If $A\in\mathcal{S}_{+}^{m}$, so that in particular $A$ is symmetric and positive definite, we write $\lambda_{\mathrm{min}}(A)$ (resp. $\lambda_{\mathrm{max}}(A)$) for the smallest (resp. largest) eigenvalue of $A$.

Where there is no risk of ambiguity, we will omit the superscripts from the state variables $q,R$ and $\gamma$. Finally, in this section we will use the symbol $\lesssim$ to denote inequality up to a multiplicative constant which may depend on any of the dimensions $d,l,m$, the functions $f,g,h$, the measure of regularity $p$, the terminal time $T$, and the bound $L$, where as usual $L>0$ is chosen such that ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{\frac{1}{p}\textrm{{-H\"{o}l}}}\leq L$.

Assumption 4.5**.**

We assume that

• •

$f=f(q,R,\gamma,u)$ and $g=g(q,R,\gamma)$ are continuous, bounded below, and locally Lipschitz in $(q,R,\gamma)$, uniformly in $u$,

• •

$h=h(\gamma,u)$ is continuous, surjective in $u$, i.e. $\{h(\gamma,u):u\in U\}=U$ for every $\gamma\in\Gamma$, Lipschitz in $\gamma$, uniformly in $u$, and bounded in $\gamma$, locally uniformly in $u$, and moreover, for some $\delta_{1}\geq 1$, satisfies

[TABLE]

• •

for some $\delta_{2}>\delta_{1}$, the running cost $f$ satisfies the asymptotic growth condition:

[TABLE]

as $|q|+\|R\|+\|\gamma\|+\|u\|\to\infty$,

• •

and the initial cost $g$ satisfies:

[TABLE]

If the correlation $\rho$ is known, then one can simply take, for example, $h(\gamma,u)=u$. If $\rho$ is uncertain then, for mostly technical reasons, one must take a little extra care to ensure that correlations close to the boundary of $\Upsilon$ (where $\lambda_{\mathrm{max}}(\rho\rho^{\top})=1$) are sufficiently penalised. In this case we assume in addition that

• •

[TABLE]

[TABLE]

Remark 4.6*.*

The surjectivity of $h$ in $u$ is assumed to ensure that, no matter the choice of terminal condition $(t,\mu,\Sigma,a)$, there always exists a choice of control $u$ such that the state trajectories remain inside their respective domains, so that in particular $R_{0}\in\mathcal{S}_{+}^{m}$ and $\rho_{0}\in\Upsilon$. This guarantees that the value function $v$ is finite-valued.

The result of Lemma 3.11 can be recovered in the current setting as follows.

Lemma 4.7**.**

Under Assumption 4.5, for any terminal condition $(t,\mu,\Sigma,a)$ and control $u$, we have that

[TABLE]

where the constant $C$ depends on $d,l,m,f,g,h,p,T$ and $L$.

Proof.

From the estimate (2.3) in Proposition 2.2, we have that

[TABLE]

We aim to bound each of the terms on the right-hand side. Recalling that $\psi(q,\gamma)=-cq$ and $\psi(q,\gamma)^{\prime}=-c(Rc^{\top}+\sigma\rho)$, we have that

[TABLE]

Writing $(\dot{\alpha},\dot{\sigma},\dot{c},\dot{\rho})=\dot{\gamma}=h(\gamma,u)$, we have

[TABLE]

By the Young–Lóeve inequality (see e.g. Theorem 6.8 in [24]), we have

[TABLE]

for any interval $[h,r]\subset[0,t]$. We calculate

[TABLE]

Recalling (4.11) and using (4.27), we have

[TABLE]

We then obtain

[TABLE]

where the limit in the above is taken over any sequence of partitions of the interval $[0,t]$ with mesh size tending to zero. Substituting (4.25), (4.26) and (4.28) into (4.24), and using the growth conditions (4.18)–(4.20) in Assumption 4.5, we deduce the result. ∎

Corollary 4.8**.**

Let $K$ be a compact subset of $\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}\times\Gamma$. There exists an $M>0$ such that, when taking the infimum over $u\in\mathcal{U}$ in (4.17) for $(t,\mu,\Sigma,a)\in[0,T]\times K$, one may restrict to controls $u$ such that the norms

[TABLE]

are all bounded by $M$.

Proof.

One can obtain a bound for $\|\gamma\|_{1;[0,t]}$ by a similar argument to that in the proof of Corollary 3.12. Since $\gamma_{t}=a$ lives in a compact set, we immediately also have a bound for $\|\gamma\|_{\infty}$.

We infer from (4.20) that both the terminal value $R_{t}=\Sigma$ and initial value $R_{0}$ of $R$ must lie in some bounded set, and by inspecting the ODE (4.12) satisfied by $R$, we deduce that the entire path of $R$ must also live in a bounded set, giving a bound for $\|R\|_{\infty}$.

Given the bounds for $\|\gamma\|_{\infty}$ and $\|R\|_{\infty}$, a bound for $\|R\|_{1;[0,t]}$ follows easily from (4.12). Finally, inspecting the equation (4.11) satisfied by $q$, in view of (4.27), we deduce a bound for $\|q\|_{\infty}$. ∎

As in the previous sections, let us approximate the rough path $\bm{\zeta}$ by a smooth path $\eta$. We then obtain the approximate value function

[TABLE]

Notation 4.9**.**

In the following, we will write $\nabla_{\mu}$ for the usual gradient with respect to $\mu$, and write $\nabla_{\Sigma}$ and $\nabla_{a}$ for the generalised gradients with respect to each of the components of $\Sigma$ and $a$ respectively. We will also write $A:B$ for the inner product of two elements $A,B$ from the same vector space. In particular, when $A,B$ are matrices, $A:B=\textrm{{tr}}(A^{\top}B)$ denotes the Frobenius inner product of $A$ and $B$.

We shall denote by $\mathcal{H}$ the class of functions $\tilde{v}\colon[0,T]\times\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}\times\Gamma\to\mathbb{R}$ which explode asymptotically; that is, those functions $\tilde{v}$ such that

[TABLE]

as $|\mu|+\|\Sigma\|+\|a\|\to\infty$, and as $\lambda_{\mathrm{min}}(\Sigma)\to 0$, and, in the case when $\rho$ is uncertain, as $\lambda_{\mathrm{max}}(\rho\rho^{\top})\to 1$.

Proposition 4.10**.**

Under Assumption 4.5, the approximate value function $v^{\eta}$, as defined in (4.29), is locally Lipschitz continuous with respect to $(t,\mu,\Sigma,a)$, and is the unique viscosity solution of the HJB equation

[TABLE]

in the class $\mathcal{H}$ which satisfies the initial condition $v^{\eta}(0,\mu,\Sigma,a)=g(\mu,\Sigma,a)$.

Proof.

As the path $\eta$ is smooth, the associated PDE (4.30) is classical, except for the nonlinearities inherited from the filtering equations. As the proof of this result is lengthy, and not intended to be the focus of the current work, we will only give a sketch of the proof.

That $v^{\eta}$ is a viscosity solution of (4.30) is a standard application of the dynamic programming principle; we refer to the proof of Proposition 4.9 in [2] for precise details.

Heuristically, as a result of (4.19) and (4.23), for terminal conditions $(\mu,\Sigma,a)$ which take extreme or close to degenerate values, i.e. when either $|\mu|+\|\Sigma\|+\|a\|\gg 1$ or $\lambda_{\mathrm{min}}(\Sigma)\approx 0$ or $\lambda_{\mathrm{max}}(\rho\rho^{\top})\approx 1$, it takes very expensive controls to allow the state trajectories $(q,R,\gamma)$ to escape these parts of their domain. It then follows from the growth conditions (4.20)–(4.22) that the value function itself must explode as one approaches these extreme and degenerate values; that is, $v^{\eta}\in\mathcal{H}$.

One can prove that $v^{\eta}$ is locally Lipschitz in all of its arguments by adapting the proof of Theorem 2.2 in [5], which in particular requires the strict inequality $\delta_{2}>\delta_{1}$ in Assumption 4.5.

The controlled dynamics do not satisfy the standard Lipschitz condition which would be required to be able to apply a standard uniqueness result for Hamilton–Jacobi equations on unbounded domains, as in e.g. Yong and Zhou [40, Chapter 4]. Nevertheless, uniqueness for an equation of the same form as (4.30) was established in [2, Section 5], and an analogous argument may be used here. The main insight of this result is that the extra condition one should impose to obtain uniqueness is that solutions belong to the space $\mathcal{H}$; that is, one should restrict to solutions which explode as they approach the boundary. ∎

The main result of this section is given by the following theorem.

Theorem 4.11**.**

Under Assumption 4.5, the value function $v$, as defined in (4.17), solves the rough HJB equation

[TABLE]

with

[TABLE]

in the sense of Definition 3.13. Moreover, writing $v=v^{\zeta}$, the map from $\mathscr{C}^{0,p}_{g}([0,T];\mathbb{R}^{d})\to\mathbb{R}$ given by $\bm{\zeta}\mapsto v^{\zeta}(t,\mu,\Sigma,a)$ is locally uniformly continuous with respect to each of the rough path metrics $\varrho_{p}$ and $\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}$, locally uniformly in $(t,\mu,\Sigma,a)$.

Proof.

Let $K$ be a compact subset of $\mathbb{R}^{m}\times\mathcal{S}_{+}^{m}\times\Gamma$ and let $\bm{\eta}\in\mathscr{C}^{p}$ be another rough path such that $\varrho_{\frac{1}{p}\textrm{{-H\"{o}l}}}(\bm{\eta},\bm{\zeta})\leq 1$. By possibly replacing $L$ by $L+1$, we may assume that ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\eta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{\frac{1}{p}\textrm{{-H\"{o}l}}}\leq L$. Let us write $q^{\eta}$ (resp. $q^{\zeta}$) for the solution of (4.11) driven by $\eta$ (resp. $\zeta$), and write $v^{\eta}$ (resp. $v^{\zeta}$) for the corresponding value function, as defined in (4.17).

By Corollary 4.8, there exists a constant $M>0$ such that, for terminal conditions $(t,\mu,\Sigma,a)\in[0,T]\times K$, we may restrict to controls $u\in\mathcal{U}^{M}\subseteq\mathcal{U}$ such that

[TABLE]

are all bounded by $M$. In the following we will allow the multiplicative constant indicated by the symbol $\lesssim$ to also depend on $M$.

By the Young–Lóeve inequality (see e.g. Theorem 6.8 in [24]), we have

[TABLE]

from which we deduce that

[TABLE]

for all $s\in[0,t]$, and thus, by Grönwall’s inequality, that

[TABLE]

Since the state variables $q^{\eta},q^{\zeta},R$ and $\gamma$ are uniformly bounded, we are free to modify the coefficients $b_{\mu},\lambda$ and $\psi$ outside of some large ball containing the domain of the state variables in its interior, without affecting the solutions $q^{\eta},q^{\zeta}$. We may therefore pretend that actually $b_{\mu}\in\textrm{{Lip}}_{b}$ and $\lambda,\psi\in C^{3}_{b}$, so that in particular the hypotheses of Proposition 2.6 are satisfied. By the same argument, we may also suppose that $f$ and $g$ are Lipschitz in $q$.

By Proposition 2.6 combined with (4.31), we obtain

[TABLE]

Using (4.31) and (4.32), we have, for any terminal condition $(t,\mu,\Sigma,a)\in[0,T]\times K$, that

[TABLE]

and we conclude as we did in the proof of Theorem 3.14. ∎

Remark 4.12*.*

As we have seen, in order to prevent degeneracy of the control problem it is necessary to control the derivative of the parameters, rather than controlling them directly. This allows us to calibrate, not only beliefs about reasonable values the parameters could take, but also at what rate they should able to vary. For example, if one believes that the true parameters should remain fairly constant then one can put a large penalty on the magnitude of this derivative. In fact, by taking the penalty to be infinite for all non-zero controls (derivatives), we obtain a setting with unknown parameters which are constant in time. The discrete-time results of [13] suggest that we should then expect the resulting filter to converge to the true parameter. (Although our observations are not independent and identically distributed as in [13], under reasonable conditions they are ergodic, and this leads to consistency properties in the likelihood function; see [19]. In this case, we expect that this would lead to the nonlinear expectation asymptotically converging to the ‘true’ expectation, and the analysis of [13] further suggests an interpretation of the nonlinear expectation in terms of confidence intervals.) Establishing precise convergence results could be the subject of future research.

Acknowledgements  A.L. Allan was supported by the Engineering and Physical Sciences Research Council [EP/L015811/1]. S.N. Cohen was supported by the Oxford-Man Institute for Quantitative Finance and the Oxford-Nie Financial Big Data Laboratory. The authors would also like to thank David J. Prömel for helpful discussions.

Appendix A Rough path estimates

Before establishing existence of solutions to the RDE (2.4), we recall some useful estimates from Friz and Zhang [25].

Lemma A.1** (Lemma 3.6 in [25]).**

Let $\psi\in C^{3}_{b}$, $\gamma\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$ and $\bm{\zeta}\in\mathscr{C}^{p}$ with ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{p}\leq L$. Let $(X,X^{\prime})\in\mathscr{D}^{p}_{\zeta}$. Then

[TABLE]

is a controlled rough path, and we have

[TABLE]

where the constant $C$ depends on $\psi,p$ and $L$.

The following lemma is a direct consequence of Lemma 3.4 in [25].

Lemma A.2**.**

Let $\psi\in C^{2}_{b}$, $\gamma,\vartheta\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$ and $\bm{\eta},\bm{\zeta}\in\mathscr{C}^{p}$ with ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\eta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{p},{\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{p}\leq L$. Let $(X,X^{\prime})\in\mathscr{D}^{p}_{\eta}$ and $(Y,Y^{\prime})\in\mathscr{D}^{p}_{\zeta}$. For any $\delta\geq 1$, we have the following estimate

[TABLE]

where the constant $C$ depends on $p$ and $L$.

Lemma A.3** (Lemma 3.5 in [25]).**

Let $\psi\in C^{3}_{b}$, $\gamma,\vartheta\in\mathcal{C}^{\frac{p}{2}\textrm{{-var}}}$ and $\bm{\eta},\bm{\zeta}\in\mathscr{C}^{p}$ with ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\eta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{p}$, ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{p}\leq L$. Let $(X,X^{\prime})\in\mathscr{D}^{p}_{\eta}$ and $(Y,Y^{\prime})\in\mathscr{D}^{p}_{\zeta}$. Suppose that

[TABLE]

and $\|\gamma\|_{\frac{p}{2}},\|\vartheta\|_{\frac{p}{2}}\leq M$ for some $M>0$. Then we have

[TABLE]

[TABLE]

where the constant $C$ depends on $\psi,p,L$ and $M$.

Proof of Theorem 2.5.

The following argument is adapted from the proof of Theorem 3.8 in [25]. Let $L>0$ be such that ${\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{p}\leq L$. We define a map $\mathcal{M}^{\gamma}_{T}\colon\mathscr{D}^{p}_{\zeta}\to\mathscr{D}^{p}_{\zeta}$ by

[TABLE]

We will show that this map has a unique fixed point. For $\delta\geq 1$, we define the ball

[TABLE]

where

[TABLE]

We will show that, for a suitable choice of $\delta$ and for $T$ sufficiently small, $\mathcal{M}^{\gamma}_{T}$ leaves $\mathcal{B}^{(\delta)}_{T}$ invariant, and then that it is a contraction on $\mathcal{B}^{(\delta)}_{T}$.

By Lemma A.1, any $(X,X^{\prime})\in\mathcal{B}^{(\delta)}_{T}$ satisfies

[TABLE]

for some constant $C_{1}\geq\frac{1}{2}$ depending only on $b,\lambda,p,L$ and $\|\gamma\|_{\frac{p}{2}}$. Let $\delta=\delta_{1}:=2C_{1}\geq 1$, so that

[TABLE]

Hence, taking $T=T_{1}$ sufficiently small, we can ensure that $\|\mathcal{M}^{\gamma}_{T_{1}}(X,X^{\prime})\|_{\zeta,p}^{(\delta_{1})}\leq 1$, so that $\mathcal{M}^{\gamma}_{T_{1}}(X,X^{\prime})\in\mathcal{B}^{(\delta_{1})}_{T_{1}}$. That is, $\mathcal{B}^{(\delta_{1})}_{T_{1}}$ is invariant under $\mathcal{M}^{\gamma}_{T_{1}}$.

Let $(X,X^{\prime}),(Y,Y^{\prime})\in\mathcal{B}^{(\delta_{1})}_{T}$ for some $T\leq T_{1}$. For any (new) $\delta\geq 1$ we have

[TABLE]

for some constant $C$ depending on $b,p$ and $L$, where we used the result of Lemma A.2 to obtain the last line.

We can take $M>0$, dependent only on $\lambda$ and $\|\gamma\|_{\frac{p}{2}}$, sufficiently large such that

[TABLE]

for all $(X,X^{\prime})\in\mathcal{B}^{(\delta_{1})}_{T}$. Noting that $\|X-Y\|_{\infty}\leq\|R^{X}-R^{Y}\|_{\frac{p}{2}}$ and applying the estimates in Lemma A.3, we then deduce that

[TABLE]

for a new constant $C_{2}>\frac{1}{2}$ which depends only on $b,\lambda,p,L$ and $M$. Let $\delta=\delta_{2}:=2C_{2}>1$. We can then choose $T=T_{2}\leq T_{1}$ sufficiently small such that $C_{2}\delta_{2}({\left|\kern-1.07639pt\left|\kern-1.07639pt\left|\bm{\zeta}\right|\kern-1.07639pt\right|\kern-1.07639pt\right|}_{p;[0,T_{2}]}+T_{2})\leq\frac{1}{2}$. We then have that

[TABLE]

which establishes the contraction property for $\mathcal{M}^{\gamma}_{T_{2}}$.

It follows that there exists a unique fixed point $(X,X^{\prime})\in\mathscr{D}_{\zeta}^{p}$ of the map $\mathcal{M}^{\gamma}_{T_{2}}$, which is then the unique solution of (2.10) in $\mathscr{D}_{\zeta}^{p}$ satisfying $X^{\prime}=\lambda(X,\gamma)$ over the time interval $[0,T_{2}]$. Noting that the time $T_{2}$ was chosen independently of the initial values $x$, $\gamma_{0}$, we may then simply paste solutions together to obtain a unique solution over the entire interval $[0,T]$ for any given $T>0$. ∎

Proof of Proposition 2.6.

Since $\|\gamma\|_{\frac{p}{2}}$ is bounded by $M$, it follows from Proposition 2.4 and the fact that $X^{\prime}=\lambda(X,\gamma)$, that there exists an $\tilde{M}>0$, depending on $b,\lambda,p,T,L$ and $M$, such that the norms

[TABLE]

and the same with $X$ and $\gamma$ replaced by $Y$ and $\vartheta$, are all bounded by $\tilde{M}$. In particular we note that the hypotheses of Lemma A.3 are satisfied. In the following the symbol $\lesssim$ will denote inequality up to a multiplicative constant which may depend on $b,\lambda,\psi,p,T,L$ and $\tilde{M}$.

For any $\delta\geq 1$, we have

[TABLE]

Since the drift $b$ is Lipschitz, it is easy to see that

[TABLE]

As $(X,X^{\prime})=(X,\lambda(X,\gamma))\in\mathscr{D}^{p}_{\eta}$ and $(Y,Y^{\prime})=(Y,\lambda(Y,\vartheta))\in\mathscr{D}^{p}_{\zeta}$, we have that

[TABLE]

Combining the results of Lemmas A.2 and A.3, we then deduce that

[TABLE]

for some constant $C_{0}>\frac{1}{2}$ which depends on $b,\lambda,p,T,L$ and $\tilde{M}$.

Let $\delta=\delta_{0}:=2C_{0}>1$. We can then take $T=T_{0}$ (depending only on $p,L$ and $C_{0}$) sufficiently small such that

[TABLE]

so that, after rearranging, we obtain

[TABLE]

It follows that the estimate in (2.11) holds over any time interval of length $T_{0}$. One can then extend this estimate to hold over the union of any finite number of such intervals (with a correspondingly larger constant $C$) by pasting via Lemma 2.3.

The bound in (A.1) also holds with $X$ and $Y$ replaced with $\int_{0}^{\cdot}\psi(X_{s},\gamma_{s})\,\mathrm{d}\bm{\eta}_{s}$ and $\int_{0}^{\cdot}\psi(Y_{s},\vartheta_{s})\,\mathrm{d}\bm{\zeta}_{s}$ respectively, so that

[TABLE]

Applying again the results of Lemmas A.2 and A.3, this time with $\delta=1$, we deduce that

[TABLE]

Combining this with (2.11), we obtain (2.12). ∎