On the compensator in the Doob-Meyer decomposition of the Snell envelope

TL;DR

This paper investigates the structure of the Snell envelope in optimal stopping problems, establishing conditions for its finite-variation part, linking it to the generator of the underlying process, and applying these results to control problems and smooth pasting.

Contribution

It provides new insights into the compensator in the Doob-Meyer decomposition of the Snell envelope, especially in the Markovian setting, and connects it to stochastic control and smooth pasting conditions.

Findings

Finite-variation part of the Snell envelope is absolutely continuous w.r.t. the decreasing part of the original process.

Conditions identified for the value function to be in the domain of the extended generator.

Dual problem of optimal stopping is characterized as a stochastic control problem with explicit control functions.

Abstract

Let $G$ be a semimartingale, and $S$ its Snell envelope. Under the assumption that $G\in\mathcal{H}^1$, we show that the finite-variation part of $S$ is absolutely continuous with respect to the decreasing part of the finite-variation part of $G$. In the Markovian setting, this enables us to identify sufficient conditions for the value function of the optimal stopping problem to belong to the domain of the extended (martingale) generator of the underlying Markov process. We then show that the \textit{dual} of the optimal stopping problem is a stochastic control problem for a controlled Markov process, and the optimal control is characterised by a function belonging to the domain of the martingale generator. Finally, we give an application to the smooth pasting condition.

Equations

$$S$$

$$\mathbb{E}\Big{[}\sup_{0\leq t\leq T}\lvert G_{t}\lvert\Big{]}<\infty,$$

$$\bar{\mathbb{G}}\text{ be the space of all adapted, caˋdlaˋg processes such that eqrefsup holds.}$$

$$v(0):=\underset{\tau \in {\mathcal{T}}_{0,T}}{\sup }\mathbb{E}[G_{\tau }].$$

$$\hat{v}(0):=\sup_{\tau\in\mathcal{T}_{0,T}}\mathbb{E}[N_{\tau}+G_{\tau}]=\mathbb{E}\Big{[}\sup_{0\leq t\leq T}\lvert G_{t}\lvert\Big{]}+\sup_{\tau\in\mathcal{T}_{0,T}}\mathbb{E}[G_{\tau}],$$

$$v(\sigma ):=\underset{\tau \in {\mathcal{T}}_{\sigma ,T}}{\mathrm{esssup}}\mathbb{E}[G_{\tau }|{\mathcal{F}}_{\sigma }],\sigma \in {\mathcal{T}}_{0,T}.$$

$$\lvert S_{t}\lvert\leq\mathbb{E}\Big{[}\sup_{0\leq s\leq T}\lvert G_{s}\lvert\Big{\lvert}\mathcal{F}_{t}\Big{]}:=N_{t}\quad\mathbb{P}\textrm{-a.s.}$$

$$S=M^{\ast }-A,$$

$$G=N+D,$$

$$\mathbb{G}\text{ be the space of semimartingales in }\bar{\mathbb{G}}.$$

$$\text{(Feller)}\subset \text{(Hunt)}\subset \text{(right)}$$

$$M_t^f:=f(X_t)-f(x)-{\int }_0^{t}h(X_s)ds,t\ge 0,x\in E,$$

$$X_{\infty }=\Delta ,G_{\infty }^{\alpha }=\underset{t\to \infty }{\mathrm{liminf}}{G}_t^{\alpha },g(\Delta )=G_{\infty }^0.$$

$$g\in \mathcal{Y}\text{, given that }g(\cdot )\text{ is }{\mathcal{E}}^e\text{-measurable and }{G}^{\alpha }\text{ is of class (D).}$$

$$V(x)=\underset{\tau \in {\mathcal{T}}_{0,T}(\mathcal{G})}{\sup }{\mathbb{E}}_x[e^{-\alpha \tau }g(X_{\tau })],x\in E.$$

$$V(x)=\underset{\tau \in {\mathcal{T}}_{0,T}(\mathcal{F})}{\sup }{\mathbb{E}}_x[e^{-\alpha \tau }g(X_{\tau })],x\in E,$$

$$e^{-\alpha \tau }V(X_{\tau })=\underset{\sigma \in {\mathcal{T}}_{\tau ,T}(\mathcal{F})}{\mathrm{esssup}}{\mathbb{E}}_x[G_{\sigma }^{\alpha }|{\mathcal{F}}_{\tau }]{\mathbb{P}}_x\text{-a.s.}$$

$$\{F_{t+s}\in A\}\in \sigma (F_s)\cup \sigma ({\theta }_s\circ X_u:0\le u\le t),\text{for all }A\in {\mathcal{E}}^{\prime }.$$

$$Y_{t}=\Bigg{(}X_{t},L^{0}_{t},\sup_{0\leq s\leq t}X_{s},\int^{t}_{0}\exp(-\int^{s}_{0}\alpha(X_{u})du)f(X_{s})ds\Bigg{)},\quad t\geq 0,$$

$$H^{\ast }=\underset{0\le t\le T}{\sup }|H_t|$$

$$\lvert\lvert H\lvert\lvert_{\mathcal{S}^{p}}=\lvert\lvert H^{*}\lvert\lvert_{L^{p}}:=\mathbb{E}\big{[}\lvert H^{*}\lvert^{p}\big{]}^{1/p},\quad 1\leq p\leq\infty.$$

$$\lvert\lvert X\lvert\lvert_{\mathcal{H}^{p}}=\lvert\lvert\bar{B}\lvert\lvert_{\mathcal{S}^{p}}+\Big{\lvert}\Big{\lvert}\int^{T}_{0}\lvert d\bar{I}_{s}\lvert\Big{\lvert}\Big{\lvert}_{L^{p}},$$

$$G=N+D,$$

$${\mu }_t:=\frac{d{A}_t}{d{D}_t^{-}},0\le t\le T,$$

$$A_t^n$$

$$A_T^n$$

$$\mathbb{E}[G_{\tau }]\ge \underset{n\to \infty }{\mathrm{limsup}}\mathbb{E}[G_{{\tau }_n}].$$

$$\frac{1}{L}\Big{(}\sum^{L}_{l=1}A^{n_{l}}_{t}\Big{)}\to A_{t},\quad\text{a.s. and in }L^{1}.$$

$$0\le A_t^{{\sigma }_n}-A_s^{{\sigma }_n}\le (D^{-}{)}_t^{{\sigma }_n}-(D^{-}{)}_s^{{\sigma }_n}\text{ a.s.},$$

$$\mathbb{E}[S_{t+ϵ}|{\mathcal{F}}_t]$$

Full text

Keywords: Doob-Meyer decomposition, optimal stopping, Snell envelope stochastic control, martingale duality, smooth pasting.

Mathematics Subject Classification: 60G40, 60G44, 60J25, 60G07, 93E20.

1 Introduction

Given a (gains) process $G=(G_{t})_{t\geq 0}$, living on the usual filtered probability space $(\Omega,\mathcal{F},\mathbb{F}=(\mathcal{F}_{t})_{t\geq 0},\mathbb{P})$, the classical optimal stopping problem is to find a maximal reward $v(0)=\sup_{\tau\geq 0}\mathbb{E}[G_{\tau}]$, where the supremum is taken over all $\mathbb{F}$ - stopping times. In order to compute $v(0)$, we consider, for each $\mathbb{F}$ - stopping time $\sigma\geq 0$, the value function $v(\sigma)=\operatorname*{ess\,sup}_{\tau\geq\sigma}\mathbb{E}[G_{\tau}\lvert\mathcal{F}_{\sigma}]$. It is, or should be, well-known (see, for example, El Karoui [16], Karatzas and Shreve [31]) that under suitable integrability and regularity conditions on the process $G$, the Snell envelope of $G$, denoted by $S=(S_{t})_{t\geq 0}$, is the minimal supermartingale which dominates $G$ and aggregates the value function $v$, so that for any $\mathbb{F}$ - stopping time $\sigma\geq 0$, $S_{\sigma}=v(\sigma)$ almost surely. Moreover, $\tau_{\sigma}:=\inf\{r\geq\sigma:S_{r}=G_{r}\}$ is the minimal optimal stopping time, so, in particular, $S_{\sigma}=v(\sigma)=\mathbb{E}[G_{\tau_{\sigma}}\lvert\mathcal{F}_{\sigma}]$ almost surely. A successful construction of the process $S$ leads, therefore, to the solution of the initial optimal stopping problem.

In the Markovian setting the gains process takes the form $G=g(X)$, where $g(\cdot)$ is some payoff function applied to an underlying Markov process $X$. Under very general conditions, the Snell envelope is then characterised as the least super-mean-valued function $V(\cdot)$ that majorizes $g(\cdot)$. A standard technique to find the value function $V(\cdot)$ is to solve the corresponding obstacle (free-boundary) problem. For an exposition of the general theory of optimal stopping in both settings we also refer to Peskir and Shiryaev [39].

The main aim of this paper is to answer the following canonical question of interest:

Question**.**

When does the value function $V(\cdot)$ belong to the domain of the extended (martingale) generator of the underlying Markov process $X$?

Very surprisingly, given how long general optimal stopping problems have been studied (see Snell [49]), we have been unable to find any general results about this.

As the title suggests, we tackle the question by considering the optimal stopping problem in a more general (semimartingale) setting first. If a gains process $G$ is sufficiently integrable, then $S$ is of class (D) and thus uniquely decomposes into the difference of a uniformly integrable martingale, say $M$, and a predictable, increasing process, say $A$, of integrable variation. From the general theory of optimal stopping it can be shown that $\bar{\tau}_{\sigma}:=\inf\{r\geq\sigma:A_{r}>0\}$ is the maximal optimal stopping time, while the stopped process $S^{\bar{\tau}_{\sigma}}=(S_{t\wedge\bar{\tau}_{\sigma}})_{t\geq 0}$ is a martingale. Therefore, the finite variation part of $S$, $A$, must be zero up to $\bar{\tau}_{\sigma}$. Now suppose that $G$ is a semimartingale itself. Then its finite variation part can be further decomposed into the sum of increasing and decreasing processes that are, as random measures, mutually singular. Off the support of the decreasing one, $G$ is (locally) a submartingale, and thus in this case it is suboptimal to stop, and we again expect $S$ to be (locally) a martingale. This also suggests that $A$ increases only if the decreasing component of the finite variation part of $G$ decreases. In particular, we prove the following fundamental result (see Theorem 3.3):

[TABLE]

This being a very natural conjecture, it is not surprising that some variants of it have already been considered. As a helpful referee pointed out to us, several versions of Theorem 3.3 were established in the literature on reflected BSDEs under various assumptions on the gains process, see El Karoui et al. [17] ($G$ is a continuous semimartingale), Crepéy and Matoussi [9] ($G$ is a càdlàg quasi-martingale), Hamadéne and Ouknine [23] ($G$ is a limiting process of a sequence of sufficiently regular semimartingales). We note that these results (except Hamadéne and Ouknine [23], where the assumed regularity of $G$ is exploited) are proved essentially by using (or appropriately extending) the related (but different) result established in Jacka [27]. There, under the assumption that $S$ and $G$ are both continuous and sufficiently integrable semimartingales, the author shows that a local time of $S-G$ at zero is absolutely continuous with respect to the decreasing part of the finite-variation process in the decomposition of $G$. Our proof of Theorem 3.3 relies on the classical methods establishing the Doob-Meyer decomposition of a supermartingale.

The first part of Section 3 is devoted to the groundwork necessary to establish Theorem 3.3. It turns out that an answer to the motivating question of this paper then follows naturally. In particular, in the second part of Section 3, in Theorem 3.11, we show that, under very general assumptions on the underlying Markov process $X$, if the payoff function $g(\cdot)$ belongs to the domain of the martingale generator of $X$, so does the value function $V(\cdot)$ of the optimal stopping problem.

In Section 4 we discuss some applications. First, we consider a dual approach to optimal stopping problems due to Davis and Karatzas [10] (see also Rogers [43], and Haugh and Kogan [24]). In particular, from the absolute continuity result announced above, it follows that the dual is a stochastic control problem for a controlled Markov process, which opens the doors to the application of all the available theory related to such problems (see Fleming and Soner [19]). Secondly, if the value function of the optimal stoping problem belongs to the domain of the martingale generator, under a few additional (but general) assumptions, we also show that the celebrated smooth fit principle holds for (killed) one-dimensional diffusions.

2 Preliminaries

2.1 General framework

Fix a time horizon $T\in(0,\infty]$. Let $G$ be an adapted, càdlàg gains process on $(\Omega,\mathcal{F},\mathbb{F}=(\mathcal{F}_{t})_{0\leq t\leq T},\mathbb{P})$, where $\mathbb{F}$ is a right-continuous and complete filtration (augmented by the null sets of $\mathcal{F}=\mathcal{F}_{T}$). We suppose that $\mathcal{F}_{0}$ is trivial. In the case $T=\infty$, we interpret \mathcal{F}_{\infty}=\sigma\Big{(}\cup_{0\leq t<\infty}\mathcal{F}_{t}\Big{)} and $G_{\infty}=\liminf_{t\to\infty}G_{t}$. For two $\mathbb{F}$-stopping times $\sigma_{1}$, $\sigma_{1}$ with $\sigma_{1}\leq\sigma_{2}$ $\mathbb{P}$-a.s., by $\mathcal{T}_{\sigma_{1},\sigma_{2}}$ we denote the set of all $\mathbb{F}$-stopping times $\tau$ such that $\mathbb{P}(\sigma_{1}\leq\tau\leq\sigma_{2})=1$. We will assume that the following condition is satisfied:

[TABLE]

and let

[TABLE]

The optimal stopping problem is to compute the maximal expected reward

[TABLE]

Remark*.*

First note that by (2.1), $\mathbb{E}[G_{\tau}]<\infty$ for all $\tau\in\mathcal{T}_{0,T}$, and thus $v(0)$ is finite. Moreover, most of the general results regarding optimal stopping problems are proved under the assumption that $G$ is a non-negative (hence the gains) process. However, under (2.1), $N=(N_{t})_{0\leq t\leq T}$ given by $N_{t}=\mathbb{E}[\sup_{0\leq s\leq T}\lvert G_{s}\lvert\lvert\mathcal{F}_{t}]$ is a uniformly integrable martingale, while $\hat{G}:=N+G$ defines a non-negative process (even if $G$ is allowed to take negative values). Then

[TABLE]

and finding $\hat{v}(0)$ is the same as finding $v(0)$. Hence we may, and shall, assume without loss of generality that $G\geq 0$.

The key to our study is provided by the family $\{v(\sigma)\}_{\sigma\in\mathcal{T}_{0,T}}$ of random variables

[TABLE]

Note that, since each deterministic time $t\in[0,T]$ is also a stopping time, (2.3) defines an adapted value process $(v_{t})_{0\leq t\leq T}$ with $v_{t}=v(t)$. We begin with a fundamental result characterising the so-called Snell envelope process, $S=(S_{t})_{0\leq t\leq T}$, of $G$. In particular, $S$ is a version of $(v_{t})_{0\leq t\leq T}$ that aggregates the value function $v(\cdot)$ at each stopping time $\sigma\in\mathcal{T}_{0,T}$ (see Appendix D in Karatzas and Shreve [31]).

Theorem 2.1** (Characterisation of $S$).**

Let $G\in\bar{\mathbb{G}}$. The Snell envelope process $S$ of $G$ satisfies $S_{\sigma}=v(\sigma)$ $\mathbb{P}$-a.s., $\sigma\in\mathcal{T}_{0,T}$, and is the minimal càdlàg supermartingale that dominates $G$.

For the proof of Theorem 2.1 under slightly more general assumptions on the gains process $G$ consult Appendix I in Dellacherie and Meyer [12] or Proposition 2.26 in El Karoui [16].

If $G\in\bar{\mathbb{G}}$, it is clear that $G$ is a uniformly integrable process. In particular, it is also of class (D), i.e. the family of random variables $\{G_{\tau}\mathbbm{1}_{\{\tau<\infty\}}:\tau\textrm{ is a stopping time}\}$ is uniformly integrable. On the other hand, a right-continuous adapted process $Z$ belongs to the class (D) if there exists a uniformly integrable martingale $\hat{N}$, such that, for all $t\in[0,T]$, $\lvert Z_{t}\lvert\leq\hat{N}_{t}$ $\mathbb{P}$-a.s. (see e.g. Dellacherie and Meyer [12], Appendix I and references therein). In our case, by the definition of $S$ and using the conditional version of Jensen’s inequality, for $t\in[0,T]$, we have

[TABLE]

But, since $G\in\bar{\mathbb{G}}$, ${N}$ is a uniformly integrable martingale, which proves the following

Lemma 2.2**.**

Suppose $G\in\bar{\mathbb{G}}$. Then $S$ is of class (D).

Let $\mathcal{M}_{0}$ denote the set of right-continuous martingales started at zero. Let $\mathcal{M}_{0,loc}$ and $\mathcal{M}_{0,UI}$ denote the spaces of local and uniformly integrable martingales (started at zero), respectively. Similarly, the adapted processes of finite and integrable variation will be denoted by $FV$ and $IV$, respectively.

It is well-known that a right-continuous (local) supermartingale $P$ has a unique decomposition $P=B-I$ where $B\in\mathcal{M}_{0,loc}$ and $I$ is an increasing ($FV$) process which is predictable. This can be regarded as the general Doob-Meyer decomposition of a supermartingale. Specialising to class (D) supermartingales we have a stronger result (this is a consequence of, for example, Protter [40] Theorem 16, p.116 and Theorem 11, p.112):

Theorem 2.3** (Doob-Meyer decomposition).**

Let $G\in\bar{\mathbb{G}}$. Then the Snell envelope process $S$ admits a unique decomposition

[TABLE]

where $M^{*}\in\mathcal{M}_{0,UI}$, and $A$ is a predictable, increasing $IV$ process.

Remark*.*

It is normal to assume that the process $A$ in the Doob-Meyer decomposition of $S$ is started at zero. The duality result alluded to in the introduction is one reason why we do not do so here.

An immediate consequence of Theorem 2.3 is that $S$ is a semimartingale. In addition, we also assume that $G$ is a semimartingale with the following decomposition:

[TABLE]

where $N\in\mathcal{M}_{0,loc}$ and $D$ is a $FV$ process. Unfortunately, the decomposition (2.5) is not, in general, unique. On the other hand, uniqueness is obtained by requiring the $FV$ term to also be predictable, at the cost of restricting only to locally integrable processes. If there exists a decomposition of a semimartingale $X$ with a predictable $FV$ process, then we say that $X$ is $special$. For a special semimartingale we always choose to work with its $canonical$ decomposition (so that a $FV$ process is predictable). Let

[TABLE]

Lemma 2.4**.**

Suppose $G\in\mathbb{G}$. Then $G$ is a special semimartingale.

See Theorems 36 and 37 (p.132) in Protter [40] for the proof.

The following lemma provides a further decomposition of a semimartingale (see Proposition 3.3 (p.27) in Jacod and Shiryaev [28]). In particular, the $FV$ term of a special semimartingale can be uniquely (up to initial values) decomposed in a predictable way, into the difference of two increasing, mutually singular $FV$ processes.

Lemma 2.5**.**

Suppose that $K$ is a càdlàg, adapted process such that $K\in FV$. Then there exists a unique pair $(K^{+},K^{-})$ of adapted increasing processes such that $K-K_{0}=K^{+}-K^{-}$ and $\int\lvert dK_{s}\lvert=K^{+}+K^{-}$. Moreover, if $K$ is predictable, then $K^{+}$, $K^{-}$ and $\int\lvert dK_{s}\lvert$ are also predictable.

2.2 Markovian setting

The Markov process

Let $(E,\mathcal{E})$ be a metrizable Lusin space endowed with the $\sigma$-field of Borel subsets of $E$. Let $X=(\Omega,\mathcal{G},\mathcal{G}_{t},X_{t},\theta_{t},\mathbb{P}_{x}:x\in E,t\in\mathbb{R}_{+})$ be a Markov process taking values in $(E,\mathcal{E})$. We assume that a sample space $\Omega$ is such that the usual semi-group of shift operators $(\theta_{t})_{t\geq 0}$ is well-defined (which is the case, for example, if $\Omega=E^{[0,\infty)}$ is the canonical path space). If the corresponding semigroup of $X$, $(P_{t})$, is the primary object of study, then we say that $X$ is a realisation of a Markov semigroup $(P_{t})$. In the case of $(P_{t})$ being sub-Markovian, i.e. $P_{t}1_{E}\leq 1_{E}$, we extend it to a Markovian semigroup over $E^{\Delta}=E\cup\{\Delta\}$, where $\Delta$ is a coffin-state. We also denote by $\mathcal{C}(X)=(\Omega,\mathcal{F},\mathcal{F}_{t},X_{t},\theta_{t},\mathbb{P}_{x}:x\in E,t\in\mathbb{R}_{+})$ the canonical realisation associated with $X$, defined on $\Omega$ with the filtration $(\mathcal{F}_{t})$ deduced from $\mathcal{F}^{0}_{t}=\sigma(X_{s}:s\leq t)$ by standard regularisation procedures (completeness and right-continuity).

In this paper our standing assumption is that the underlying Markov process $X$ is a right process (consult Getoor [20], Sharpe [46] for the general theory). Essentially, right processes are the processes satisfying Meyer’s regularity hypotheses (hypothèses droites) HD1 and HD2. If a given Markov semigroup $(P_{t})$ satisfies HD1 and $\mu$ is an arbitrary probability measure on $(E,\mathcal{E})$, then there exists a homogeneous $E$-valued Markov process $X$ with transition semigroup $(P_{t})$ and initial law $\mu$. Moreover, a realisation of such $(P_{t})$ is right-continuous (Sharpe [46], Theorem 2.7). Under the second fundamental hypothesis, HD2, $t\to f(X_{t})$ is right-continuous for every $\alpha$-excessive function $f$. Recall, for $\alpha>0$, a universally measurable function $f:E\to\mathbb{R}$ is $\alpha$-super-median if $e^{-\alpha t}P_{t}f\leq f$ for all $t\geq 0$, and $\alpha$-excessive if it is $\alpha$-super-median and $e^{-\alpha t}P_{t}f\to f$ as $t\to 0$. If $(P_{t})$ satisfies HD1 and HD2 then the corresponding realisation $X$ is strong Markov (Getoor [20], Theorem 9.4 and Blumenthal and Getoor [7], Theorem 8.11).

Remark*.*

One has the following inclusions among classes of Markov processes:

[TABLE]

Let $\mathcal{L}$ be a given extended infinitesimal (martingale) generator of $X$ with a domain $\mathbb{D}(\mathcal{L})$, i.e. we say a Borel function $f:E\to\mathbb{R}$ belongs to $\mathbb{D}(\mathcal{L})$ if there exists a Borel function $h:E\to\mathbb{R}$, such that $\int^{t}_{0}\lvert h(X_{s})\lvert ds<\infty$, $\forall t\geq 0$, $\mathbb{P}_{x}$-a.s. for each $x$ and the process $M^{f}=(M^{f}_{t})_{t\geq 0}$, given by

[TABLE]

is a local martingale under each $\mathbb{P}_{x}$ (see Revuz and Yor [42] p.285), and then we write $h=\mathcal{L}f$.

Remark*.*

Note that if $A\in\mathcal{E}$ and $\mathbb{P}_{x}(\lambda(\{t:\;X_{t}\in A\}=0)=1$ for each $x\in E$, where $\lambda$ is Lebesgue measure, then $h$ may be altered on $A$ without affecting the validity of (2.6), so that, in general, the map $f\to h$ is not unique. This is why we refer to a martingale generator.

Optimal stopping problem

Let $X=(\Omega,\mathcal{G},\mathcal{G}_{t},X_{t},\theta_{t},\mathbb{P}_{x}:x\in E,t\in\mathbb{R}_{+})$ be a right process. Given a function $g:E\to\mathbb{R}$, $\alpha\geq 0$ and $T\in\mathbb{R}_{+}\cup\{\infty\}$ define a corresponding gains process $G^{\alpha}$ (we simply write $G$ if $\alpha=0$) by $G^{\alpha}_{t}=e^{-\alpha t}g(X_{t})$ for $t\in[0,T]$. In the case of $T=\infty$, we make the following conventions:

[TABLE]

Let $\mathcal{E}^{e},\mathcal{E}^{u}$ be the $\sigma$-algebras on $E$ generated by excessive functions and universally measurable sets, respectively (recall that $\mathcal{E}\subset\mathcal{E}^{e}\subset\mathcal{E}^{u}$). We write

[TABLE]

For a filtration $(\hat{\mathcal{G}}_{t})$, and $(\hat{\mathcal{G}}_{t})$ - stopping times $\sigma_{1}$ and $\sigma_{2}$, with $\mathbb{P}_{x}[0\leq\sigma_{1}\leq\sigma_{2}\leq T]=1$, $x\in E$, let $\mathcal{T}_{\sigma_{1},\sigma_{2}}(\hat{\mathcal{G}})$ be the set of $(\hat{\mathcal{G}}_{t})$ - stopping times $\tau$ with $\mathbb{P}_{x}[\sigma_{1}\leq\tau\leq\sigma_{2}]=1$. Consider the following optimal stopping problem:

[TABLE]

By convention we set $V(\Delta)=G^{\alpha}_{\infty}$. The following result is due to El Karoui et al. [18].

Theorem 2.6**.**

Let $X=(\Omega,\mathcal{G},\mathcal{G}_{t},X_{t},\theta_{t},\mathbb{P}_{x}:x\in E,t\in\mathbb{R}_{+})$ be a right process with canonical filtration $(\mathcal{F}_{t})$. If $g\in\mathcal{Y}$, then

[TABLE]

and $(e^{-\alpha t}V(X_{t}))$ is a Snell envelope of $G^{\alpha}$, i.e. for all $x\in E$ and $\tau\in\mathcal{T}_{0,T}(\mathcal{F})$

[TABLE]

The first important consequence of the theorem is that we can (and will) work with the canonical realisation $\mathcal{C}(X)$. The second one provides a crucial link between the Snell envelope process in the general setting and the value function in the Markovian framework.

Remark*.*

The restriction to gains processes of the form $G=g(X)$ (or $G^{\alpha}$ if $\alpha>0$) is much less restrictive than might appear. Given that we work on the canonical path space with $\theta$ being the usual shift operator, we can expand the state-space of $X$ by appending an adapted functional $F$, taking values in the space $(E^{\prime},\mathcal{E}^{\prime})$, with the property that

[TABLE]

This allows us to deal with time-dependent problems, running rewards and other path-functionals of the underlying Markov process.

Lemma 2.7**.**

Suppose $X$ is a canonical Markov process $X$ taking values in the space $(E,\mathcal{E})$ where $E$ is a locally compact, countably based Hausdorff space and $\mathcal{E}$ is its Borel $\sigma$-algebra. Suppose also that $F$ is a path functional of X satisfying (2.7) and taking values in the space $(E^{\prime},\mathcal{E}^{\prime})$ where $E^{\prime}$ is a locally compact, countably based Hausdorff space with Borel $\sigma$-algebra $\mathcal{E}^{\prime}$, then, defining $Y=(X,F)$, $Y$ is still Markovian. If $X$ is a strong Markov process and $F$ is right-continuous, then $Y$ is strong Markov. If $X$ is a Feller process and $F$ is right-continuous , then $Y$ is strong Markov, has a càdlàg modification and the completion of the natural filtration of $X$, $\mathbb{F}$, is right-continuous and quasi-left continuous, and thus Y is a right process.

Example 2.8**.**

If $X$ is a one-dimensional Brownian motion, then $Y$, defined by

[TABLE]

where $L^{0}$ is the local time of $X$ at [math], is a Feller process on the filtration of $X$.

3 Main results

In this section we retain the notation of Section 2.1 and Section 2.2.

3.1 General framework

The assumption that $G\in\mathbb{G}$ (i.e. $G$ is a semimartingale with integrable supremum and $G=N+D$ is its canonical decomposition), neither ensures that $N\in\mathcal{M}_{0}$, nor that $D$ is an $IV$ process, the latter, it turns out, being sufficient for the main result of this section to hold. In order to prove Theorem 3.3 we will need a stronger integrability condition on $G$.

For any adapted càdlàg process $H$, define

[TABLE]

and

[TABLE]

Remark*.*

Note that $\bar{\mathbb{G}}=\mathcal{S}^{1}$, so that under the current conditions we have that $G\in\mathcal{S}^{1}$.

For a special semimartingale $X$ with canonical decomposition $X=\bar{B}+\bar{I}$, where $\bar{B}\in\mathcal{M}_{0,loc}$ and $\bar{I}$ is a predictable $FV$ process, define the $\mathcal{H}^{p}$ norm, for $1\leq p\leq\infty$, by

[TABLE]

and, as usual, write $X\in\mathcal{H}^{p}$ if $\lvert\lvert X\lvert\lvert_{\mathcal{H}^{p}}<\infty$.

Remark*.*

A more standard definition of the $\mathcal{H}^{p}$ norm is with $\lvert\lvert\bar{B}\lvert\lvert_{\mathcal{S}^{p}}$ replaced by $\lvert\lvert[\bar{B},\bar{B}]^{1/2}_{T}\lvert\lvert_{L^{p}}$. However, the Burkholder-Davis-Gundy inequalities (see Protter [40], Theorem 48 and references therein) imply the equivalence of these norms.

The following lemma follows from the fact that $\bar{I}^{*}\leq\int^{T}_{0}\lvert d\bar{I}_{s}|$, $\mathbb{P}-$a.s:

Lemma 3.1**.**

On the space of semimartingales, the $\mathcal{H}^{p}$ norm is stronger than $\mathcal{S}^{p}$ for $1\leq p<\infty$, i.e. convergence in $\mathcal{H}^{p}$ implies convergence in $\mathcal{S}^{p}$.

In general, it is challenging to check whether a given process belongs to $\mathcal{H}^{1}$, and thus the assumption that $G\in\mathcal{H}^{1}$ might be too stringent. On the other hand, under the assumptions in the Markov setting (see Section 3.2), we will have that $G$ is $locally$ in $\mathcal{H}^{1}$. Recall that a semimartingale $X$ belongs to $\mathcal{H}^{p}_{loc}$, for $1\leq p\leq\infty$, if there exists a sequence of stopping times $\{\sigma_{n}\}_{n\in\mathbb{N}}$, increasing to infinity almost surely, such that for each $n\geq 1$, the stopped process $X^{\sigma_{n}}$ belongs to $\mathcal{H}^{p}$. Hence, the main assumption in this section is the following:

Assumption 3.2**.**

$G$* is a semimartingale in both $\mathcal{S}^{1}$ and $\mathcal{H}^{1}_{loc}$.*

Remark*.*

Given that $G\in\mathcal{H}^{1}$, Lemma 3.1 implies that Assumption 3.2 is satisfied, and thus all the results of Section 2.1 hold. Moreover, we then have a canonical decomposition of $G$

[TABLE]

with $N\in\mathcal{M}_{0,UI}$ and a predictable $IV$ process $D$. On the other hand, under Assumption 3.2, (3.4) holds only for the stopped process $G^{\sigma_{n}}$, $n\geq 1$.

We finally arrive to the main result of this section:

Theorem 3.3**.**

Suppose Assumption 3.2 holds. Let $D^{-}$ $(D^{+})$ denote the decreasing (increasing) components of $D$, as in Lemma 2.5. Then $A$ is, as a measure, absolutely continuous with respect to $D^{-}$ almost surely on $[0,T]$, and $\mu$, defined by

[TABLE]

satisfies $0\leq\mu_{t}\leq 1$ almost surely.

Remark*.*

As is usual in semimartingale calculus, we treat a process of bounded variation and its corresponding Lebesgue-Stiltjes signed measure as synonymous.

The proof of Theorem 3.3 is based on the discrete-time approximation of the predictable $FV$ processes in the decompositions of $S$ (2.4) and $G$ (2.5). In particular, let $\mathcal{P}_{n}=\{0=t^{n}_{0}<t^{n}_{1}<t^{n}_{2}<...<t^{n}_{k_{n}}=T\}$, $n=1,2,...$, be an increasing sequence of partitions of $[0,T]$ with $\max_{1\leq k\leq k_{n}}t^{n}_{k}-t^{n}_{k-1}\to 0$ as $n\to\infty$. Let $S^{n}_{t}=S_{t^{n}_{k}}$ if $t^{n}_{k}\leq t<t^{n}_{k+1}$ and $S^{n}_{T}=S_{T}$ define the discretizations of $S$, and set

[TABLE]

If $S$ is regular in the sense that for every stopping time $\tau$ and nondecreasing sequence $(\tau_{n})_{n\in\mathbb{N}}$ of stopping times with $\tau=\lim_{n\to\infty}\tau_{n}$, we have $\lim_{n\to\infty}\mathbb{E}[S_{\tau_{n}}]=\mathbb{E}[S_{\tau}]$, or equivalently, if $A$ is continuous, Doléans [14] showed that $A^{n}_{t}\to A_{t}$ uniformly in $L^{1}$ as $n\to\infty$ (see also Rogers and Williams [44], VI.31, Theorem 31.2). Hence, given that $S$ is regular, we can extract a subsequence $\{A^{n_{l}}_{t}\}$, such that $\lim_{l\to\infty}A^{n_{l}}_{t}=A_{t}$ a.s. On the other hand, it is enough for $G$ to be regular:

Lemma 3.4**.**

Suppose $G\in\bar{\mathbb{G}}$ is a regular gains process. Then so is its Snell envelope process $S$.

See Appendix A for the proof.

Remark*.*

If it is not known that $G$ is regular, Kobylanski and Quenez [32], in a slightly more general setting, showed that $S$ is still regular, provided that $G$ is upper semicontinuous in expectation along stopping times, i.e. for all $\tau\in\mathcal{T}^{0,T}$ and for all sequences of stopping times $(\tau_{n})_{n\geq 1}$ such that $\tau_{n}\uparrow\tau$, we have

[TABLE]

The case where $S$ is not regular is more subtle. In his classical paper Rao [41] utilised the Dunford-Pettis compactness criterion and showed that, in general, $A^{n}_{t}\to A_{t}$ only weakly in $L^{1}$ as $n\to\infty$ (a sequence $(X_{n})_{n\in\mathbb{N}}$ of random variables in $L^{1}$ converges weakly in $L^{1}$ to $X$ if for every bounded random variable $Y$ we have that $\mathbb{E}[X_{n}Y]\to\mathbb{E}[XY]$ as $n\to\infty$).

Recall that $weak$ convergence in $L^{1}$ does not imply convergence in probability, and therefore, we cannot immediately deduce an almost sure convergence along a subsequence. However, it turns out that by modifying the sequence of approximating random variables, the required convergence can be achieved. This has been done in recent improvements of the Doob-Meyer decomposition (see Jakubowski [29] and Beiglböck et al. [4]. Also, Siorpaes [48] showed that there is a subsequence that works for all $(t,\omega)\in[0,T]\times\Omega$ simultaneously). In particular, Jakubowski proceeds as Rao, but then uses Komlós’s theorem [34] and proves the following:

Theorem 3.5**.**

There exists a subsequence $\{n_{l}\}$ such that for $t\in\cup^{\infty}_{n=1}\mathcal{P}_{n}$ and as $L\to\infty$

[TABLE]

Proof of Theorem 3.3.

Let $(\sigma_{n})_{n\geq 1}$ be a localising sequence for $G$ such that, for each $n\geq 1$, $G^{\sigma_{n}}=(G_{t\wedge\sigma_{n}})_{0\leq t\leq T}$ is in $\mathcal{H}^{1}$. Similarly, set $S^{\sigma_{n}}=(S_{t\wedge\sigma_{n}})_{0\leq t\leq T}$ for a fixed $n\geq 1$. We need to prove that

[TABLE]

since then, as $\sigma_{n}\uparrow\infty$ almost surely, as $n\to\infty$, and by uniqueness of $A$ and $D^{-}$, the result follows. In particular, since $A$ is increasing, the first inequality in (3.6) is immediate, and thus we only need to prove the second one.

After localisation we assume that $G\in\mathcal{H}$. For any $0\leq t\leq T$ and $0\leq\epsilon\leq T-t$ we have that

[TABLE]

where $\tau\in\mathcal{T}_{t+\epsilon,T}$ is arbitrary. Therefore

[TABLE]

Then by the definition of $S$ and using (3.7) together with the properties of the $essential$ $supremum$ (see also Lemma A.1 in the Appendix A) we obtain

[TABLE]

The first equality in (3.1) follows by noting that $\mathcal{T}_{t+\epsilon,T}\subset\mathcal{T}_{t,T}$, and that for any $\tau\in\mathcal{T}_{t+\epsilon,T}$ the term inside the expectation vanishes. Using the decomposition of $G$ and by observing that, for all $\tau\in\mathcal{T}_{t,t+\epsilon}$, $(D^{+}_{\tau}-D^{+}_{t+\epsilon})\leq 0$, while $N$ is a uniformly integrable martingale, we obtain

[TABLE]

Finally, for $0\leq s<t\leq T$, applying Theorem 3.5 to $A$ together with (3.1) gives

[TABLE]

where $k^{\prime}\leq k$ are such that $t^{n_{l}}_{k^{\prime}}\leq s<t^{n_{l}}_{k^{\prime}+1}$ and $t^{n_{l}}_{k}\leq t<t^{n_{l}}_{k+1}$ . Note that $D^{-}$ is also the predictable, increasing $IV$ process in the Doob-Meyer decomposition of the class (D) supermartingale $(G-D^{+})$. Therefore we can approximate it in the same way as $A$, so that $D^{-}_{t}-D^{-}_{s}$ is the almost sure limit along, possibly, a further subsequence $\{n_{l_{k}}\}$ of $\{n_{l}\}$, of the right hand side of (3.1). Here we rely on the special property of the subsequence $\{n_{l}\}$. In particular, it can be chosen such that convergence (3.5) also works along suitable subsequence of any further subsequence, see Remark 1 in Jakubowski [29]. ∎

We finish this section with a lemma that gives an easy test as to whether the given process belongs to $\mathcal{H}^{1}_{loc}$ (consult Appendix A for the proof).

Lemma 3.6**.**

Let $X\in\mathbb{G}$ with a canonical decomposition $X=L+K$, where $L\in\mathcal{M}_{0,loc}$ and $K$ is a predictable $FV$ process. If the jumps of $K$ are uniformly bounded by some finite constant $c>0$, then $X\in\mathcal{H}^{1}_{loc}$.

3.2 Markovian setting

In the rest of the section (and the paper) we consider the following optimal stopping problem:

[TABLE]

for a measurable function $g:E\to\mathbb{R}$ and a Markov process $X$ satisfying the following set of assumptions:

Assumption 3.7**.**

$X$* is a right process.*

Assumption 3.8**.**

$\sup_{0\leq t\leq T}\lvert g(X_{t})\lvert\in L^{1}(\mathbb{P}_{x})$, $x\in E$.

Assumption 3.9**.**

$g\in\mathbb{D}(\mathcal{L})$, i.e. $g(\cdot)$ belongs to the domain of a martingale generator of $X$.

Remark*.*

Lemma 2.7 tells us that if $X$ is Feller and $F$ is an adapted path-functional of the form given in (2.7) then (a modification of) $(X,F)$ satisfies Assumption 3.7.

Example 3.10**.**

Let $X=(X_{t})_{t\geq 0}$ be a Markov process and let $\mathbb{D}(\hat{\mathcal{L}})$ be the domain of a classical infinitesimal generator of $X$, i.e. the set of measurable functions $f:E\to\mathbb{R}$, such that $\lim_{t\to 0}(\mathbb{E}_{x}[f(X_{t})]-f(x))/t$ exists. Then $\mathbb{D}(\mathcal{\hat{L}})\subset\mathbb{D}(\mathcal{L})$. In particular,

if $X=(X_{t})_{t\geq 0}$ is a solution of an SDE driven by a Brownian motion in $\mathbb{R}^{d}$, then $C^{2}\subset\mathbb{D}(\mathcal{\hat{L}})$;

• 2.

if the state space $E$ is finite (so that $X$ is a continuous time Markov chain), then any measurable and bounded $f:E\to\mathbb{R}$ belongs to $\mathbb{D}(\mathcal{\hat{L}})$

• 3.

if $X$ is a Lévy process on ${\mathbb{R}}^{d}$ with finite variance increments then $C^{2}({\mathbb{R}}^{d},{\mathbb{R}})\subset\mathbb{D}(\mathcal{\hat{L}})$

Note that the gains process is of the form $G=g(X)$, while by Theorem 2.6, the corresponding Snell envelope is given by

[TABLE]

In a similar fashion to that in the general setting, Assumption 3.8 ensures the class (D) property for the gains and Snell envelope processes. Moreover, under Assumption 3.9,

[TABLE]

and the $FV$ process in the semimartingale decomposition of $G=g(X)$ is absolutely continuous with respect to Lebesgue measure, and therefore predictable, so that (3.12) is a canonical semimartingale decomposition of $G=g(X)$. Then, by Assumption 3.8, and using Lemma 3.6, we also deduce that $g(X)\in\mathcal{H}^{1}_{loc}$.

Remark*.*

When $T<\infty$, the optimal stopping problem, in general, is time-inhomogeneous, and we need to replace the process $X_{t}$ by the process $Z_{t}=(t,X_{t})$, $t\in[0,T]$, so that (3.11) reads

[TABLE]

where $\tilde{g}:[0,T]\times E\to\mathbb{R}$ is a new payoff function (consult Peskir and Shiryaev [39] for examples). In this case, Assumption 3.9 should be replaced by a requirement that there exists a measurable function $\tilde{h}:[0,T]\times E\to\mathbb{R}$ such that $M^{\tilde{g}}_{t}:=\tilde{g}(Z_{t})-\tilde{g}(0,x)-\int^{t}_{0}\tilde{h}(Z_{s})ds$ defines a local martingale.

The crucial result of this section is the following:

Theorem 3.11**.**

Suppose Assumptions 3.7, 3.8 and 3.9 hold. Then $V\in\mathbb{D}(\mathcal{L})$.

Proof.

In order to be consistent with the notation in the general framework, let

[TABLE]

Recall Lemma 2.5. Then $D^{+}$ and $D^{-}$ are explicitly given (up to initial values) by

[TABLE]

In particular, $D^{-}$ is, as a measure, absolutely continuous with respect to Lebesgue measure. By applying Theorem 3.3, we deduce that

[TABLE]

where $\mu$ is a non-negative Radon-Nikodym derivative with $0\leq\mu_{s}\leq 1$. Then we also have that $\int^{t}_{0}\lvert\mu_{s}\mathcal{L}g(X_{s})^{-}\lvert ds<\infty$, for every $0\leq t\leq T$.

In order to finish the proof we are left to show that there exists a suitable measurable function $\lambda:E\to\mathbb{R}$ such that $A_{t}=\int^{t}_{0}\mu_{s}\mathcal{L}g(X_{s})^{-}ds=\int^{t}_{0}\lambda(X_{s})ds$ a.s., for all $t\in[0,T]$. For this, recall that a process $Z$ (on $(\Omega,\mathcal{G},\mathcal{G}_{t},X_{t},\theta_{t},\mathbb{P}_{x}:x\in E,t\in\mathbb{R}_{+})$ or just on $\mathcal{C}(X)$) is additive if $Z_{0}=0$ a.s. and $Z_{t+s}=Z_{t}+Z_{s}\circ\theta_{t}$ a.s., for all $s,t\in[0,T]$. Moreover, for any measurable function $f:E\to\mathbb{R}$, $Z^{f}_{t}=f(X_{t})-f(x)$ defines an additive process. In particular, if $Z^{f}$ is also a semimartingale, then the martingale and $FV$ processes in the decomposition of $Z^{f}$ are also additive (Çinlar et al. [8] gives necessary and sufficient conditions for $Z^{f}$ to be a semimartingale).

Finally, we have that $A_{t}=\int^{t}_{0}\mu_{s}\mathcal{L}g(X_{s})^{-}ds$, $t\in[0,T]$, is an increasing additive process such that $dA_{t}\ll dt$. Set $K_{t}=\liminf_{s\downarrow 0,s\in\mathbb{Q}}(A_{t+s}-A_{t})/s$ and $\beta(x)=\mathbb{E}_{x}[K_{0}]$, $x\in E$. Then by Proposition 3.56 in Çinlar et al. [8], we have that, for $t\in[0,T]$, $A_{t}=\int^{t}_{0}\beta(X_{s})ds$ $\mathbb{P}_{x}$-a.s. for each $x\in E$. ∎

Remark*.*

In some specific examples it is possible to relax Assumption 3.9. Let $\mathcal{S}:=\{x\in E:V(x)=g(x)\}$ be the stopping region. It is well-known that $S=V(X)$ is a martingale on the go region $\mathcal{S}^{c}$, i.e. $M^{c}$ given by

[TABLE]

is a martingale (see Lemma A.2). This implies that $\int_{0}^{t}1_{(X_{s-}\in\mathcal{S}^{c})}dA_{s}=0$, and therefore we note that in order for $V\in\mathbb{D}(\mathcal{L})$, we need $D$ to be absolutely continuous with respect to Lebesgue measure $\lambda$ only on the stopping region i.e. that $\int_{0}^{\cdot}1_{(X_{s-}\in\mathcal{S})}dD_{s}\ll\lambda$. For example, let $E=\mathbb{R}$, fix $K\in\mathbb{R}_{+}$ and consider $g(\cdot)$ given by $g(x)=(K-x)^{+}$, $x\in E$. We can easily show, under very weak conditions, that $\mathcal{S}\subset[0,K]$ and so we need only have that $\int_{0}^{\cdot}1_{(X_{s-}<K)}dD_{s}$ is absolutely continuous.

4 Applications: duality, smooth fit

In this section we retain the setting of Section 3.2.

4.1 Duality

Let $x\in E$ be fixed. As before, let $\mathcal{M}_{0,UI}^{x}$ denote all the right-continuous uniformly integrable càdlàg martingales (started at zero) on the filtered space $(\Omega,\mathcal{F},\mathbb{F},\mathbb{P}_{x})$, $x\in E$. The main result of Rogers [43] in the Markovian setting reads:

Theorem 4.1**.**

Suppose Assumptions 3.7 and 3.8 hold. Then

[TABLE]

We call the right hand side of (4.1) the $dual$ of the optimal stopping problem. In particular, the right hand side of (4.1) is a ”generalised stochastic control problem of Girsanov type”, where a controller is allowed to choose a martingale from $\mathcal{M}_{0,UI}^{x}$, $x\in E$. Note that an optimal martingale for the dual is $M^{*}$, the martingale appearing in the Doob-Meyer decomposition of $S$, while any other martingale in $\mathcal{M}_{0,UI}^{x}$ gives an upper bound of $V(x)$. We already showed that $M^{*}=M^{V}$, which means that, when solving the dual problem, one can search only over martingales of the form $M^{f}$, for $f\in\mathbb{D}(\mathcal{L})$, or equivalently over the functions $f\in\mathbb{D}(\mathcal{L})$. We can further define $\mathcal{D}_{\mathcal{M}_{0,UI}}\subset\mathbb{D}(\mathcal{L})$ by

[TABLE]

To conclude that $V\in\mathcal{D}_{\mathcal{M}_{0,UI}}$ we need to show that $V$ is superharmonic, i.e. for all stopping times $\sigma\in\mathcal{T}^{0,T}$ and all $x\in E$, $\mathbb{E}_{x}[V(X_{\sigma})]\leq V(x)$. But this follows immediately from the Optional Sampling theorem, since $S=V(X)$ is a uniformly integrable supermartingale. Hence, as expected, we can restrict our search for the best minimising martingale to the set $\mathcal{D}_{\mathcal{M}_{0,UI}}$.

Theorem 4.2**.**

The dual problem, i.e. the right hand side of (4.1), is a stochastic control problem for a controlled Markov process when $G=g(X)$ and the assumptions of Theorem 3.11 hold.

Proof.

For any $f\in\mathcal{D}_{\mathcal{M}^{x}_{0,UI}}$, $x\in E$ and $y,z\in\mathbb{R}$, define processes $Y^{f}$ and $Z^{f}$ via

[TABLE]

and to allow arbitrary starting positions, set $Z^{f}_{t}=Z^{f}_{0,t}\vee z$, for $z\geq g(x)+y$. Note that, for any $f\in\mathbb{D}(\mathcal{L})$, $Y^{f}$ is an additive functional of $X$. Lemma 2.7 implies that if $f\in\mathcal{D}_{\mathcal{M}_{0,UI}}$ then $(X,Y^{f},Z^{f})$ is a Markov process.

Define $\hat{V}:E\times\mathbb{R}^{2}\to\mathbb{R}$ by

[TABLE]

It is clear that this is a stochastic control problem for the controlled Markov process $(X,Y^{f},Z^{f})$, where the admissible controls are functions in $\mathcal{D}_{\mathcal{M}_{0,UI}}$. Moreover, since $V\in\mathcal{D}_{\mathcal{M}_{0,UI}}$, by virtue of Theorem 4.1, and adjusting initial conditions as necessary, we have

[TABLE]

a ∎

4.2 Some remarks on the smooth pasting condition

We will now discuss the implications of Theorem 3.11 for the smoothness of the value function $V(\cdot)$ of the optimal stopping problem given in (3.11).

Remark*.*

While in Theorem 4.3 (resp. Theorem 4.5) we essentially recover (a small improvement of) Theorem 2.3 in Peskir [37] (resp. Theorem 2.3 in Samee [45]), the novelty is that we prove the results by means of stochastic calculus, as opposed to the analytic approach in [37] (resp. [45]).

In addition to Assumptions 3.8 and 3.9, we now assume that $X$ is a one-dimensional diffusion in the Itô-McKean [26] sense, so that $X$ is a strong Markov process with continuous sample paths. We also assume that the state space $E\subset\mathbb{R}$ is an interval with endpoints $-\infty\leq a\leq b\leq+\infty$. Nnote that the diffusion assumption implies Assumption 3.7. Finally, we assume that $X$ is $regular$: for any $x,y\in$ int$(E)$, $\mathbb{P}_{x}[\tau_{y}<\infty]>0$, where $\tau_{y}=\min\{t\geq 0:X_{t}=y\}$. Let $\alpha\geq 0$ be fixed; $\alpha$ corresponds to a killing rate of the sample paths of $X$.

The case without killing: $\alpha=0$

Let $s(\cdot)$ denote a scale function of $X$, i.e. a continuous, strictly increasing function on $E$ such that for $l$, $r$, $x\in E$, with $a\leq l<x<r\leq b$, we have

[TABLE]

see Revuz and Yor [42], Proposition 3.2 (p.301) for the proof of existence and properties of such a function.

From (4.2), using regularity of $X$ and that $V(X)$ is a supermartingale of class (D) we have that $V(\cdot)$ is $s$-concave:

[TABLE]

Theorem 4.3**.**

Suppose the assumptions of Theorem 3.11 are satisfied, so that $V\in\mathbb{D}(\mathcal{L})$. Further assume that $X$ is a regular, strong Markov process with continuous sample paths. Let $Y=s(X)$, where $s(\cdot)$ is a scale function of $X$.

Assume that for each $y\in[s(a),s(b)]$, the local time of $Y$ at $y$, $L^{y}$, is singular with respect to Lebesgue measure. Then, if $s\in{C}^{1}$, $V(\cdot)$, given by (3.11), belongs to ${C}^{1}$. 2. 2.

Assume that $([Y,Y]_{t})_{t\geq 0}$ is, as a measure, absolutely continuous with respect to Lebesgue measure. If $s^{\prime}(\cdot)$ is absolutely continuous, then $V\in C^{1}$ and $V^{\prime}(\cdot)$ is also absolutely continuous.

Remark*.*

If $\mathcal{G}$ is the filtration of a Brownian motion, $B$, then $Y=s(X)$ is a stochastic integral with respect to $B$ (a consequence of martingale representation):

[TABLE]

Moreover, Proposition 3.56 in Çinlar et al. [8] ensures that $\sigma_{t}=\sigma(Y_{t})$ for a suitably measurable function $\sigma$ and

[TABLE]

In this case, both, the singularity of the local time of Y and absolute continuity of $[Y,Y]$ (with respect to Lebesgue measure), are inherited from those of Brownian motion. On the other hand, if X is a regular diffusion (not necessarily a solution to an SDE driven by a Brownian motion), absolute continuity of $[Y,Y]$ still holds, if the speed measure of $X$ is absolutely continuous (with respect to Lebesgue measure).

Proof.

Note that $Y=s(X)$ is a Markov process, and let $\mathcal{K}$ denote its martingale generator. Moreover, $V(x)=W(s(x))$ (see Lemma 4.4 and the following remark), where, on the interval $[s(a),s(b)]$, $W(\cdot)$ is the smallest nonnegative concave majorant of the function $\hat{g}(y)=g\circ s^{-1}(y)$. Then, since $V\in\mathbb{D}(\mathcal{L})$,

[TABLE]

and thus

[TABLE]

Therefore, $W\in\mathbb{D}(\mathcal{K})$, since

[TABLE]

for $y\in[s(a),s(b)]$, $0\leq t\leq T$, with $\mathcal{K}W=\mathcal{L}V\circ s^{-1}\leq 0$.

On the other hand, using the generalised Itô formula for concave/convex functions (see e.g. Revuz and Yor [42], Theorem 1.5 p.223) we have

[TABLE]

for $y\in[s(a),s(b)]$, $0\leq t\leq T$, where $L^{z}_{t}$ is the local time of $Y_{t}$ at $z$, and $\nu$ is a non-negative $\sigma$-finite measure corresponding to the second derivative of $-W$ in the sense of distributions. Then, by the uniqueness of the decomposition of a special semimartingale, we have that, for $t\in[0,T]$,

[TABLE]

In order to prove the first claim, using the Lebesgue decomposition theorem, split $\nu$ into $\nu=\nu_{c}+\nu_{s}$, where $\nu_{c}$ and $\nu_{s}$ are measures, absolutely continuous and singular (with respect to Lebesgue measure), respectively, so that

[TABLE]

Now suppose that $\nu_{s}(\{z_{0}\})>0$ for some $z_{0}\in(s(a),s(b))$. Then, using (4.6) and (4.7), we have

[TABLE]

Since $L_{t}^{z_{0}}$ is positive with positive probability and, by assumption, $L^{y}$, $y\in[s(a),s(b)]$, is singular with respect to Lebesgue measure, the right hand side of (4.8) contradicts absolute continuity of the left hand side. Therefore, $\nu_{s}(\{z_{0}\})=0$, and since $z_{0}$ was arbitrary, we have that $\nu_{s}$ does not charge points (so that $\nu_{s}$ is singular continuous with respect to Lebesgue measure). It follows that $W\in C^{1}$. Since $s\in{C}^{1}$ by assumption, we conclude that $V\in{C}^{1}$.

We now prove the second claim. By assumption, $[Y,Y]$ is absolutely continuous with respect to Lebesgue measure (on the time axis). Invoking Proposition 3.56 in Çinlar et al. [8] again, we have that

[TABLE]

(as in Remark Remark). A time-change argument allows us to conclude that $Y$ is a time-change of a BM and that we may neglect the set $\{t:\sigma^{2}(Y_{t})=0\}$ in the representation (4.2). Thus

[TABLE]

where $N$ is the zero set of $\sigma$. Then, using the occupation time formula (see, for example, Revuz and Yor [42], Theorem 1.5 p.223) we have that

[TABLE]

where $f:[s(a),s(b)]\to\mathbb{R}$ is given by $f:y\mapsto-\frac{\mathcal{K}W}{\sigma}1_{N^{c}}(y)$. Now observe that, for $0\leq r\leq t\leq T$, \eta([r,t]):=\int^{s(b)}_{s(a)}f(z)\Big{(}L_{t}^{z}-L_{r}^{z}\Big{)}dz and \pi([r,t]):=\int^{s(b)}_{s(a)}\Big{(}L_{t}^{z}-L_{r}^{z}\Big{)}\nu(dz) define measures on the time axis, which, by virtue of (4.6), are equal (and thus both are absolutely continuous with respect to Lebesgue measure). Now define $T^{\underline{l},\bar{l}}:=\{t:Y_{t}\in[\underline{l},\bar{l}]\}$, $s(a)\leq\underline{l}\leq\bar{l}\leq s(b)$. Then the restrictions of $\eta$ and $\pi$ to $T^{\underline{l},\bar{l}}$, $\eta\lvert_{T^{\underline{l},\bar{l}}}$ and $\pi\lvert_{T^{\underline{l},\bar{l}}}$, are also equal. Moreover, since $Y$ is a local martingale, it is also a semimartingale. Therefore, for every $0\leq t\leq T$, $L_{t}^{z}$ is carried by the set $\{t:Y_{t}=z\}$ (see Protter [40], Theorem 69 p.217). Hence, for each $t\in[0,T]$,

[TABLE]

and, since $\underline{l}$ and $\bar{l}$ are arbitrary, the left and right hand sides of (4.9) define measures on $[s(a),s(b)]\subseteq\mathbb{R}$, which are equal. It follows that $f(z)dz=\nu(dz)$, and thus, by uniqueness of the Lebesgue decomposition of $\sigma$-finite measures, $\nu_{s}=0$. This proves that $W\in C^{1}$ and $W^{\prime}(\cdot)$ is absolutely continuous on $[s(a),s(b)]$ with Radon-Nykodym derivative $f$. Since the product and composition of absolutely continuous functions are absolutely continuous, we conclude that $V^{\prime}(\cdot)$ is absolutely continuous (since $s^{\prime}(\cdot)$ is, by assumption). ∎

Remark*.*

We note that for a smooth fit principle to hold, it is not necessary that $s\in{C}^{1}$. Given that all the other conditions of Theorem 4.3 hold, it is sufficient that $s(\cdot)$ is differentiable at the boundary of the continuation region. On the other hand, if $g\in\mathbb{D}(\mathcal{L})$, $V\in C^{1}$, even if $g\notin C^{1}$.

Moreover, since $V=g$ on the stopping region, Theorem 4.3 tells us that $g\in C^{1}$ on the interior of the stopping region. However, the question whether this stems already from the assumption that $g\in\mathbb{D}(\mathcal{L})$ is more subtle. For example, if $g\in\mathbb{D}(\mathcal{L})$ and $g$ is a difference of two convex functions, then by the generalised Itô formula and the local time argument (similarly to the proof of Theorem 4.3) we could conclude that $g\in C^{1}$ on the whole state space $E$.

Case with killing: $\alpha>0$

We now generalise the results of the Theorem 4.3 in the presence of a non-trivial killing rate. Consider the following optimal stopping problem

[TABLE]

Note that, since $\alpha>0$, using the regularity of $X$ together with the supermartingale property of $V(X)$ we have that

[TABLE]

Define increasing and decreasing functions $\psi,\phi:E\to\mathbb{R}$, respectively, by

[TABLE]

where $c\in E$ is arbitrary. Then, $(\Psi_{t})_{0\leq t\leq T}$ and $(\Phi_{t})_{0\leq t\leq T}$, given by

[TABLE]

respectively, are local martingales (and also supermartingales, since $\psi,\phi$ are non-negative); see Dynkin [15] and Itô and McKean [26].

Let $p_{1},p_{2}:[l,r]\to[0,1]$ (where $[l,r]\subseteq E$) be given by

[TABLE]

Continuity of paths of $X$ implies that $p_{i}(\cdot),i=1,2$, are both continuous (the proof of continuity of the scale function in (4.2) can be adapted for a killed process). In terms of the functions $\psi(\cdot)$, $\phi(\cdot)$ of (4.12), using appropriate boundary conditions, one calculates

[TABLE]

Let $\tilde{s}:E\to\mathbb{R}_{+}$ be the continuous increasing function defined by $\tilde{s}(x)=\psi(x)/\phi(x)$. Substituting (4.13) into (4.11) and then dividing both sides by $\phi(x)$ we get

[TABLE]

so that $V(\cdot)/\phi(\cdot)$ is $\tilde{s}$-concave.

Recall that Eq. 4.11 essentially follows from $V(\cdot)$ being $\alpha$-superharmonic, so that it satisfies $\mathbb{E}_{x}[e^{-\alpha\tau}V(X_{\tau})]\leq V(x)$ for $x\in E$ and any stopping time $\tau$. Since $\Phi$ and $\Psi$ are local martingales, it follows that the converse is also true, i.e. given a measurable function $f:E\to\mathbb{R}$, $f(\cdot)/\phi(\cdot)$ is $\tilde{s}$-concave if and only if $f(\cdot)$ is $\alpha$-superharmonic (Dayanik and Karatzas [11], Proposition 4.1). This shows that a value function $V(\cdot)$ is the minimal majorant of $g(\cdot)$ such that $V(\cdot)/\phi(\cdot)$ is $\tilde{s}$-concave.

Lemma 4.4**.**

Suppose $[l,r]\subseteq E$ and let $W(\cdot)$ be the smallest nonnegative concave majorant of $\tilde{g}:=(g/\phi)\circ\tilde{s}^{-1}$ on $[\tilde{s}(l),\tilde{s}(r)]$, where $\tilde{s}^{-1}$ is the inverse of $\tilde{s}$. Then $V(x)=\phi(x)W(\tilde{s}(x))$ on $[l,r]$.

Proof.

Define $\hat{V}(x)=\phi(x)W(\tilde{s}(x))$ on $[l,r]$. Then, trivially, $\hat{V}(\cdot)$ majorizes $g(\cdot)$ and $\hat{V}(\cdot)/\phi(\cdot)$ is $\tilde{s}$-concave. Therefore $V(x)\leq\hat{V}(x)$ on $[l,r]$.

On the other hand, let $\hat{W}(y)=(V/\phi)(\tilde{s}^{-1}(y))$ on $[\tilde{s}(l),\tilde{s}(r)]$. Since $V(x)\geq g(x)$ and $(V/\phi)(\cdot)$ is $\tilde{s}$-concave on $[l,r]$, $\hat{W}(\cdot)$ is concave and majorizes $(g/\phi)\circ\tilde{s}^{-1}(\cdot)$ on $[\tilde{s}(l),\tilde{s}(r)]$. Hence, $W(y)\leq\hat{W}(y)$ on $[\tilde{s}(l),\tilde{s}(r)]$.

Finally, $(V/\phi)(x)\leq(\hat{V}/\phi)(x)=W(\tilde{s}(x))\leq\hat{W}(\tilde{s}(x))=(V/\phi)(x)$ on $[l,r]$. ∎

Remark*.*

When $\alpha=0$, let $(\psi,\phi)=(s,1)$. Then Lemma 4.4 is just Proposition 4.3. in Dayanik and Karatzas [11].

With the help of Lemma 4.4 and using parallel arguments to those in the proof of Theorem 4.3 we can formulate sufficient conditions for $V$ to be in ${C}^{1}$ and have absolutely continuous derivative.

Theorem 4.5**.**

Suppose the assumptions of Theorem 3.11 are satisfied, so that $V\in\mathbb{D}(\mathcal{L})$. Further assume that $X$ is a regular Markov process with continuous sample paths. Let $\psi(\cdot),\phi(\cdot)$ be as in (4.12) and consider the process $Y=\tilde{s}(X)$.

Assume that, for each $y\in[\tilde{s}(a),\tilde{s}(b)]$, the local time of $Y$ at $y\in[\tilde{s}(a),\tilde{s}(b)]$, $\hat{L}^{y}$, is singular with respect to Lebesgue measure. Then if $\psi,\phi\in{C}^{1}$, $V(\cdot)$, given by (4.10), belongs to ${C}^{1}$. 2. 2.

Assume that $[Y,Y]$ is, as a measure, absolutely continuous with respect to Lebesgue measure. If $\psi^{\prime}(\cdot),\phi^{\prime}(\cdot)$ are both absolutely continuous, then $V^{\prime}(\cdot)$ is aslo absolutely continuous.

Proof.

First note that $Y$ is not necessarily a local martingale, while $\Phi Y$ is. Indeed, $\Phi Y=\Psi$. Hence

[TABLE]

is the difference of two local martingales, and thus is a local martingale itself. Using the generalised Itô formula for concave/convex functions, we have

[TABLE]

for $y\in[\tilde{s}(l),\tilde{s}(r)]$, $0\leq t\leq T$, where $\hat{L}^{z}_{t}$ is the local time of $Y_{t}$ at $z$, and $\nu$ is a non-negative $\sigma$-finite measure corresponding to the derivative $W^{{}^{\prime\prime}}$ in the sense of distributions.

On the other hand, if $g\in\mathbb{D}(\mathcal{L})$, then $V\in\mathbb{D}(\mathcal{L})$. Therefore,

[TABLE]

Then, similarly to before, from the uniqueness of the decomposition of the Snell envelope, we have that the martingale and $FV$ terms in (4.14) and (4.15) coincide. Hence, for $t\in[0,T]$,

[TABLE]

Using the same arguments as in the proof of Theorem 4.3 we can show that both statements of this theorem hold. The details are left to the reader. ∎

Acknowledgments

We are grateful to two anonymous referees and Prof. Goran Peskir for useful comments and suggestions.

Appendix A

Lemma A.1**.**

For each $0\leq t\leq T$, the family of random variables $\{\mathbb{E}[G_{\tau}\lvert\mathcal{F}_{t}]:\tau\in\mathcal{T}_{t,T}\}$ is directed upwards, i.e. for any $\sigma_{1}$, $\sigma_{2}\in\mathcal{T}_{t,T}$, there exists $\sigma_{3}\in\mathcal{T}_{t,T}$, such that

[TABLE]

Proof.

Fix $t\in[0,T]$. Suppose $\sigma_{1}$, $\sigma_{2}\in\mathcal{T}_{t,T}$ and define $A:=\{\mathbb{E}[G_{\sigma_{1}}\lvert\mathcal{F}_{t}]\geq\mathbb{E}[G_{\sigma_{2}}\lvert\mathcal{F}_{t}]\}$. Let $\sigma_{3}:=\sigma_{1}\mathbbm{1}_{A}+\sigma_{2}\mathbbm{1}_{A^{c}}$. Note that $\sigma_{3}\in\mathcal{T}_{t,T}$. Using $\mathcal{F}_{t}$-measurability of $A$, we have

[TABLE]

which proves the claim. ∎

Lemma A.2**.**

Let $G\in\bar{\mathbb{G}}$ and $S$ be its Snell envelope with decomposition $S=M^{*}-A$. For $0\leq t\leq T$ and $\epsilon>0$, define

[TABLE]

Then $A_{K^{\epsilon}_{t}}=A_{t}$ a.s. and the processes $(A_{K^{\epsilon}_{t}})$ and $A$ are indistinguishable.

Proof.

From the directed upwards property (Lemma A.1) we know that $\mathbb{E}[S_{t}]=\sup_{\tau\in\mathcal{T}_{t,T}}\mathbb{E}[G_{\tau}]$. Then for a sequence $(\tau_{n})_{n\in\mathbb{N}}$ of stopping times in $\mathcal{T}_{t,T}$, such that $\lim_{n\to\infty}\mathbb{E}[G_{\tau_{n}}]=\mathbb{E}[S_{t}]$, we have

[TABLE]

since $M^{*}$ is uniformly integrable. Hence, since $A$ is non-decreasing,

[TABLE]

and thus we have equalities throughout. By passing to a sub-sequence we can assume that

[TABLE]

The first equality in (A.2) implies that $K^{\epsilon}_{t}\leq\tau_{n_{0}}$ a.s., for some large enough $n_{0}\in\mathbb{N}$, and thus $A_{K^{\epsilon}_{t}}\leq A_{\tau_{n}}$, for all $n_{0}\leq n$. Since $A$ is non-decreasing, we also have that $0\leq A_{K^{\epsilon}_{t}}-A_{t}\leq A_{\tau_{n}}-A_{t}$ a.s., $n_{0}\leq n$, and from the second equality in (A.2) we conclude that $A_{K^{\epsilon}_{t}}=A_{t}$ a.s. The indistinguishability follows from the right-continuity of $G$ and $S$. ∎

A.1 Proofs of results in Section 2

Proof of Lemma 2.7.

The completed filtration generated by a Feller process satisfies the usual assumptions, in particular, it is both right-continuous and quasi-left-continuous. The latter means that for any predictable stopping time $\sigma$, $\mathcal{F}_{\sigma-}=\mathcal{F}_{\sigma}$. Moreover, every càdlàg Feller process is left-continuous over stopping times and satisfies the strong Markov property. On the other hand, every Feller process admits a càdlàg modification (these are standard results and can be found, for example, in Revuz and Yor [42] or Rogers and Williams [44]). All that remains is to show that the addition of the functional $F$ leaves $(X,F)$ strong Markov. This is elementary from (2.7). ∎

A.2 Proofs of results in Section 3

Proof of Lemma 3.4.

Let $(\tau_{n})_{n\in\mathbb{N}}$ be a nondecreasing sequence of stopping times with $\lim_{n\to\infty}\tau_{n}=\tau$, for some fixed $\tau\in\mathcal{T}_{0,T}$. Since $S$ is a supermartingale, $\mathbb{E}[S_{\tau_{n}}]\geq\mathbb{E}[S_{\tau}]$, for every $n\in\mathbb{N}$. For a fixed $\epsilon>0$, $K^{\epsilon}_{\tau_{n}}$ (defined by Eq. A.1) is a stopping time, and by Lemma A.2, $A_{K^{\epsilon}_{\tau_{n}}}=A_{\tau_{n}}$ a.s. Therefore, since $M^{*}$ is uniformly integrable,

[TABLE]

Thus, by the definition of $K^{\epsilon}_{\tau_{n}}$,

[TABLE]

Let $\hat{\tau}:=\lim_{n\to\infty}K^{\epsilon}_{\tau_{n}}$. Note that the sequence $(K^{\epsilon}_{\tau_{n}})_{n\in\mathbb{N}}$ is non-decreasing and dominated by $K^{\epsilon}_{\tau}$. Hence $\tau\leq\hat{\tau}\leq K^{\epsilon}_{\tau}$. Finally, using the regularity of $G$ we obtain

[TABLE]

Since $\epsilon$ is arbitrary, the result follows. ∎

Proof of Lemma 3.6.

For $n\geq 1$, define

[TABLE]

Clearly $\tau_{n}\uparrow\infty$ as $n\to\infty$. Then for each $n\geq 1$

[TABLE]

Therefore, since $X\in\mathbb{G}$,

[TABLE]

and thus, $\lvert\lvert X^{\tau_{n}}\lvert\lvert_{\mathcal{H}^{1}}<\infty$, for all $n\geq 1$. ∎