Uncertainty Quantification for Markov Processes via Variational Principles and Functional Inequalities

TL;DR

This paper develops explicit bounds for uncertainty quantification in Markov processes by combining variational principles with functional inequalities, applicable to steady-states in both discrete and continuous-time settings.

Contribution

It introduces a novel approach that integrates variational formulas with functional inequalities to derive scalable uncertainty bounds for Markov processes.

Findings

Bounds are well-behaved in the infinite-time limit.

Applicable to steady-states of discrete and continuous-time Markov processes.

Provides explicit uncertainty quantification bounds for time-averaged observables.

Abstract

Information-theory based variational principles have proven effective at providing scalable uncertainty quantification (i.e. robustness) bounds for quantities of interest in the presence of nonparametric model-form uncertainty. In this work, we combine such variational formulas with functional inequalities (Poincar{\'e}, $\log$-Sobolev, Liapunov functions) to derive explicit uncertainty quantification bounds for time-averaged observables, comparing a Markov process to a second (not necessarily Markov) process. These bounds are well-behaved in the infinite-time limit and apply to steady-states of both discrete and continuous-time Markov processes.

Equations

$$\text{Bound the bias }{E}_{\tilde{P}}[F]-E_P[F]\text{ where }\tilde{P}\in {\mathcal{U}}_r(P).$$

$$\text{Bound }{\tilde{E}}^{\tilde{\mu }}\left[\frac{1}{T}{\int }_0^{T}f(X_t)dt\right]-\int fd{\mu }^{\ast }.$$

$$E^{\mu }\left[\frac{1}{T}{\int }_0^{T}f(X_t)dt\right]\to \int fd{\mu }^{\ast }.$$

$${\text{err}}_T=E^{\mu }\left[\frac{1}{T}{\int }_0^{T}f(X_t)dt\right]-{\tilde{E}}^{\tilde{\mu }}\left[\frac{1}{T}{\int }_0^{T}f(X_t)dt\right].$$

$$\pm \left({\tilde{E}}^{\tilde{\mu }}\left[\frac{1}{T}{\int }_0^{T}f(X_t)dt\right]-{\mu }^{\ast }[f]\right)$$

$$\le$$

$$\frac{1}{T}{\Lambda }_{P_T^{{\mu }^{\ast }}}^{{\hat{f}}_T}(\pm c)\le \kappa (V_{\pm c}),$$

$$\kappa (V)\equiv \sup \left\{⟨A[g],g⟩+\int V|g{|}^{2}d{\mu }^{\ast }:g\in D(A,\mathbb{R}),\Vert g{\Vert }_{L^2({\mu }^{\ast })}=1\right\},$$

$$V_{\pm c}(x)\equiv \pm c\left(f(x)-{\mu }^{\ast }[f]\right),{\mu }^{\ast }[f]\equiv \int fd{\mu }^{\ast }.$$

$$D^{2}V(x)\ge {\alpha }^{-1}I,\alpha >0.$$

$$\pm \left({\tilde{E}}^{\tilde{\mu }}\left[\frac{1}{T}{\int }_0^{T}f(X_t)dt\right]-{\mu }^{\ast }[f]\right)\le \sqrt{2{\sigma }^2\eta }+M^{\pm }\eta ,$$

$$M^{\pm }=\alpha \Vert (f-{\mu }^{\ast }[f]{)}^{\pm }{\Vert }_{\infty },{\sigma }^2=2\alpha {\mathrm{Var}}_{{\mu }^{\ast }}[f],\eta =\frac{1}{T}R({\tilde{P}}_T^{\tilde{\mu }}||P_T^{{\mu }^{\ast }}).$$

$$\mathcal{E}(P)=\left\{f:\Omega \to \mathbb{R}:E_P[e^{\pm c_{0}f}]<\infty \text{ for some }{c}_0>0\right\}.$$

$${\Lambda }_P^f(c)=\log E_P[e^{cf}]$$

$$\hat{f}=f-E_P[f]$$

$${\Lambda }_P^{\hat{f}}(c)=\log E_P[e^{c(f-E_P[f])}]={\Lambda }_P^f(c)-c{E}_P[f].$$

$$R(\widetilde{P}||P)=\left\{\begin{array}[]{cl}E_{\widetilde{P}}\left[\log\left(\frac{d\widetilde{P}}{dP}\right)\right]&\textrm{ if }\widetilde{P}\ll P\\ +\infty&\textrm{ otherwise}\end{array}\,.\right.$$

$$\log E_P[e^f]=\underset{\tilde{P}:R(\tilde{P}||P)<\infty }{\sup }\left\{E_{\tilde{P}}[f]-R(\tilde{P}||P)\right\}.$$

$$-\underset{c>0}{\inf }\left\{\frac{{\Lambda }_P^{\hat{f}}(-c)+R(\tilde{P}||P)}{c}\right\}\le E_{\tilde{P}}[f]-E_P[f]\le \underset{c>0}{\inf }\left\{\frac{{\Lambda }_P^{\hat{f}}(c)+R(\tilde{P}||P)}{c}\right\}.$$

$${\Xi }^{\pm }(\Lambda ,\eta )\equiv \underset{c>0}{\inf }\left\{\frac{\Lambda (\pm c)+\eta }{c}\right\}.$$

$$-{\Xi }^{-}\left({\Lambda }_P^{\hat{f}},R(\tilde{P}||P)\right)\le E_{\tilde{P}}[f]-E_P[f]\le {\Xi }^{+}\left({\Lambda }_P^{\hat{f}},R(\tilde{P}||P)\right).$$

$$\Xi (\tilde{P}||P;\pm f)\equiv {\Xi }^{\pm }\left({\Lambda }_P^{\hat{f}},R(\tilde{P}||P)\right)$$

$$\Xi (\tilde{P}||P,f)=\sqrt{2{\mathrm{Var}}_P[f]R(\tilde{P}||P)}+O(R(\tilde{P}||P)).$$

$$\underset{\tilde{P}\in {\mathcal{U}}_{\eta }}{\sup }\left\{E_{\tilde{P}}[f]-E_P[f]\right\}=E_{P^{\eta }}[f]-E_P[f]=\Xi (P^{\eta }||P;f).$$

$$d{P}^{\eta }=e^{cf-{\Lambda }_P^f(c)}dP,$$

$$\Lambda (\pm c)\le \frac{{\sigma }^2{c}^2}{2(1-c{M}^{\pm })}$$

$${\Xi }^{\pm }(\Lambda ,\eta )\le \sqrt{2{\sigma }^2\eta }+M^{\pm }\eta .$$

$$\underset{c>0}{\inf }\left\{\frac{\Lambda (\pm c)+\eta }{c}\right\}\le \sqrt{2{\Lambda }^{\prime \prime }(0)\eta }+o(\sqrt{\eta })$$

$$P^{\mu }(\cdot )=\int P^x(\cdot )\mu (dx),{\tilde{P}}^{\tilde{\mu }}(\cdot )=\int {\tilde{P}}^x(\cdot )\tilde{\mu }(dx).$$

$$P_T^x\equiv P^x{|}_{{\mathcal{F}}_T},{\tilde{P}}_T^x\equiv {\tilde{P}}^x{|}_{{\mathcal{F}}_T},$$

$$f_T={\int }_0^{T}f(X_t)dt.$$

$$\pm \left({\tilde{E}}^{\tilde{\mu }}\left[\frac{1}{T}{\int }_0^{T}f(X_t)dt\right]-{\mu }^{\ast }[f]\right)\le {\Xi }^{\pm }\left(\frac{1}{T}{\Lambda }_{P_T^{{\mu }^{\ast }}}^{{\hat{f}}_T},\frac{1}{T}R({\tilde{P}}_T^{\tilde{\mu }}||P_T^{{\mu }^{\ast }})\right),$$

Full text

1 Introduction

Information-theory based variational principles have proven effective at providing uncertainty quantification (i.e. robustness) bounds for quantities of interest in the presence of nonparametric model-form uncertainty chowdhary_dupuis_2013 ; DKPP ; doi:10.1063/1.4789612 ; doi:10.1137/15M1047271 ; KRW ; DKPR ; GKRW ; BREUER20131552 ; doi:10.1287/moor.2015.0776 ; GlassermanXu . In the present work, we combine these tools with functional inequalities to obtain improved and explicit uncertainty quantification (UQ) bounds for both discrete and continuous-time Markov processes on general state spaces.

In our approach, we are given a baseline model, described by a probability measure $P$; this is the model one has ‘in hand’ and that is amenable to analysis/simulation, but it may contain many sources of error and uncertainty. Perhaps it depends on parameters with uncertain values (obtained from experiment, Monte Carlo simulation, variational inference, etc.) or is obtained via some approximation procedure (dimension reduction, neglecting memory terms, asymptotic approximation, etc.) In short, any quantity of interest computed from $P$ has (potentially) significant uncertainty associated with it. Mathematically we chose to express this uncertainty by considering a (nonparametric) family, $\mathcal{U}(P)$, of alternative models that we postulate contains the inaccessible ‘true’ model.

Loosely stated, given some observable $F$, the uncertainty quantification goal considered here is

[TABLE]

The subscript $r$ indicates that the ‘neighborhood’ of alternative models, $\mathcal{U}_{r}(P)$, is often defined in terms of an error tolerance, $r>0$. For our purposes, the appropriate notion of neighborhood will be expressed in terms of relative entropy, which can be interpreted as measuring the loss of information due to uncertainties. We do not discuss in full generality how to choose the tolerance level $r$ but there are cases where one has enough information about the ‘true’ model to choose an appropriate tolerance; see Section 6.

Remark 1

Note that in Eq. (1), and the remainder of this paper, we consider the case where the quantity-of-interest is the expected value of some function, but extensions of these ideas to other quantities of interest are possible DKPR .

More specifically, here we work with a Markov process $(X_{t},P^{\mu})$ with initial distribution $\mu$ and stationary distribution $\mu^{*}$, and compare it to an alternative (not necessarily Markov) process $(X_{t},\widetilde{P}^{\widetilde{\mu}})$; we study the problem of bounding the bias when the finite-time averages of a real-valued observable, $f$, are computed by sampling from the alternative process:

[TABLE]

Here, $\widetilde{E}^{\widetilde{\mu}}$ denotes the expectation with respect to $\widetilde{P}^{\widetilde{\mu}}$ and similarly for $P^{\mu}$, $E^{\mu}$. (Discrete-time processes will also be considered in Section 5.)

Eq. (2) is a (less studied) variant of the classical problem of the convergence of ergodic averages to the expectation in the stationary distribution:

[TABLE]

By combining information on the problems Eq. (2) and Eq. (3), one can also obtain bounds on the finite time sampling error:

[TABLE]

In this work, we focus on the robustness problem, Eq. (2).

There are classical inequalities addressing Eq. (1) (ex: Csiszar-Kullback-Pinsker, Le Cam, Scheffé, etc.), but they exhibit poor scaling properties with problem dimension and/or in the infinite-time limit, and so are inappropriate for bounding Eq. (2). This problem can be addressed by using tight information inequalities based on the Gibbs variational principle that are summarized in Section 2. See KRW for a detailed discussion of these issues.

Other recent works have also focused on robustness bounds for Markov processes, often with the goal of providing error bounds for approximate Markov chain Monte Carlo samplers. Bounds on the difference between the distributions (finite-time or stationary) of Markov processes have been obtained in both total-variation 10.2307/30040824 ; ferre2013 ; Alquier2016 ; Medina-Aguayo2016 ; pmlr-v32-bardenet14 ; 2015arXiv150803387J and Wasserstein distances rudolf2018 ; 2014arXiv1405.0182P ; 2016arXiv160506420H .

One benefit of the approach taken in the present work is that the bounds naturally incorporate information on the specific observable, $f$, under investigation; for instance, the asymptotic variance of $f$ under the baseline model appears in the bound in Theorem 4.2, below. When the end goal is robustness bounds for time-averages of $f$, this observable specificity has the potential to yield tighter bounds; see also 2014arXiv1405.0182P for bounds that incorporate similar information on the observable.

Our method utilizes relative entropy to quantify the distance between models. A drawback, compared to the total-variation and Wasserstein distance approaches, is the requirement of absolute continuity; however, this is satisfied in many cases of interest. As we will see, one benefit of utilizing relative entropy is that the alternative model does not have to be a Markov process. The second main innovation here is the use of various functional inequalities, in combination with relative entropy, to bound Eq. (2). The end result is computable, finite-time UQ bounds that are also well behaved in the long-time limit.

1.1 Summary of Results

The basis for all of our bounds is Theorem 2.1 in Section 2:

[TABLE]

along with Corollary 1 in Section 3:

[TABLE]

In the above, $\Lambda_{P^{\mu^{*}}_{T}}^{\widehat{f}_{T}}(\pm c)$ is the cumulant generating function of $\widehat{f}_{T}$ (see Eq. (34) for details), $R(\widetilde{P}^{\widetilde{\mu}}_{T}||P^{\mu^{*}}_{T})$ is the relative entropy of the processes up to time $T$ (see Eq. (15)), $\langle\cdot,\cdot\rangle$ denotes the inner product on $L^{2}(\mu^{*})$, and $(A,D(A,\mathbb{R}))$ is the generator of the Markov semigroup for the process $(X_{t},P^{\mu})$ on $L^{2}(\mu^{*})$. Again, we emphasize that the alternative process, $(X_{t},\widetilde{P}^{\widetilde{\mu}})$, does not need to be Markov; for an example involving semi-Markov processes, see Section 6.2.

Eq. (5) is derived by employing the Gibbs variational principle (hence the relation to relative entropy). Eq. (6), which is based on a theorem proven in WU2000435 , results from a connection between the cumulant generating function and the Feynman-Kac semigroup (hence the appearance of the generator, $A$). Also, note that the bound is expected to behave well in the limit $T\to\infty$, as $R(\widetilde{P}^{\widetilde{\mu}}_{T}||P^{\mu^{*}}_{T})/T$ converges to the relative entropy rate of the processes, under suitable ergodicity assumptions.

Eq. (6) allows us to employ our primary new tool for UQ, that is, functional inequalities. By functional inequalities, we mean bounds on the generator, $A$, that will yield bounds on $\kappa(V_{\pm c})$; we will cover Poincaré, $\log$-Sobolev, and $F$-Sobolev inequalities, as well as Liapunov functions. Our results rely heavily on the bounds obtained in WU2000435 ; lezaud:hal-00940906 ; cattiaux_guillin_2008 ; Guillin2009 ; doi:10.1137/S0040585X97986667 where concentration inequalities for ergodic averages were obtained.

The method outlined above leads to explicit UQ bounds, expressed in terms of the following quantities:

Spectral properties of the generator, $A$, of the dynamics of the baseline model, $P$, in the stationary regime (i.e., on $L^{2}(\mu^{*})$); see Eq. (5)-Eq. (7). This term depends on the chosen observable but does not depend on the alternative model; functional inequalities are only required for the base model (which is often the simpler model). This is one of the strengths of the method, though computing explicit, tight constants for these functional inequalities is still a very difficult problem in general. 2. 2.

The path-space relative entropy up to time $T$, $R(\widetilde{P}^{\widetilde{\mu}}_{T}||P^{\mu^{*}}_{T})$, of the alternative model with respect to the base. This term depends heavily on the difference in dynamics between the two models; in particular, a nontrivial bound requires absolute continuity of the path-space distributions. This is a drawback of the relative-entropy based method employed here, but it does hold in many cases of interest; see Section 6 for examples.

While most of our results do not assume reversibility of the base process, bounds based on Eq. (7) only involve the symmetric part of the generator and so are generally less than ideal, or even useless, for many nonreversible systems. This is a drawback of the approach pursued here.

Remark 2

For certain hypocoercive systems, ergodicity can be proven by working with an alternative metric; see Dolbeault2009 ; Dolbeault2015 ; 2018arXiv180804299D . It is possible that the functional-inequality based UQ techniques developed below could be adapted to this more general setting; a step in that direction can be found in 2019arXiv190711973B .

For a simple example of the type of result obtained below, consider diffusion on $\mathbb{R}^{n}$ in a $C^{2}$ potential, $V$, i.e. the generator is $A=\Delta-\nabla V\cdot V$ and the invariant measure is $\mu^{*}=e^{-V(x)}dx$. Suppose the Hessian of $V$ is bounded below:

[TABLE]

Our results give a Bernstein-type UQ bound for any bounded $f$:

[TABLE]

(This bound can also be improved by using the asymptotic variance; see Section 4.2.) Section 4.4.1 contains further discussion of diffusions.

The remainder of the paper is structured as follows. Necessary background on UQ for both general measures and processes will be given in Section 2, leading up to a connection with both the Feynman-Kac semigroup and the relative entropy rate. Relevant properties of the Feynman-Kac semigroup are given in Section 3, culminating with the bound Eq. (6). The use of functional inequalities to obtain explicit UQ bounds from Eq. (6) will be explored in Section 4. In Section 5 we show how these ideas can be adapted to discrete-time processes. Finally, the problem of bounding the relative entropy rate will be addressed in Section 6.

2 Uncertainty Quantification for Markov Processes

2.1 UQ via Variational Principles

In this subsection, we recall several earlier results regarding the variational-principle approach to UQ, as developed in chowdhary_dupuis_2013 ; DKPP ; BREUER20131552 ; GlassermanXu ; BLM . In particular, Proposition 2, quoted from DKPP , will be a critical tool in our derivation of UQ bounds for Markov processes.

Let $P$ be a probability measure on a measurable space $(\Omega,\mathcal{F})$. We consider the class of random variables $f:\Omega\to\mathbb{R}$ with a well-defined and finite moment generating function in a neighborhood of the origin:

[TABLE]

It is not difficult to prove (see e.g. DemboZeitouni ) that the cumulant generating function

[TABLE]

is a convex function, finite and infinitely differentiable in some interval $(c_{-},c_{+})$ with $-\infty\leq c_{-}<0<c_{+}\leq\infty$ and equal to $+\infty$ outside of $[c_{-},c_{+}]$. Moreover if $f\in\mathcal{E}(P)$ then $f$ has moments of all orders and we write

[TABLE]

for the centered observable of mean [math]. We will often use the cumulant generating function for the centered observable $\widehat{f}$:

[TABLE]

Recall also the relative entropy (or Kullback-Leibler divergence), defined by

[TABLE]

It has the property of a divergence, that is $R(\widetilde{P}||P)\geq 0$ and $R(\widetilde{P}||P)=0$ if and only if $\widetilde{P}=P$.

A key ingredient in our approach is the Gibbs variational principle which relates the cumulant generating function and relative entropy; see Proposition 1.4.2 in dupuis2011weak .

Proposition 1 (Gibbs Variational Principle)

Let $f:\Omega\to\mathbb{R}$ be bounded and measurable. Then

[TABLE]

As shown in chowdhary_dupuis_2013 ; DKPP , the Gibbs variational principle implies the following UQ bounds for the expected values: (a similar inequality is used in the context of concentration inequalities, see e.g. BLM , and was also used independently in BREUER20131552 ; GlassermanXu ). For a proof of the version stated here, see pages 85-86 in DKPP .

Proposition 2 (Gibbs information inequality)

If $R(\widetilde{P}||P)<\infty$ and $f\in\mathcal{E}(P)$ then $f\in L^{1}(\widetilde{P})$ and

[TABLE]

Remark 3

Note that even if $R(\widetilde{P}||P)=\infty$, the bound Eq. (17) trivially holds as long as $E_{\widetilde{P}}[f]$ is defined. To avoid clutter in the statement of our results, when $R(\widetilde{P}||P)=\infty$ we will consider the bound to be satisfied for any $f\in\mathcal{E}(P)$, even if $E_{\widetilde{P}}[f]$ is undefined.

Optimization problems of the form in Eq. (17) will appear frequently, hence we make the following definition:

Definition 1

Given any $\Lambda:\mathbb{R}\to[0,\infty]$ and $\eta>0$, let

[TABLE]

With this, we can rewrite the bound (17) as

[TABLE]

Eq. (19) is the starting point for all UQ bounds derived in this paper. From it, we see which quantities must be controlled in order to make the UQ bounds explicit: the relative entropy and the cumulant generating function. The former will be discussed in Section 6. For Markov processes, the latter can be bounded via a connection with the Feynman-Kac semigroup and functional inequalities; this connection between functional inequalities and UQ bounds is the main focus and innovation of the current work, and we begin discussing it in Section 2.4. First we recall some general properties of the bounds (19).

2.2 Properties of $\Xi^{\pm}$

The objects

[TABLE]

appearing in the Gibbs information inequality, Eq. (19), have many remarkable properties, of which we recall a few.

Proposition 3

Assume R($\widetilde{P}||P)<\infty$ and $f\in\mathcal{E}(P)$. We have:

(Divergence)* $\Xi(\widetilde{P}||P;f)$ is a divergence, i.e. $\Xi(\widetilde{P}||P,f)\geq 0$ and*

$\Xi(\widetilde{P}||P;f)=0$* if and only if either $P=\widetilde{P}$ or $f$ is constant $P$ a.s.* 2. 2.

(Linearization)* If $R(\widetilde{P}||P)$ is sufficiently small we have*

[TABLE] 3. 3.

(Tightness)* For $\eta>0$ consider $\mathcal{U}_{\eta}=\{\widetilde{P}\,;\,R(\widetilde{P}||P)\leq\eta\}$. There exists $\eta^{*}$ with $0<\eta^{*}\leq\infty$ such that for any $\eta<\eta^{*}$ there exists a measure $P^{\eta}$ with*

[TABLE]

The measure $P^{\eta}$ has the form

[TABLE]

where $c=c(\eta)$ is the unique nonnegative solution of $R(P^{\eta}||P)=\eta$.

Proof

Items 1 and 2 are proved in DKPP ; see also doi:10.1287/moor.2015.0776 for item 2. Various versions of the proof of item 3 can be found in in chowdhary_dupuis_2013 or DKPP . See Proposition 3 in DKPR for a more detailed statement of the result; see also similar results in BREUER20131552 ; doi:10.1111/mafi.12050 .∎

The tightness property in Proposition 3 is very attractive and ultimately relies on the presence of the cumulant generating function $\Lambda^{\widehat{f}}_{P}(c)$, which encodes the entire law of $f$. However, this generally this makes the bound very difficult or impossible to compute explicitly; we will need to weaken Eq. (19) to obtain more usable bounds. Functional inequalities are one tool we will employ (see Section 4). Another ingredient, which we discuss next, will be explicit bounds on the optimization problem in the definition of $\Xi^{\pm}(\Lambda,\eta)$. Such an approach was put forward in GKRW where various concentration inequalities such as Hoeffding, sub-Gaussian, and Bennett bounds are discussed. For this paper we will almost exclusively use the following Bernstein-type bound:

Lemma 1

Suppose there exist $\sigma>0$, $M^{\pm}\geq 0$ such that

[TABLE]

for all $0<c<1/M^{\pm}$. Then for all $\eta\geq 0$ we have

[TABLE]

Note that $M^{\pm}=0$ covers the case of a (one-sided) sub-Gaussian concentration bound.

Proof

Bound $\Lambda$ using Eq. (24) and solve the resulting optimization problem on $0<c<1/M^{\pm}$. ∎

From the point of view of concentration inequalities, the bound Eq. (24) is not very tight; indeed it holds for the cumulant generating function $\Lambda^{\widehat{f}}_{P}$ of any random variable $f\in\mathcal{E}(P)$, but explicit constants may be hard to come by. In the context of Markov process it has however been proved to be extremely useful, see WU2000435 ; lezaud:hal-00940906 ; cattiaux_guillin_2008 ; Guillin2009 and in particular doi:10.1137/S0040585X97986667 .

Second, we will need a linearization bound, generalizing Eq. (21):

Lemma 2

Let $\Lambda:\mathbb{R}\to[0,\infty]$ be $C^{2}$ on a neighborhood of [math], $\Lambda(0)=\Lambda^{\prime}(0)=0$, and $\Lambda^{\prime\prime}(0)>0$. Then

[TABLE]

as $\rho\searrow 0$. If $\Lambda^{\prime\prime}$ is Lipschitz at [math] then the error bound improves to $O(\eta)$.

Proof

The bound follows from Taylor expansion of $\Lambda(c)$; see the proof of Theorem 2.8 in DKPP . ∎

2.3 UQ for Markov Processes

One of the main advantages of the Gibbs information inequality, Eq. (17), over classical information inequalities (such as the Kullback-Leibler-Cziszàr inequality) is how it scales with time when applied to the distributions of processes on path space. See KRW for a detailed discussion of this issue. This strength will become apparent as we proceed.

The following assumption details the setting in which we will work for the remainder of this paper:

Assumption 1

Let $\mathcal{X}$ be a Polish space and suppose we have a time homogeneous, $\mathcal{X}$-valued, càdlàg Markov family $(\Omega,\mathcal{F},\mathcal{F}_{t},X_{t},P^{x})$, $x\in\mathcal{X}$, with transition probability kernel $p_{t}$ (see the statement of Theorem C1 in Appendix C for the precise definition of a Markov family that we use).

Also assume we have a second probability kernel $\widetilde{P}^{x}$, $x\in\mathcal{X}$, on $(\Omega,\mathcal{F})$ with $({X}_{0})_{*}\widetilde{P}^{x}=\delta_{x}$ for each $x\in\mathcal{X}$.

Remark 4

We are not assuming $(X_{t},\widetilde{P}^{x})$ are Markov processes.

One of of the families, $P^{x}$ or $\widetilde{P}^{x}$, is thought of as the base model, and the other as some alternative (or approximate) model, but which is which can vary with the application. From a mathematical perspective, the primary factors distinguishing $P^{x}$ and $\widetilde{P}^{x}$ are:

Our methods require information on the spectrum of the generator of the semigroup associated with $p_{t}$. 2. 2.

$(X_{t},P^{x})$ must be Markov, but $(X_{t},\widetilde{P}^{x})$ can be non-Markovian.

$P^{x}$ and $\widetilde{P}^{x}$ should be chosen with these points in mind; in the remainder of this paper, we will refer to the former as the base model and the latter as the alternative model.

Definition 2

Given initial distributions $\mu$ and $\widetilde{\mu}$ on $\mathcal{X}$, we also define the probability measures

[TABLE]

Note that Assumption 1 implies that $X_{t}$ is a Markov process for the space $(\Omega,\mathcal{F}_{t},P^{\mu})$ with initial distribution $\mu$ and time-homogeneous transition probabilities $p_{t}$.

We will also need the finite-time restrictions, which can be thought of as the distributions on path space up to some $T>0$:

[TABLE]

and similarly for $P^{\mu}_{T}$ and $\widetilde{P}^{\widetilde{\mu}}_{T}$. Finally, we let $E^{\mu}$ denote the expected value with respect to $P^{\mu}$ and similarly for $\widetilde{E}^{\widetilde{\mu}}$.

Now fix a bounded measurable $f:\mathcal{X}\to\mathbb{R}$ (the boundedness assumption will be relaxed later) and an invariant measure $\mu^{*}$ for $p_{t}$. As mentioned in the introduction, there are many classical techniques for studying convergence of the ergodic averages of $f$ under $P^{\mu}$ to the average in the invariant measure, $\mu^{*}[f]$. Therefore, in this paper we consider the much less studied problem of bounding the bias when the finite-time averages are computed by sampling from the alternative distribution; see Eq. (2).

2.4 UQ Bounds via the Feynman-Kac Semigroup

Due to our interest in the problem (2), we start the $P$-process in the invariant distribution $\mu^{*}$, while the $\widetilde{P}$-process is started in an arbitrary distribution $\widetilde{\mu}$.

Given a bounded measurable function $f$ on $\mathcal{X}$ and $T>0$, define the bounded and $\mathcal{F}_{T}$-measurable function

[TABLE]

Applying the Gibbs information inequality, Eq. (17), to $f_{T}$, $\widetilde{P}^{\widetilde{\mu}}_{T}$, $P^{\mu^{*}}_{T}$ and dividing by $T$ yields:

Theorem 2.1

[TABLE]

where

[TABLE]

Remark 5

Recall the definition

[TABLE]

All of the UQ bounds we obtain will be of the form

[TABLE]

for some $\Lambda:\mathbb{R}\to[0,\infty]$ and $\eta>0$; we will refer back to these equations often.

To produce a more explicit bound from Eq. (30), one needs to bound the cumulant generating function as well as the relative entropy. The latter will be addressed in Section 6. As for the former, observe that the cumulant generating function can be written

[TABLE]

Eq. (34) is related to the Feynman-Kac semigroup on $L^{2}(\mu^{*})$ with potential $V$:

[TABLE]

More specifically,

[TABLE]

and so we obtain:

Lemma 3

Under Assumption 1, for any bounded measurable $f:\mathcal{X}\to\mathbb{R}$, Eq. (33) holds with

[TABLE]

In the following two sections, we discuss how functional inequalities can be used to obtain more explicit bounds on the norm of the Feynman-Kac semigroup.

3 Bounding the Feynman-Kac Semigroup

The Lumer-Phillips theorem (a variant of the Hille-Yosida theorem) is our tool of choice for bounding the Feynman-Kac semigroup; see Chapter IX, p. 250 in Yosida1971 or Corollary 3.20 in Chapter II of engel2006short . This is the same strategy used in WU2000435 ; cattiaux_guillin_2008 ; doi:10.1137/S0040585X97986667 to obtain concentration inequalities.

First we state some of the basic properties of the Feynman-Kac semigroup, adapted from WU2000435 ; cattiaux_guillin_2008 .

Proposition 4

Let $V:\mathcal{X}\to\mathbb{R}$ be bounded and measurable and $\mu^{*}$ be an invariant probability measure for $p_{t}$. The operators $\mathcal{P}^{V}_{t}$, $t\geq 0$, defined in Eq. (35), are bounded linear operators on $L^{2}(\mu^{*})$ that form a strongly continuous semigroup.

If $(A,D(A))$ denotes the generator of $\mathcal{P}_{t}\equiv\mathcal{P}_{t}^{0}$ on $L^{2}(\mu^{*})$ then the generator of $\mathcal{P}_{t}^{V}$ on $L^{2}(\mu^{*})$ is $(A+V,D(A))$.

Remark 6

$D(A)$ consists of complex-valued functions. We will use $D(A,\mathbb{R})$ to denote the real-valued functions in the domain of $A$.

To bound the norm of the Feynman-Kac semigroup, we use the following Hilbert space version of the Lumer-Phillips theorem (again, see Yosida1971 ; engel2006short , as well as Theorem II.3.23 in engel2006short for a proof that Eq. (39) implies $A-\alpha$ is dissipative):

Proposition 5

Let $H$ be a Hilbert space and $Q(t)$ be a strongly continuous semigroup on $H$ with generator $(A,D(A))$. Suppose that there is an $\alpha\in\mathbb{R}$ such that

[TABLE]

for all $x\in D(A)$ with $\|x\|=1$. Then $\|Q(t)\|\leq e^{\alpha t}$ for all $t\geq 0$.

Propositions 4 and 5 together yield a bound on the Feynman-Kac semigroup, in terms of the generator; this result, and generalizations, were proven in WU2000435 (see Case I in the proof of Theorem 1).

Proposition 6

Let $V:\mathcal{X}\to\mathbb{R}$ be bounded and measurable, and for $t\geq 0$ consider the Feynman-Kac semigroup $\mathcal{P}_{t}^{V}:L^{2}(\mu^{*})\to L^{2}(\mu^{*})$ with generator $(A+V,D(A))$.

Define

[TABLE]

where $\langle\cdot,\cdot\rangle$ denotes the inner product on $L^{2}(\mu^{*})$.

Then the operator norm satisfies the bound

[TABLE]

for all $t\geq 0$.

Combining Eq. (42) with Eq. (36) and Eq. (30), we obtain UQ bounds that are expressed in terms of the generator of the dynamics of the baseline process and the relative entropy of the alternative process with respect to the base:

Corollary 1

Under Assumption 1, for any bounded measurable $f:\mathcal{X}\to\mathbb{R}$, the UQ bound Eq. (33) holds with

[TABLE]

From Eq. (33) we see that functional inequalities, by which we mean bounds on the generator $A$ that lead to bounds on $\kappa(V_{\pm c})$, can be used to produce UQ bounds. Also, note that the only remaining $T$-dependence is in the relative entropy term, $R(\widetilde{P}^{\widetilde{\mu}}_{T}||P^{\mu^{*}}_{T})/T$. This will often have a finite limit (the relative entropy rate) as $T\to\infty$; for examples, see Section 6.2, as well as 10.2307/3216060 , the supplementary materials to DKPP , and Appendix 1 of kipnis2013scaling . Hence Corollary 1 shows that one can expect UQ bounds that are well behaved as $T\to\infty$.

Remark 7

Proposition 6 is stated for bounded $V$, but it can be extended to certain unbounded $V$ under the additional assumption that the symmetrized Dirichlet form is closable; see Theorem 1 in WU2000435 . However, as noted in Corollary 3 in this same reference (and outlined in Proposition 7 below), that assumption can be avoided in the presence of functional inequalities by working with bounded $V$ and then taking limits; this is the strategy we employ here.

4 UQ Bounds From Functional Inequalities

In this section, we explore the consequences of several important classes of functional inequalities: Poincaré, $\log$-Sobolev, and Liapunov functions. Discussion of $F$-Sobolev inequalities, a generalization of the classical $\log$-Sobolev case, can be found in Appendix B.

4.1 Poincaré Inequality

First we consider the case where the generator satisfies a Poincaré inequality with constant $\alpha>0$, meaning:

[TABLE]

for all $g\in D(A,\mathbb{R})$. This can equivalently be written

[TABLE]

for all $g\in D(A)$, where $P^{\perp}$ is the orthogonal projector onto $1^{\perp}$.

In the presence of a Poincaré inequality, Proposition 6 is most useful when combined with the following perturbation result. A version of this result is contained in WU2000435 , but we present it here in a slightly more general form. The proof is given in Appendix A.

Lemma 4

Let $H$ be a Hilbert space, $A:D(A)\subset H\to H$ be a linear operator, and $B:H\to H$ be a bounded self-adjoint operator. Suppose there exist $D>0$ and $x_{0}\in H$ with $\|x_{0}\|=1$ such that

[TABLE]

for all $x\in D(A)$, where $P^{\perp}$ is the orthogonal projector onto $x_{0}^{\perp}$.

Define

[TABLE]

Then for any $0\leq c<D/B^{+}$ we have

[TABLE]

Remark 8

The above lemma applies to a general Hilbert space. In this paper, we will apply it to $H=L^{2}(\mu^{*})$ (with the associated $L^{2}$-inner product), and $x_{0}=1$ (constant function), in which case $P^{\perp}[f]=f-\mu^{*}[f]$.

The multiplication operator by $V_{\pm 1}$ is a bounded self-adjoint operator and $\langle V_{\pm 1}1,1\rangle=0$. Therefore Lemma 4 implies:

Lemma 5

For all $0\leq c<1/(\alpha\|(f-\mu^{*}[f])^{\pm}\|_{\infty})$ we have

[TABLE]

From this, combined with Corollary 1, we obtain the following UQ bound:

Theorem 4.1

Under Assumption 1, if $A$ satisfies the Poincaré inequality, Eq. (44), then for any bounded measurable $f:\mathcal{X}\to\mathbb{R}$ the bounds Eq. (33) and Eq. (25) hold with

[TABLE]

4.2 Poincaré Inequality for Reversible Processes

When the combination of $\mu^{*}$ and $p_{t}$ are reversible, i.e. the generator $A$ is self-adjoint on $L^{2}(\mu^{*})$, and if a Poincaré inequality, Eq. (44), also holds with constant $\alpha>0$, then one can obtain a UQ bound in terms of the asymptotic variance of $f$, instead of the variance of $f$ under $\mu^{*}$.

First, define the Poisson operator

[TABLE]

a bounded linear operator on $L^{2}_{0}(\mu^{*})\equiv\{f\in L^{2}(\mu^{*}):\mu^{*}[f]=0\}$ with norm bound $\|L\|\leq\alpha$. The asymptotic variance of $f\in L^{2}(\mu^{*},\mathbb{R})$ is defined by

[TABLE]

Note that $0\leq\sigma^{2}(f)\leq 2\alpha\operatorname{Var}_{\mu^{*}}[f]$.

Using these objects, one can obtain the following Bernstein-type bound. A simple proof appears below Remark 2.3 in doi:10.1137/S0040585X97986667 ; we outline the essential ideas below. See lezaud:hal-00940906 and Guillin2009 for similar earlier results.

Lemma 6

For all $0<c<1/(\alpha\|(f-\mu^{*}[f])^{\pm}\|_{\infty})$ we have

[TABLE]

Proof

The cases where $\sigma^{2}(f)=0$ or one of $\|(f-\mu^{*}[f])^{\pm}\|_{\infty}=0$ are trivial, so suppose not. Using the self-adjoint functional calculus, one can see that $L$ inverts $A$ on $D(A)\cap L^{2}_{0}(\mu^{*})$ and

[TABLE]

for all real-valued $f\in L^{2}_{0}(\mu^{*})$, $g\in D(A,\mathbb{R})$.

Hence, for any $g\in D(A,\mathbb{R})$ with $\|g\|_{L^{2}(\mu^{*})}=1$ and any bounded, measurable $V$ (not necessarily related to $f$ at this point):

[TABLE]

Using the Poincaré inequality and solving for $\langle-A[g],g\rangle$ gives

[TABLE]

Letting $V=V_{\pm 1}=\pm(f-\mu^{*}[f])$ in Eq. (56) and using the result to bound $\kappa$, Eq. (41), results in

[TABLE]

Eq. (53) then follows from solving the optimization problem. ∎

As with Theorem 4.1, the Bernstein-type bound, Eq. (53), implies a UQ bound:

Theorem 4.2

Under Assumption 1, if the generator satisfies the Poincaré inequality Eq. (44) and is self-adjoint on $L^{2}(\mu^{*})$ then for any bounded measurable $f:\mathcal{X}\to\mathbb{R}$ the bounds Eq. (33) and Eq. (25) hold with

[TABLE]

Other variations can be derived using a Liapunov function. First we need a couple of definitions, taken from Section 4 of doi:10.1137/S0040585X97986667 . Also, see this reference for further Liapunov function results that could likely be adapted to produce UQ bounds.

Definition 3

A measurable function $G:\mathcal{X}\to\mathbb{R}$ is in the $\mu^{*}$-extended domain of the generator, $D_{e,\mu^{*}}(A)$, if there is some measurable $g:\mathcal{X}\to\mathbb{R}$ such that $\int_{0}^{t}|g|(X_{s})ds<\infty$ $P^{\mu^{*}}$-a.s. and one $P^{\mu^{*}}$-version of

[TABLE]

is a local $P^{\mu^{*}}$-martingale.

$U\in D_{e,\mu^{*}}(A)$ is called a Liapunov function if $U\geq 1$ and there exist a measurable $\phi:\mathcal{X}\to(0,\infty)$ and $b>0$ such that

[TABLE]

As shown in doi:10.1137/S0040585X97986667 , given a Liapunov function one can derive a bound on $\kappa(V_{\pm c})$; our method then produces a corresponding UQ bound:

Theorem 4.3

In addition to Assumption 1, assume the generator, $A$, is self-adjoint on $L^{2}(\mu^{*})$ and satisfies the Poincaré inequality Eq. (44), and that we have a Liapunov function $U$ with $-A[U]/U\geq\phi-b$.

Given an observable $f\in L^{2}(\mu^{*},\mathbb{R})$ with $\|(f-\mu^{*}[f])^{\pm}/\phi\|_{\infty}<\infty$, we have the UQ bounds Eq. (33) and Eq. (25), where

[TABLE]

Proof

First let $V$ be a bounded measurable function. This part of the proof proceeds similarly to that of Lemma 6, but rather than taking the supremum of $V^{+}$ in Eq. (4.2), one instead uses Eq. (60) to compute the following bound, where $g\in D(A,\mathbb{R})$ with $\|g\|_{L^{2}(\mu^{*})}=1$:

[TABLE]

Next, use the bound found in Lemma 5.6 in Guillin2009 ,

[TABLE]

and proceed as in Lemma 6 to obtain

[TABLE]

for all $0<c<1/((1+\alpha b)\|V^{\pm}/\phi\|_{\infty})$. If $f$ is bounded then applying this to $V=f-\mu^{*}[f]$ and using Corollary 1 and Lemma 1 gives the claimed UQ bound.

For general $f\in L^{2}(\mu^{*},\mathbb{R})$ with $\|(f-\mu^{*}[f])^{\pm}/\phi\|_{\infty}<\infty$, we employ a similar method to Corollary 3 in WU2000435 : Define $V=f-\mu^{*}[f]$ and $V^{n}=V1_{|V|<n}$ (not to be confused with the $n$th power of $V$). Applying the above result to $V^{n}$ and then using Fatou’s lemma and $L^{2}$-continuity of the asymptotic variance gives

[TABLE]

Having extended the bound on the cumulant generating function to such $f$, the claimed UQ bound follows from Proposition 2. ∎

4.3 Poincaré Inequality Examples

The study of Poincaré inequalities has a long history which we do not attempt to recount here. For a detailed discussion, see wang2006functional , which covers Poincaré inequalities for both continuous-time Markov chains and diffusions. Criteria for diffusions can also be found, for example, in bakry2008 ; BAKRY2008727 .

The following example illustrates that the Bernstein-type bounds used in this paper can be sharp for Markov processes.

4.3.1 A simple Liapunov example: the $M/M/\infty$ queue.

Following doi:10.1137/S0040585X97986667 , let us consider the (simple) example of an $M/M/\infty$ queuing system which has infinitely many servers, each with a service rate $\rho$ and an arrival rate $\lambda$. The state space is $\mathbb{N}$ and the generator is given by

[TABLE]

The invariant measure $\mu^{*}$ is a Poisson distribution with parameter $\lambda/\rho$. An explicit computation shows (see e.g. Chafai ) that $\operatorname{Var}_{\mu^{*}}[\mathcal{P}_{t}f]\leq e^{-2\rho t}\operatorname{Var}_{\mu^{*}}[f]$ and thus the Poincaré constant is $1/\rho$.

To construct a Liapunov function take $U(n)=\kappa^{n}$ with $\kappa>1$; we then have

[TABLE]

and we can apply Theorem 4.3 to any function $f$ with $|f|\leq C(n+\delta)$ for some $\delta>0$.

It is instructive to consider further the case of the mean number of customers in the queue, i.e., $f=n$ and $\widehat{f}=f-\mu^{*}[f]=n-\lambda/\rho$. From Eq. (66) we obtain

[TABLE]

and thus $U$ is an eigenvector for $A+\rho(1-\kappa^{-1})\widehat{f}$ with eigenvalue $\lambda\frac{(\kappa-1)^{2}}{\kappa}$. By the Perron-Frobenius theorem and Rayleigh’s principle we obtain that

[TABLE]

is the maximal eigenvalue of $A+c\widehat{f}$ and thus $\Lambda\left(\rho(1-\kappa^{-1})\right)=\lambda\frac{(\kappa-1)^{2}}{\kappa}$ or equivalently $\Lambda(c)=\frac{\lambda c^{2}}{\rho^{2}(1-c\rho^{-1})}$. Since $A\widehat{f}(n)=\lambda-\rho n$ we can solve the Poisson equation: $(-A)^{-1}\widehat{f}=\widehat{f}/\rho$ and thus the asymptotic variance is $\sigma^{2}(f)=2\langle(-A)^{-1}\widehat{f}\,,\,\widehat{f}\rangle=2\rho^{-1}\operatorname{Var}_{\mu^{*}}[f]=2\lambda/\rho^{2}$. As a consequence we have

[TABLE]

which shows that Bernstein bounds can be sharp in the context of Markov processes, contrary to the IID setting.

4.3.2 Poincaré Inequality from Exponential Convergence

It is well-known that, when the generator, $A$, is self-adjoint, a Poincaré inequality is equivalent to exponential convergence in the $L^{2}(\mu^{*})$-norm. Here, we discuss a method for deriving a Poincaré inequality from exponential convergence in alternative norms.

First, note that one only needs exponential $L^{2}$-convergence on a subset with dense span to conclude a Poincaré inequality (see Lemma 1.2 in cattiaux:hal-00461085 ):

Lemma 7

*Suppose $(A,D(A))$ is self-adjoint, $F\subset L^{2}(\mu^{*})$ has dense span, and there exists $\alpha>0$ such that the following holds:

For every $f\in F$ there exists $C_{f}\geq 0$ such that*

[TABLE]

Then a Poincaré inequality, Eq. (44), holds with constant $\alpha$.

The following result shows how to obtain a Poincaré inequality (with an explicit constant) from exponential convergence in a pair of weighted norms.

Theorem 4.4

Suppose $(A,D(A))$ is self-adjoint, and $W:\mathcal{X}\to[1,\infty)$ is measurable. Define the following norms on measurable functions $\phi:\mathcal{X}\to\mathbb{R}$ and signed measures $\pi$ on $\mathcal{X}$:

[TABLE]

Suppose we have $\lambda\geq 0,\rho\geq 0$ with at least one nonzero, and that for every bounded measurable $h:\mathcal{X}\to[0,\infty)$ with $\int hd\mu=1$ there exist $C_{h},D_{h}\in[0,\infty)$ such that for all $t\geq 0$:

[TABLE]

and the measure $d\nu=hd\mu^{*}$ satisfies

[TABLE]

where $\mathcal{P}^{\dagger}_{t}$ denotes the action of the semigroup $p_{t}$ on measures.

Then $A$ satisfies the Poincaré inequality

[TABLE]

Proof

The proof is similar to that of Theorem 2.1 in BAKRY2008727 . The key is to take $h$ as above, let $d\nu=hd\mu^{*}$, use symmetry of $\mathcal{P}_{t}$ to compute

[TABLE]

and then apply Lemma 7. ∎

Exponential convergence in norms of the form $|\cdot|_{W}$ can be obtained from the existence of an appropriate Liapunov function (see HarierNotes ; HairerMattingly2011 ), making Theorem 4.4 a practical method for obtaining Poincaré inequalities.

Remark 9

The proof of Lemma 7 can be generalized to only require Eq. (71) to hold along a sequence $t_{n}^{f}$ converging to $\infty$. Hence, Theorem 4.4 can also be generalized to only require Eq. (73) and Eq. (74) along a common sequence $t_{n}^{h}\to\infty$.

4.4 $\log$-Sobolev Inequalities

Next consider the $\log$-Sobolev inequality with constant $\beta>0$:

[TABLE]

for all $g\in D(A,\mathbb{R})$ with $\|g\|_{L^{2}(\mu^{*})}=1$.

We will employ the following generalization of the Feynman-Kac semigroup for (possibly) unbounded potentials. The subsequent proposition was shown in Corollary 4 in WU2000435 . For completeness purposes, we outline the proof.

Proposition 7

Let $A$ be the generator of $\mathcal{P}_{t}$ and $\mu^{*}$ be an invariant measure for the adjoint semigroup, $\beta>0$, and assume the $\log$-Sobolev inequality, Eq. (77), holds for $\mu^{*}$ with constant $\beta$.

Finally, suppose that $V\in L^{1}(\mu^{*})$ with $\int e^{\beta V}d\mu^{*}<\infty$. Then $\mathcal{P}_{t}^{V}:L^{2}(\mu^{*})\to L^{2}(\mu^{*})$, defined by

[TABLE]

are well-defined linear operators and the operator norm satisfies the bound

[TABLE]

Proof

First assume $V$ is bounded. Eq. (42) gives $\|\mathcal{P}_{t}^{V}\|\leq e^{t\kappa(V)}$. Applying the $\log$-Sobolev inequality together with the Gibbs variational principle, Eq. (16), we obtain

[TABLE]

which proves the claim.

The case of unbounded $V$ satisfying the assumptions of the theorem is obtained by letting $V^{n}=V1_{|V|\leq n}$, and then using Fatou’s lemma, the result for bounded $V$, and dominated convergence to compute

[TABLE]

∎

Using Proposition 7, a UQ bound of the form Eq. (33) can be derived that covers a class of unbounded observables:

Theorem 4.5

In addition to Assumption 1, assume the $\log$-Sobolev inequality, Eq. (77), holds and we have an observable $f\in L^{1}(\mu^{*},\mathbb{R})$ and $c_{-}<0<c_{+}$ such that for all $c\in(c_{-},c_{+})$:

[TABLE]

Then a UQ bound of the form Eq. (33) holds with

[TABLE]

In addition, the asymptotic result Eq. (26) holds with

[TABLE]

Proof

The bound Eq. (79) implies $E^{\mu^{*}}[\exp(cf_{T})]<\infty$ for $c\in(c_{-},c_{+})$, hence $f_{T}\in\mathcal{E}(P^{\mu^{*}}_{T})$ and the Gibbs information inequality, Eq. (17), applies. As in Eq. (36), the cumulant generating function can be bounded using the Feynman-Kac semigroup bound, Eq. (79). Combining this with Eq. (17) yields a bound of the form Eq. (33), with $\Lambda$ as defined in Eq. (84). ∎

The ideas in this section can be extended to $F$-Sobolev inequalities; see Appendix B.

4.4.1 Example: Diffusions

Let $V$ be a $C^{2}$ potential, bounded below, and growing sufficiently fast at infinity. Consider the diffusion with generator $A=\Delta-\nabla V\cdot\nabla$ and invariant measure $\mu^{*}(dx)=e^{-V(x)}dx$. First, it is useful to note that a $\log$-Sobolev inequality with constant $\beta$ implies a Poincaré inequality with constant $\alpha=\beta/2$ rothaus1978 . In CarlenLoss , the following sufficient condition for a $\log$-Sobolev inequality was obtained:

Suppose $A$ satisfies a Poincaré inequality with constant $\alpha$ (references on Poincaré inequalities can be found in Section 4.3) and that

[TABLE]

Then $A$ satisfies a $\log$-Sobolev inequality with constant

[TABLE]

As a second example, if the Hessian of $V$ is bounded below,

[TABLE]

for some $\beta>0$ (unrelated to the $\beta$ in Eq. (87)) then a $\log$-Sobolev inequality holds with constant $\beta$ BakryEmery . A UQ bound corresponding to the associated Poincaré inequality with constant $\alpha\equiv\beta/2$ was given in the introduction in Eq. (10).

5 Functional Inequalities and UQ for Discrete-Time Markov Processes

In this section we show how the above framework can be applied to obtain UQ bounds for invariant measures of discrete-time Markov processes.

Again, let $\mathcal{X}$ be a Polish space, and suppose we have one-step transition probabilities $p(x,dy)$ and $\widetilde{p}(x,dy)$ on $\mathcal{X}$ with invariant measures $\mu^{*}$ and $\widetilde{\mu}^{*}$ respectively. Assume that $R(\widetilde{\mu}^{*}||\mu^{*})<\infty$.

Define the bounded linear operator $\mathcal{P}$ on $L^{2}(\mu^{*})$,

[TABLE]

and similarly for $\widetilde{\mathcal{P}}$ on $L^{2}(\widetilde{\mu}^{*})$.

We obtain UQ bounds for expectations in $\mu^{*}$ and $\widetilde{\mu}^{*}$ by constructing continuous-time processes with these same invariant distributions. Specifically, in Appendix C (see Theorem C1) we obtain càdlàg Markov families $(\Omega,\mathcal{F},\mathcal{F}_{t},X_{t},\{P^{x}\}_{x\in\mathcal{X}})$ and $(\Omega,\mathcal{F},\mathcal{F}_{t},X_{t},\{\widetilde{P}^{x}\}_{x\in\mathcal{X}})$, whose transition probabilities $p_{t}$ and $\widetilde{p}_{t}$, respectively, (not to be confused with $p$ and $\widetilde{p}$) satisfy the following:

$\mu^{*}$ is invariant for $p_{t}$ for all $t\geq 0$, and similarly for $\widetilde{\mu}^{*}$ and $\widetilde{p}_{t}$ (see Theorem C2). 2. 2.

The continuous-time semigroup, $\mathcal{P}_{t}$, on $L^{2}(\mu^{*})$ constructed from $p_{t}$ is

[TABLE]

Specifically, $\mathcal{P}_{t}$ has bounded generator $A=\mathcal{P}-I$ (see Theorem C2). Note that we will also refer to $A$ as the generator of the discrete-time Markov process. 3. 3.

The relative entropy rate of the continuous-time process can be bounded by the relative entropy of the discrete-time process as follows:

[TABLE]

for all $T>0$ (see Theorem C4 and Corollary C1).

Remark 10

While the above construction, and the computation of the relative entropy, is standard for countable state spaces (see the discussion in Section 6.1), for our purposes it is necessary to work with general state spaces; to the best of our knowledge, the relative entropy bound Eq. (91) is new in this case.

General state spaces are of interest, for example, when one is working with Markov chain Monte Carlo samplers, $p(x,dy)$ and $\widetilde{p}(x,dy)$, for measures, $\mu^{*}$ and $\widetilde{\mu}^{*}$ respectively, on $\mathbb{R}^{n}$. In this setting, to use our UQ method, one can construct the ancillary continuous-time Markov chain on $\mathbb{R}^{n}$, as outlined in Appendix C, and then apply the relative entropy bound Eq. (91).

The Markov families $P^{x}$ and $\widetilde{P}^{x}$, obtained via the above construction, satisfy Assumption 1. Hence, if the generator $\mathcal{P}-I$ satisfies any of the functional inequalities covered in Section 3 then the general results therein imply UQ bounds for expectations in the invariant measures $\mu^{*}$ and $\widetilde{\mu}^{*}$, with Eq. (91) providing a bound on the relative entropy rate.

Remark 11

Note that here, we must take $\widetilde{\mu}=\widetilde{\mu}^{*}$ for the bounds to apply to the original discrete-time process, otherwise one obtains UQ bounds for ergodic averages of $f(X_{t})$ under the auxiliary continuous-time Markov family.

For example, a Poincaré inequality for the generator $\mathcal{P}-I$,

[TABLE]

implies that for any bounded measurable $f:\mathcal{X}\to\mathbb{R}$, we have

[TABLE]

This follows from Theorem 4.1, after taking $T\to\infty$ (recall the assumption $R(\widetilde{\mu}^{*}||\mu^{*})<\infty$).

We illustrate these discrete-time UQ bounds with a pair of examples:

5.1 Example: Random Walk on a Hypercube

Consider the symmetric random walk on the $d$-dimensional hypercube $\mathcal{X}=\{-1,1\}^{d}$ i.e. the transition probabilities are defined by uniformly randomly selecting a coordinate, $i\in\{1,...,d\}$, and then independently and uniformly selecting the sign, $1$ or $-1$, with which to update the selected component.

The uniform measure, $\mu^{*}$, on $\mathcal{X}$ is invariant and the process is reversible on $(\mathcal{X},\mu^{*})$. The eigenvalues and eigenvectors of the transition matrix can be found explicitly; see Example 12.15 in levin2017markov . In particular, the second largest eigenvalue is $\lambda_{2}=1-1/d$, hence we obtain the following Poincaré inequality:

[TABLE]

Assuming $R(\widetilde{\mu}^{*}||\mu^{*})<\infty$, we then obtain the UQ bound Eq. (93) with $\alpha=d$.

5.2 Example: Exclusion Chain

Derivation of functional inequalities for many discrete-time Markov processes can be found in diaconis1996 . Here we investigate the resulting UQ bounds for one of these examples; see Section 4.6 in the above reference and also Diaconis1993 for further details and proofs regarding this example.

Let $(V,E)$ be a symmetric, connected graph with $n$ vertices. Let $d(x)$ be the degree of a vertex $x\in V$ and $d_{0}=\max_{x}d(x)$. Fix $r\leq n$. The $r$-exclusion process is a Markov chain with state space being the set of cardinality $r$ subsets of $V$. Informally stated, the transition probabilities are defined as follows: Given an $r$-subset $A$ (i.e., state of the chain), pick an element $x\in A$ with probability proportional to its degree. Uniformly randomly pick a vertex $y$ out of all those connected with $x$. If $y$ is not in $A$ then transition to the new state $(A\setminus\{x\})\cup\{y\}$. Otherwise, the chain remains at the set $A$.

For each $(x,y)\in V\times V$, fix a path $\gamma_{x,y}$ from $x$ to $y$ in the graph and let $|\gamma_{x,y}|$ be its length. Define

[TABLE]

The generator of this Markov chain satisfies both a Poincaré inequality and a $\log$-Sobolev inequality with respective constants being

[TABLE]

Then, assuming $R(\widetilde{\mu}^{*}||\mu^{*})<\infty$, the above Poincaré inequality implies the UQ bound Eq. (93) with $\alpha$ as in Eq. (96), and the $\log$-Sobolev inequality results in

[TABLE]

with $\beta$ and $\eta$ as in Eq. (96) and Eq. (93) respectively.

6 Bounding the Relative Entropy Rate

For any $\eta>0$, the results derived in the previous sections provide UQ bounds over the class of all alternative models that satisfy a relative entropy bound of the form

[TABLE]

In this section, we study in more detail the dependence of $H_{T}$ on $T$ and on the models $\widetilde{P}^{\widetilde{\mu}}$ and $P^{\mu^{*}}$. Specifically, we derive upper bounds on $H_{T}$ in various settings that can be substituted for $H_{T}$ in the general UQ bound Eq. (33). Here, it will make little difference whether the initial distribution for the $P$-process is invariant or not, so we no longer make that assumption when deriving the relative entropy bounds; $\mu$ will denote an arbitrary initial distribution.

Deriving bounds on the relative entropy is a very application-specific problem. We will cover several examples in detail: continuous-time Markov chains, semi-Markov processes, change of drift in SDEs, and numerical methods for SDEs with additive noise.

6.1 Example: Continuous-Time Markov Chains

Let $\mathcal{X}$ be a countable set, $P^{\mu}$, $\widetilde{P}^{\widetilde{\mu}}$ be probability measures on $(\Omega,\mathcal{F})$ and $X_{t}:\Omega\to\mathcal{X}$ such that $P^{\mu}$ (resp. $\widetilde{P}^{\widetilde{\mu}}$) makes $(\Omega,\mathcal{F},X_{t})$ a continuous-time Markov chain (CTMC) with transition probabilities $a(x,y)$ (resp. $\widetilde{a}(x,y)$), jump rates $\lambda(x)$ (resp. $\widetilde{\lambda}(x)$), and initial distribution $\mu$ (resp. $\widetilde{\mu}$). Let $\mathcal{F}_{t}$ be the natural filtration for $X_{t}$ and $X^{J}_{n}$ be the embedded jump chain with jump times $J_{n}$.

Suppose $\widetilde{\mu}\ll\mu$, $\lambda$ and $\widetilde{\lambda}$ are positive and bounded above, and for all $x,y\in\mathcal{X}$ we have $a(x,y)=0$ iff $\widetilde{a}(x,y)=0$. Then for any $T>0$ we have $\widetilde{P}^{\widetilde{\mu}}|_{\mathcal{F}_{T}}\ll P^{\mu}|_{\mathcal{F}_{T}}$ and

[TABLE]

To simplify further, if $\widetilde{\mu}=\widetilde{\mu}^{*}$ is an invariant measure then

[TABLE]

See the supplementary materials to DKPP and Proposition 2.6 in Appendix 1 of kipnis2013scaling for details regarding these results.

6.2 Example: Semi-Markov Processes

As we have noted previously, our results require $(X_{t},P^{x})$ to be Markov, but do not require the alternative model $(X_{t},\widetilde{P}^{x})$ to be Markov. Here we discuss one such class of examples, that of a semi-Markov perturbation of a continuous-time Markov chain.

Semi-Markov processes are continuous-time jump processes with memory (i.e., with nonexponential waiting times). Such a process is defined by a jump chain, $X^{J}_{n}$, jump times, $J_{n}$, and waiting times (i.e. jump intervals), $\Delta_{n+1}\equiv J_{n+1}-J_{n}$, that satisfy

[TABLE]

$\widetilde{Q}_{x,y}(t)$ is called the semi-Markov kernel; see, for example, janssen2006applied ; limnios2012semi for further details. Note that a continuous-time Markov chain with embedded jump Markov-chain transition probabilities $a(x,y)$ and jump rates $\lambda(x)$ is described by the semi-Markov kernel

[TABLE]

A semi-Markov perturbation of Eq. (101) with the same embedded jump Markov-chain but with modified (nonexponential) waiting times is described by a kernel of the form

[TABLE]

Remark 12

Phase-type distributions constitute a useful semiparametric description of such alternative waiting-time distributions, going beyond the exponential case to describe systems with memory; see doi:10.1002/asm.3150100403 ; bladt2017matrix for details.

The relative entropy rate,

[TABLE]

between semi-Markov processes was obtained in 10.2307/3216060 under the appropriate ergodicity assumptions. When the base process has the form Eq. (101) and the alternative process has the form Eq. (102), the relative entropy rate can be expressed in terms of the relative entropy of the waiting-time distributions:

[TABLE]

where $\pi$ is the invariant distribution for the Markov chain $a(x,y)$.

Remark 13

The quantity $\widetilde{m}_{\pi}$ is the mean sojourn time under the invariant distribution, $\pi$, and $\sum_{x}\pi(x)R(\widetilde{H}_{x}||H_{x})$ can be thought of as the mean relative entropy of a single jump (comparing the alternative and base model waiting-time distributions). The formula for $\eta$, Eq. (104), therefore has the intuitive meaning of an information loss per unit time.

6.2.1 Semi-Markov Perturbations of a $M/M/\infty$-Queue

As a concrete example, we consider semi-Markov perturbations of an $M/M/\infty$-queue with service rate $\rho$ and with an arrival rate $\lambda$. The embedded jump Markov-chain is given by

[TABLE]

and the waiting-times are exponentially distributed with jump rates

[TABLE]

Eq. (105), has invariant distribution

[TABLE]

Taking $T\to\infty$ in Eq. (30) and using Eq. (69), Eq. (70), and Eq. (104) we therefore obtain the following asymptotic upper bound on the average queue length in the alternative model:

[TABLE]

where

[TABLE]

Note that the only ingredient from the alternative model that is needed in Eq. (108) is $\widetilde{H}_{x}$, and given this, the bounds are generally straightforward to evaluate.

6.3 Example: Change of Drift for SDEs

Next, consider the case where $P^{x}$ and $\widetilde{P}^{x}$ are the distributions on $C([0,\infty),\mathbb{R}^{n})$ of the solution flows $X_{t}^{x}$ and $\widetilde{X}_{t}^{x}$ of a pair of SDEs. More precisely:

Assumption 2

Assume:

$X^{x}_{t}$* and $\widetilde{X}^{x}_{t}$ are weak solutions to the $\mathbb{R}^{n}$-valued SDEs, on filtered probability spaces satisfying the usual conditions karatzas2014brownian :*

[TABLE]

where $W_{t}$ and $\widetilde{W}_{t}$ are $m$-dimensional Wiener processes. We let $P$ and $\widetilde{P}$ denote the probability measures of the respective spaces where the SDEs are defined.

Here we think of $b:\mathbb{R}^{n}\to\mathbb{R}^{n}$ and $\sigma:\mathbb{R}^{n}\to\mathbb{R}^{n\times m}$ as the measurable drift and diffusion for the base process, and we assume the modified drift has the form $\widetilde{b}=b+\sigma\beta$ for some measurable $\beta:\mathbb{R}^{n}\to\mathbb{R}^{m}$. 2. 2.

$X^{x}_{t}$* and $\widetilde{X}^{x}_{t}$ are jointly continuous in $(t,x)$.* 3. 3.

$X^{x}_{t}$* satisfies the following flow property:*

For any bounded, measurable $G:C([0,\infty),\mathbb{R}^{n})\to\mathbb{R}$, we have

[TABLE] 4. 4.

$X^{x}_{t}$* and $\beta$ satisfy the Novikov condition*

[TABLE]

for all $x\in\mathbb{R}^{n}$, $T>0$. 5. 5.

For every $T>0$, solutions to Eq. (111) satisfy uniqueness in law, up to time $T$.

Given this, we define $P^{x}=(X^{x})_{*}P$ and $\widetilde{P}^{x}=(\widetilde{X}^{x})_{*}\widetilde{P}$ i.e. the distributions on path space, with the Borel sigma algebra:

[TABLE]

where $\pi_{t}$ is evaluation at time $t$. Finally, define $X_{t}\equiv\pi_{t}$. One can easily show that the above properties are sufficient to guarantee that Assumption 1 holds.

Remark 14

The existence of flows of solutions $X^{x}_{t}$ and $\widetilde{X}^{x}_{t}$ that satisfy the above conditions is guaranteed, for example, if $b$ and $\sigma$ satisfy a linear growth bound

[TABLE]

and the following local Lipschitz bound:

For each $\ell$ there exists $K_{\ell}$ such that

[TABLE]

on $\|x\|,\|y\|\leq\ell$, and if $\beta:\mathbb{R}^{n}\to\mathbb{R}^{m}$ is also bounded and locally Lipschitz.

Fixing $T>0$, Girsanov’s theorem allows one to bound the relative entropy, $R(\widetilde{P}^{x}_{T}||P^{x}_{T})$, that appears in the UQ bound Eq. (33). See the supplementary materials to DKPP for more details:

Lemma 8

Under Assumption 2, and given initial distributions $\mu$ and $\widetilde{\mu}$ for the base and alternative models respectively, we have

[TABLE]

6.4 Example: Euler-Maruyama Methods for SDEs with Additive Noise

As the final example, we consider SDEs with additive noise, approximated by a (generalized) Euler-Maruyama (EM) method.

Assumption 3

Let $W_{t}$ be an $n$-dimensional Wiener process on filtered probability spaces satisfying the usual conditions, $b:\mathbb{R}^{n}\to\mathbb{R}^{n}$ satisfy the linear boundedness and local Lipschitz properties as described in Remark 14, and $X^{x}_{t}$ be the strong solutions to the SDEs

[TABLE]

Recall that versions can be chosen so that $X^{x}_{t}$ is jointly continuous in $(t,x)$ and $X^{x}_{t}$ satisfies the flow property Eq. (112).

We fix $\Delta t>0$ and assume we are given a measurable vector field $\widetilde{b}_{\Delta t}:\mathbb{R}^{n}\to\mathbb{R}^{n}$ (the drift for the generalized EM method). We define the approximating process $\widetilde{X}^{x}_{0}=x$,

[TABLE]

We emphasize that, for the purposes of employing the theory we have developed (i.e., to employ functional inequalities satisfied by the generator of Eq. (117)), it is necessary to extend $\widetilde{X}^{x}_{t}$ to all $t\geq 0$, and not just define it at the mesh points $j\Delta t$.

Let $P$ denote the probability measure on the space where the SDE is defined. Similarly to the previous example, we define $P^{x}=(X^{x})_{*}P$ and $\widetilde{P}^{x}=(\widetilde{X}^{x})_{*}P$, probability measures on

[TABLE]

Assumption 3 is sufficient to guarantee that Assumption 1 holds. The chain rule for relative entropy (see Theorem C.3.1 in dupuis2011weak ) can be used to obtain

[TABLE]

Let $T=N\Delta t$ for $N\in\mathbb{Z}^{+}$. For the purposes of bounding the relative entropy term

[TABLE]

it will be useful to define the Polish space $\mathcal{Y}\equiv C([0,\Delta t],\mathbb{R}^{n})$ and the following one-step transition probabilities for a discrete-time Markov process on $\mathcal{Y}$:

[TABLE]

Letting $\otimes_{1}^{N}q$ denote the composition on $\mathcal{Y}^{N}$, the Markov property implies

[TABLE]

for all $x\in\mathbb{R}^{n}$, and similarly for $\widetilde{q}$, $\widetilde{X}^{x}$.

Therefore, using the chain rule for relative entropy again, we obtain

[TABLE]

for all $x\in\mathbb{R}^{n}$. Hence we arrive at:

Lemma 9

[TABLE]

The one-step relative entropy can be bounded via Girsanov’s theorem, similarly to Lemma 8; on each time interval of length $\Delta t$, the tilde process is simply the solution to an SDE with constant drift and additive noise.

Lemma 10

Under Assumption 3

[TABLE]

where $\widetilde{p}^{\Delta t}_{j}(x,dy)=(\widetilde{X}^{x}_{j\Delta t})_{*}P$.

6.4.1 Euler-Maruyama Error Bounds

We end this section by specializing the results to the Euler-Maruyama method, $\widetilde{b}_{\Delta t}\equiv b$.

If we assume $b$ is $C^{1}$ with bounded first derivative and $Db$ is $L$-Lipschitz then Taylor expanding $b$ gives

[TABLE]

and therefore

[TABLE]

where $\|\cdot\|_{F}$ denotes the Frobenius matrix norm.

This is not the tightest possible bound and alternatives can be obtained by Taylor expanding further, but it gives an idea of the type of result that can be obtained under various smoothness assumptions on $b$.

If the initial distributions have the form $d\widetilde{\mu}=e^{-\widetilde{\phi}}dx$ and $d\mu=e^{-\phi}dx$, where $\widetilde{\phi}$ and $\phi$ are known functions, then the relative entropy term takes the form

[TABLE]

If one can efficiently sample from $\widetilde{\mu}$ then Eq. (128) and Eq. (129) can be estimated via Monte Carlo methods, providing UQ bounds that involve a mixture of a priori and a posteriori data.

Appendix A Proof of the Perturbation Bound

Lemma A1

Let $H$ be a Hilbert space, $A:D(A)\subset H\to H$ be a linear operator, and $B:H\to H$ be a bounded self-adjoint operator. Suppose there exist $D>0$ and $x_{0}\in H$ with $\|x_{0}\|=1$ such that

[TABLE]

for all $x\in D(A)$, where $P^{\perp}$ is the orthogonal projector onto $x_{0}^{\perp}$.

Define

[TABLE]

Then for any $0\leq c<D/B^{+}$ we have

[TABLE]

Proof

Let $x\in D(A)$ with $\|x\|=1$. Define $a=\langle x_{0},x\rangle$. (Here we will use the convention of linearity in the second argument). We have $\|P^{\perp}x\|^{2}=1-|a|^{2}$, and so $|a|\leq 1$ with equality if and only if $P^{\perp}x=0$.

We can decompose $x=ax_{0}+\sqrt{1-|a|^{2}}v$, where either $v=0$ and $|a|=1$ if $P^{\perp}x=0$ or $v=P^{\perp}x/\sqrt{1-|a|^{2}}$ and $\|v\|=1$ if $P^{\perp}x\neq 0$. In either case, $v\perp x_{0}$.

With this, we have

[TABLE]

where $B^{+}$ is given by Eq. (131).

Restricting to $0\leq c<D/B^{+}$, if $\|Bx_{0}\|=0$ then the supremum is [math] and we have the result. Otherwise, the supremum is positive and we can use $\beta\leq 1/\beta$ and then change variables in the supremum to $r=\sqrt{1-\beta^{2}}/\beta$, thereby obtaining

[TABLE]

∎

The previous lemma is closest in spirit to the probabilistic application, as $\|Bx_{0}\|^{2}$ plays the role of the variance. However, one can work with non-self-adjoint perturbations, if one instead uses the definition

[TABLE]

and makes the replacement $\|Bx_{0}\|\to\|(B+B^{*})x_{0}/2\|$ in Eq. (132). The proof is similar.

Appendix B $F$-Sobolev Inequalities

Proposition 7 can be generalized to the $F$-Sobolev case; see the proof of Theorem 2.3 in cattiaux_guillin_2008 :

Proposition B1

Let $A$ be the generator of $\mathcal{P}_{t}$ and $\mu^{*}$ be an invariant measure. Suppose we have a function $F:(0,\infty)\to\mathbb{R}$ satisfying the following:

$F$* is strictly increasing,* 2. 2.

$F$* is concave (hence continuous),* 3. 3.

$F(1)=0$, 4. 4.

$F(x)\to\infty$* as $x\to\infty$,* 5. 5.

$F(xy)\leq F(x)+F(y)$* for all $x,y\geq 0$.*

(Note that this implies $F^{-1}:(F(0^{+}),\infty)\to(0,\infty)$ exists, is increasing, convex, and continuous.)

Assume the $F$-Sobolev inequality holds for $\mu^{*}$:

[TABLE]

Finally, suppose that $V\in L^{1}(\mu^{*})$ with $V>F(0^{+})$ and $\int F^{-1}(V)d\mu^{*}<\infty$. Then $\mathcal{P}_{t}^{V}:L^{2}(\mu^{*})\to L^{2}(\mu^{*})$, defined by

[TABLE]

are well-defined linear operators and the operator norm satisfies the bound

[TABLE]

Note that if $F(0^{+})=-\infty$ then certain unbounded observables are allowed, namely those that satisfy the integrability condition Eq. (81).

This proposition leads to a UQ bound of the form, Eq. (33). The proof is analogous to the $\log$-Sobolev case from Section 4.4.

Theorem B1

In addition to Assumption 1, assume the $F$-Sobolev inequality, Eq. (136), holds for some function, $F$, having the properties listed in Proposition B1, $f\in L^{1}(\mu^{*},\mathbb{R})$, and there exists $c_{-}<0<c_{+}$ such that, for all $c\in(c_{-},c_{+})$:

[TABLE]

Then a UQ bound of the form Eq. (33) holds with

[TABLE]

In addition, if $\operatorname{Var}_{\mu^{*}}[f]>0$, $F$ and $F^{-1}$ are smooth, $F^{\prime}(1)>0$, $(F^{-1})^{\prime\prime}(0)>0$, and $c\to\mu^{*}[F^{-1}(V_{c})]$ is smooth on a neighborhood of [math] and can be differentiated under the integral then Eq. (26) holds with

[TABLE]

Appendix C Continuous-Time Jump Processes on General State Spaces

As discussed in Section 5, to apply our UQ bounds to the invariant measure of discrete-time Markov processes, $\mathcal{P}$ and $\widetilde{P}$, one needs to construct an ancillary continuous-time Markov process with generators $\mathcal{P}-I$ and $\widetilde{\mathcal{P}}-I$, and also compute the associated relative entropy. While the construction of continuous-time Markov processes from their generators is well known (see, for example, Chapter 4.2 in ethier2009markov or Chapter 3.3 in liggett2010continuous ), and the relative entropy computation is known in the countable-state-space case (see Section 6.1), we require a formula for the relative entropy in the general case of a Polish state space. To the best of our knowledge, this computation is new, though it closely mirrors the established results; hence we present only a short outline.

In order to obtain an explicit formula for the Radon-Nikodym derivative, and thereby compute the relative entropy, it is useful to utilize an explicit construction, as in the countable-state-space case (see, for example, Appendix 1 of kipnis2013scaling ), rather than invoking more general existence theorems:

Let $(\mathcal{X},\mathcal{B}_{\mathcal{X}})$ be a Polish space and $p(x,dy)$ be a probability kernel on $\mathcal{X}$. Given $\lambda>0$ define the probability kernel, $p^{J}$, on the Polish space $(\mathcal{X}\times(0,\infty),\mathcal{B}_{\mathcal{X}}\bigotimes\mathcal{B}_{(0,\infty)})$:

[TABLE]

For any probability measure $\pi$ on $(\mathcal{X},\mathcal{B}_{\mathcal{X}})$, let $P^{\pi}$ (for $\pi=\delta_{x}$ we simply write $P^{x}$) be the unique probability measure on $(\Omega,\mathcal{F})\equiv(\prod_{n=0}^{\infty}(\mathcal{X}\times(0,\infty)),\bigotimes_{n=0}^{\infty}(\mathcal{B}_{\mathcal{X}}\bigotimes\mathcal{B}_{(0,\infty)}))$ generated by the transition probabilities $p^{J}$ and initial distribution $\pi\times(\lambda e^{-\lambda t}dt)$. Also, define the jump process, jump intervals, and jump times:

[TABLE]

where $\pi_{i}$ denote projections onto components. The jump rates are positive constants, so one obtains $J_{n}(\omega)\to\infty$ a.s. as $n\to\infty$.

$(X_{n}^{J},\Delta_{n})$ is a Markov process under $P^{\pi}$ with transition probabilities $p^{J}$ and initial distribution $\pi\times(\lambda e^{\lambda t}dt)$. Use this to define the associated càdlàg process

[TABLE]

and the probability kernels on $\mathcal{X}$,

[TABLE]

Finally, let $\mathcal{F}_{t}$ be the natural filtration for $X_{t}$.

With this setup, we have the following theorem:

Theorem C1

$(\Omega,\mathcal{F},\mathcal{F}_{t},X_{t},P^{x})$, $x\in\mathcal{X}$, is a càdlàg Markov family with transition probabilities $p_{t}$. More specifically:

$(\Omega,\mathcal{F},\mathcal{F}_{t})$, $t\geq 0$ is a filtered probability space and $X_{t}$ is an $\mathcal{X}$-valued, $\mathcal{F}_{t}$-adapted, càdlàg process. 2. 2.

$p_{t}(x,dy)$, $t\geq 0$, are time homogeneous transition probabilities on $\mathcal{X}$. 3. 3.

$P^{x}$, $x\in\mathcal{X}$ are probability measures with $(X_{0})_{*}P^{x}=\delta_{x}$ for each $x\in\mathcal{X}$. 4. 4.

For every measurable set $F$, $x\to P^{x}(F)$ is universally measurable. 5. 5.

For each $x\in\mathcal{X}$, $P^{x}(X_{t+s}\in B|\mathcal{F}_{s})=p_{t}(X_{s},B)$ $P^{x}$-a.s. In particular, $p_{t}(x,B)=P^{x}(X_{t}\in B)$.

One also obtains realizability of the semigroup $\exp(t\lambda(\mathcal{P}-I))$ by a probability kernel:

Theorem C2

If $\mu^{*}$ is an invariant measure for $p$ then $\mu^{*}$ is invariant for ${p}_{t}$ for all $t\geq 0$ and the bounded linear operators on $L^{2}(\mu^{*})$,

[TABLE]

satisfy

[TABLE]

for all $t\geq 0$, where the right-hand side is the operator exponential for bounded operators on $L^{2}(\mu^{*})$.

These results are all straightforward to prove by using the same strategy as the discrete-state-space case.

The formula for the Radon-Nikodym derivative for two measures constructed as above is also straightforward; the only complication is that here, the jump chain $(X_{n}^{J},\Delta_{n})$ is generally not recoverable from $X_{t}$; specifically, the $J_{n}$ are not $\mathcal{F}_{t}$-stopping times (this is because ‘jumps’ do not necessarily change the state, unlike the construction commonly used when the state space is discrete). Hence, we must derive a formula for the Radon-Nikodym derivative on the enlarged filtration

[TABLE]

Otherwise, the computation closely mirrors the discrete case (again, see kipnis2013scaling ) and one arrives at:

Theorem C3

Suppose we have probability measures $\widetilde{\mu}$, $\mu$ and probability kernels $\widetilde{p}(x,dy)$, $p(x,dy)$ on $\mathcal{X}$. Assume that $\widetilde{\mu}\ll\mu$ and $\widetilde{p}(x,\cdot)\ll p(x,\cdot)$ for $\widetilde{\mu}$ a.e. $x$. In particular, we have $h\in L^{+}(\mathcal{X}\times\mathcal{X})$ such that

[TABLE]

Given $\lambda>0$, construct the probability measures $\widetilde{P}^{\widetilde{\mu}}$ and $P^{\mu}$ on $\Omega$ from $\widetilde{p}$ and $p$ respectively, and define the process $X_{t}$ as in Eq. (146).

Suppose $(\widetilde{\mathcal{P}}^{\dagger})^{n}[\widetilde{\mu}]\ll\widetilde{\mu}$ for all $n$ (in particular, if $\widetilde{\mu}$ is invariant for $\widetilde{p}$). Then for any $t\geq 0$ we have $\widetilde{P}^{\widetilde{\mu}}|_{\mathcal{G}_{t}}\ll P^{\mu}|_{\mathcal{G}_{t}}$ and

[TABLE]

By an analogous computation to the CTMC case, Eq. (99), the formula for the Radon-Nikodym derivative (152) leads to the following formula for the relative entropy:

Theorem C4

Suppose we have probability measures $\widetilde{\mu}$, $\mu$ and probability kernels $\widetilde{p}(x,dy)$, $p(x,dy)$ on $\mathcal{X}$. Assume that $\widetilde{\mu}\ll\mu$ and $\widetilde{p}(x,\cdot)\ll p(x,\cdot)$ for $\widetilde{\mu}$ a.e. $x$.

Suppose $(\widetilde{\mathcal{P}}^{\dagger})^{n}[\widetilde{\mu}]\ll\widetilde{\mu}$ for all $n$ (in particular, if $\widetilde{\mu}$ is invariant for $\widetilde{p}$). Then for any $t\geq 0$ we have

[TABLE]

where $h$ is as defined in Eq. (151).

It is also useful to note that, by the data processing inequality (see Theorem 14 in 1705001 ), $\mathcal{F}_{t}\subset\mathcal{G}_{t}$ implies

[TABLE]

When $\widetilde{\mu}$ is an invariant measure we obtain the following simpler formula:

Corollary C1

Suppose we have probability measures $\widetilde{\mu}^{*}$, $\mu$ and probability kernels $\widetilde{p}(x,dy)$, $p(x,dy)$ on $\mathcal{X}$. If $\widetilde{\mu}^{*}$ is invariant for $\widetilde{p}$ then for all $t>0$

[TABLE]

This is the relative entropy bound that was used in Section 5, when applying our UQ results to invariant measures of discrete-time Markov processes.

Acknowledgments

The research of L. R.-B. was partially supported by the National Science Foundation (NSF) under the grant DMS-1515712 and the Air Force Office of Scientific Research (AFOSR) under the grant FA-9550-18-1-0214.