Kemeny's constant for one-dimensional diffusions

TL;DR

This paper generalizes Kemeny's constant from finite Markov chains to one-dimensional diffusions, showing that the expected hitting time of a randomly chosen point (according to the invariant measure) is constant in the starting point and finite under certain boundary conditions.

Contribution

It extends the concept of Kemeny's constant to continuous one-dimensional diffusion processes, establishing conditions for its finiteness and invariance.

Findings

Expected hitting time is constant in the starting point.

Finiteness of the expected hitting time depends on boundary conditions.

Generalizes Kemeny's constant from Markov chains to diffusions.

Abstract

Let $X(\cdot)$ be a non-degenerate, positive recurrent one-dimensional diffusion process on $\mathbb{R}$ with invariant probability density $\mu(x)$, and let $\tau_y=\inf\{t\ge0: X(t)=y\}$ denote the first hitting time of $y$. Let $\mathcal{X}$ be a random variable independent of the diffusion process $X(\cdot)$ and distributed according to the process's invariant probability measure $\mu(x)dx$. Denote by $\mathcal{E}^\mu$ the expectation with respect to $\mathcal{X}$. Consider the expression $$ \mathcal{E}^\mu E_x\tau_\mathcal{X}=\int_{-\infty}^\infty (E_x\tau_y)\mu(y)dy, \ x\in\mathbb{R}. $$ In words, this expression is the expected hitting time of the diffusion starting from $x$ of a point chosen randomly according to the diffusion's invariant distribution. We show that this expression is constant in $x$, and that it is finite if and only if $\pm\infty$ are entrance boundaries for the diffusion. This result generalizes to diffusion processes the corresponding result in the setting of finite Markov chains, where the constant value is known as Kemeny's constant.

Equations

$${\mathcal{E}}^{\mu }{E}_x{\tau }_{\mathcal{X}}={\int }_{-\infty }^{\infty }(E_x{\tau }_y)\mu (y)dy,x\in \mathbb{R}.$$

$$L=\frac{1}{2}a(x)\frac{d^2}{d{x}^2}+b(x)\frac{d}{dx}.$$

$$i.E_x{\tau }_y<\infty ,\text{for all}x,y\in \mathbb{R};$$

$$\displaystyle ii.\ \int_{-\infty}^{\infty}\frac{1}{a(x)}\exp\big{(}2\int_{0}^{x}\frac{b(t)}{a(t)}dt\big{)}dx<\infty;$$

$$iii.\text{There exists an invariant probability density}\mu (x)\text{for the process}X(\cdot ).$$

$$\mu (x)=\frac{c_0}{a(x)}\exp (2{\int }_0^x\frac{b(t)}{a(t)})dt,$$

$$\int_{-\infty}^{\infty}\frac{1}{a(x)}\exp\big{(}2\int_{0}^{x}\frac{b(t)}{a(t)}dt\big{)}dx<\infty.$$

$${\mathcal{E}}^{\mu }{E}_x{\tau }_{\mathcal{X}}={\int }_{-\infty }^{\infty }(E_x{\tau }_y)\mu (y)dy,x\in \mathbb{R}.$$

$$\int^{\infty}dx\frac{1}{a(x)}\exp\big{(}2\int_{0}^{x}\frac{b(s)}{a(s)}ds\big{)}\int_{0}^{x}dy\exp\big{(}-2\int_{0}^{y}\frac{b(s)}{a(s)}ds\big{)}<\infty.$$

$$\int_{-\infty}dx\frac{1}{a(x)}\exp\big{(}2\int_{0}^{x}\frac{b(s)}{a(s)}ds\big{)}\int_{x}^{0}dy\exp\big{(}-2\int_{0}^{y}\frac{b(s)}{a(s)}ds\big{)}<\infty.$$

$$2{\int }_{-\infty }^{\infty }dy\mu (y){\int }_y^{\infty }dz\frac{\mu ([z,\infty ))}{\mu (z)a(z)}\text{and}2{\int }_{-\infty }^{\infty }dy\mu (y){\int }_{-\infty }^{y}dz\frac{\mu ((-\infty ,z])}{\mu (z)a(z)}.$$

$${\mathcal{E}}^{\mu }{E}_x{\tau }_{\mathcal{X}}={\int }_{-\infty }^{\infty }(E_x{\tau }_y)\mu (y)dy=\infty ,\text{for all}x\in \mathbb{R}.$$

$${\int }_{-\infty }^{\infty }\frac{\mu ([y,\infty ))}{\mu (y)}dy<\infty .$$

$$E_x{T}_y=\{\begin{array}{l}2{\int }_y^{x}dz\exp (-2{\int }_0^z\frac{b(t)}{a(t)}dt){\int }_z^{\infty }dw\frac{1}{a(w)}\exp (2{\int }_0^w\frac{b(t)}{a(t)}dt),-\infty <y<x;\\ 2{\int }_x^{y}dz\exp (-2{\int }_0^z\frac{b(t)}{a(t)}dt){\int }_{-\infty }^{z}dw\frac{1}{a(w)}\exp (2{\int }_0^w\frac{b(t)}{a(t)}dt),x<y<\infty .\end{array}$$

$${\int }_x^{\infty }(E_x{\tau }_y)\mu (y)dy=$$

$$\displaystyle 2\int_{x}^{\infty}dy\frac{c_{0}}{a(y)}\exp\big{(}2\int_{0}^{y}\frac{b(t)}{a(t)}dt\big{)}\int_{x}^{y}dz\exp(-2\int_{0}^{z}\frac{b(t)}{a(t)}dt)\int_{-\infty}^{z}dw\frac{1}{a(w)}\exp(2\int_{0}^{w}\frac{b(t)}{a(t)}dt).$$

$$\int_{x}^{\infty}dy\frac{c_{0}}{a(y)}\exp\big{(}2\int_{0}^{y}\frac{b(t)}{a(t)}dt\big{)}\int_{x}^{y}dz\exp(-2\int_{0}^{z}\frac{b(t)}{a(t)}dt)<\infty,$$

$$L{u}_{y,+}=-1\text{in}(y,\infty );u_{y,+}(y)=0;$$

$$L{u}_{y,-}=-1\text{in}(-\infty ,y);u_{y,-}(y)=0.$$

$$F(x)={\int }_{-\infty }^x{u}_{y,+}(x)\mu (y)dy+{\int }_x^{\infty }{u}_{y,-}(x)\mu (y)dy.$$

$$F^{\prime }(x)={\int }_{-\infty }^x{u}_{y,+}^{\prime }(x)\mu (y)dy+{\int }_x^{\infty }{u}_{y,-}^{\prime }(x)\mu (y)dy,$$

$$F^{\prime\prime}(x)=\int_{-\infty}^{x}u^{\prime\prime}_{y,+}(x)\mu(y)dy+\int_{x}^{\infty}u^{\prime\prime}_{y,-}(x)\mu(y)dy+\mu(x)\big{(}u_{x,+}^{\prime}(x)-u_{x,-}^{\prime}(x)\big{)}.$$

$$LF(x)=\int_{-\infty}^{x}\mu(y)Lu_{y,+}(x)dy+\int_{x}^{\infty}\mu(y)Lu_{y,-}(x)dy+\frac{1}{2}a(x)\mu(x)\big{(}u_{x,+}^{\prime}(x)-u_{x,-}^{\prime}(x)\big{)}.$$

$${\int }_{-\infty }^x\mu (y)L{u}_{y,+}(x)dy+{\int }_x^{\infty }\mu (y)L{u}_{y,-}(x)dy=-{\int }_{-\infty }^{\infty }\mu (y)dy=-1.$$

$$u_{x,+}^{\prime }(x)-u_{x,-}^{\prime }(x)=2\exp (-2{\int }_0^x\frac{b(t)}{a(t)}dt){\int }_x^{\infty }dw\frac{1}{a(w)}\exp (2{\int }_0^w\frac{b(t)}{a(t)}dt)-$$

$$2\exp (-2{\int }_0^x\frac{b(t)}{a(t)}dt){\int }_{-\infty }^{x}dw\frac{1}{a(w)}\exp (2{\int }_0^w\frac{b(t)}{a(t)}dt)=$$

$$2\exp (-2{\int }_0^x\frac{b(t)}{a(t)}dt){\int }_{-\infty }^{\infty }dw\frac{1}{a(w)}\exp (2{\int }_0^w\frac{b(t)}{a(t)}dt)=\frac{2}{c_0}\exp (-2{\int }_0^x\frac{b(t)}{a(t)}dt).$$

$$\frac{1}{2}a(x)\mu(x)\big{(}u_{x,+}^{\prime}(x)-u_{x,-}^{\prime}(x)\big{)}=\frac{c_{0}}{2}\exp(2\int_{0}^{x}\frac{b(t)}{a(t)}dt)\times\frac{2}{c_{0}}\exp(-2\int_{0}^{x}\frac{b(t)}{a(t)}dt)=1.$$

$$F(x)={\int }_{-\infty }^{\infty }(E_x{\tau }_y)\mu (y)dy=$$

$$2{\int }_x^{\infty }dy\mu (y){\int }_x^{y}dz\exp (-2{\int }_0^z\frac{b(t)}{a(t)}dt){\int }_{-\infty }^{z}dw\frac{1}{a(w)}\exp (2{\int }_0^w\frac{b(t)}{a(t)}dt)+$$

$$2{\int }_{-\infty }^{x}dy\mu (y){\int }_y^{x}dz\exp (-2{\int }_0^z\frac{b(t)}{a(t)}dt){\int }_z^{\infty }dw\frac{1}{a(w)}\exp (2{\int }_0^w\frac{b(t)}{a(t)}dt).$$

Full text

[TABLE]

In words, this expression is the expected hitting time of the diffusion starting from $x$ of a point chosen randomly according to the diffusion’s invariant distribution. We show that this expression is constant in $x$, and that it is finite if and only if $\pm\infty$ are entrance boundaries for the diffusion. This result generalizes to diffusion processes the corresponding result in the setting of finite Markov chains, where the constant value is known as Kemeny’s constant.

1. Introduction and Statement of Results

Let $\{X_{n}\}_{n=0}^{\infty}$ be an irreducible, discrete time Markov chain on a finite state space $S$, and denote it’s invariant probability measure by $\mu$. For $j\in S$, let $\hat{\tau}_{j}=\inf\{n\geq 1:X_{n}=j\}$ denote the first passage time to $j$. Denoting expectations for the process starting from $i\in S$ by $E_{i}$, consider the quantity $\sum_{j\in S}\mu_{j}E_{i}\hat{\tau}_{j}$. In their book on Markov chains [3], Kemeny and Snell showed that the above quantity is independent of the initial state $i$, and this quantity has become known as Kemeny’s constant, which we denote by $K$. Let $\tau_{j}=\inf\{n\geq 0:X_{n}=j\}$ denote the first hitting time of $j$. We note that $\sum_{j\in S}\mu_{j}E_{i}\tau_{j}$ is also independent of $i$, and is equal to $K-1$. This follows from the well-known fact that $E_{i}\hat{\tau}_{i}=\frac{1}{\mu_{i}}$ [2].

In [1], the authors analysed the Kemeny constant phenomenon for positive recurrent, discrete time and continuous time Markov chains on a denumerably infinite state space $S$. They showed that the quantity $\sum_{j\in S}\mu_{j}E_{i}\hat{\tau}_{j}$ is either infinite for all $i\in S$, or else is finite and independent of $i$. They conjectured that this quantity is always infinite in the discrete time setting, and they proved this in the case of discrete time birth and death chains on $\{0,1,\cdots\}$. In the case of continuous time birth and death chains on $\{0,1,\cdots\}$, they proved that the Kemeny constant is finite if and only if $+\infty$ is an entrance boundary for the process.

In this paper, we consider the corresponding problem in the context of one-dimensional diffusion processes on $\mathbb{R}$. Consider a non-degenerate one-dimensional diffusion process $X(\cdot)$ on $\mathbb{R}$ generated by

[TABLE]

We assume that $a$ is continuous and positive, and that $b$ is locally bounded and measurable. Denote probabilities and expectations for the Markov process $X(\cdot)$ starting from $x\in\mathbb{R}$ by $P_{x}$ and $E_{x}$. For $y\in\mathbb{R}$, let $\tau_{y}=\inf\{t\geq 0:X(t)=y\}$ denote the first hitting time of $y$. It is well-known [4] that the following conditions are equivalent:

[TABLE]

If these conditions hold, we say that the process is positive recurrent. In fact then, one has

[TABLE]

for a normalizing constant $c_{0}>0$.

From now on we assume that the diffusion is positive recurrent; that is, we assume that

[TABLE]

Let $\mathcal{X}$ be a random variable independent of the diffusion process $X(\cdot)$ and distributed according to the process’s invariant probability measure $\mu(x)dx$. Denote by $\mathcal{E}^{\mu}$ the expectation with respect to $\mathcal{X}$. We consider the expression

[TABLE]

In words, this expression is the expected hitting time of the diffusion starting from $x$ of a point chosen randomly according to the diffusion’s invariant distribution.

There immediately arises the question of whether or not this expression is finite. Note the following tradeoff:* On the one hand, the more negative (positive) the drift is in a neighborhood of $+\infty$ ($-\infty$), the faster is the decay of the invariant density $\mu(y)$ at $+\infty$ ($-\infty$). However on the other hand, the more negative (positive) the drift is in a neighborhood of $+\infty$ ($-\infty$), the larger $E_{x}\tau_{y}$ will be in a neighborhood of $+\infty$ ($-\infty$).*

It turns out that the finiteness or infiniteness of the expression depends on whether or not $\pm\infty$ are entrance boundaries for the process. We recall that $+\infty$ is called an entrance boundary if  $\lim_{x\to\infty}P_{x}(\tau_{y}<t)>0$, for some $y\in\mathbb{R}$ and some $t>0$. Similarly, $-\infty$ is called an entrance boundary if $\lim_{x\to-\infty}P_{x}(\tau_{y}<t)>0$, for some $y\in\mathbb{R}$ and some $t>0$. (Actually, equivalently, “some $y\in\mathbb{R}$ and some $t>0$” can be replaced by “all $y\in\mathbb{R}$ and all $t>0$.”) Given that the process is positive recurrent, that is, given that (1.3) holds, here is the criterion for an entrance boundary at $+\infty$:

[TABLE]

See [4, chapter 8], where the term “explosion inward from infinity” is used instead of entrance boundary. The condition (1.4) appears as (iv) in Theorem 4.1 in chapter 8. In that theorem, which does not assume positive recurrence, an additional requirement, denoted as (iii), is also stated; namely, \int^{\infty}\exp\big{(}-2\int_{0}^{x}\frac{b(s)}{a(s)}ds\big{)}dx=\infty. However, an application of the Cauchy-Schwarz inequality shows that this condition holds automatically if (1.3) holds. Similarly, given that the process is positive recurrent, here is the criterion for an entrance boundary at $-\infty$:

[TABLE]

We will prove the following theorem. Let $\mu$ denote the probability measure with density $\mu(x)dx$.

Theorem 1**.**

Assume that the diffusion is positive recurrent; that is, assume that (1.3) holds. If $\pm\infty$ are both entrance boundaries for the diffusion, that is, if (1.4) and (1.5) both hold, then $\mathcal{E}^{\mu}E_{x}\tau_{\mathcal{X}}=\int_{-\infty}^{\infty}(E_{x}\tau_{y})\mu(y)dy$ is finite and independent of $x\in\mathbb{R}$. Two alternative expressions for the value of this constant are

[TABLE]

If at least one of $\pm\infty$ is not an entrance boundary, that is if at least one of (1.4) and (1.5) does not hold, then

[TABLE]

Remark 1. Given a continuously differentiable, strictly positive probability density $\mu$ and given a continuously differentiable diffusion matrix $a$, if one chooses the drift b(x)=\frac{1}{2}\big{(}a(x)\frac{\mu^{\prime}(x)}{\mu(x)}+a^{\prime}(x)\big{)}, then the diffusion process with generator $L$ will have invariant probability density $\mu$. Thus, given such a density $\mu$, the diffusion processes for which $\mu$ is the invariant density can be indexed by their diffusion matrices $a$. From (1.6) we see that given the invariant density $\mu$, the expression $\mathcal{E}^{\mu}E_{x}\tau_{\mathcal{X}}$ is monotone decreasing as a function of the diffusion matrix $a$. Furthermore, we see that for sufficiently large $a$ it will be finite and for sufficiently small $a$ it will be infinite. In particular then, given $\mu$ we can find a diffusion with invariant density $\mu$ for which $\pm\infty$ are entrance boundaries and we can find such a diffusion for which $\pm\infty$ are not entrance boundaries.

Remark 2. Let $\mu$ be a continuously differentiable, strictly positive probability density as in Remark 1. Since the two expressions in (1.6) must be both finite or both infinite, it is easy to see that in the case of constant diffusion coefficient, $a\equiv\text{const.}$, the expression $\mathcal{E}^{\mu}E_{x}\tau_{\mathcal{X}}$ is finite if and only if

[TABLE]

In particular, if $\mu(x)\sim\text{const.}e^{-k|x|^{l}}$, for $k,l>0$, then (1.8) holds if and only if $l>2$,

2. Proof of Theorem 1

We first proof that $\int_{-\infty}^{\infty}(E_{x}\tau_{y})\mu(y)dy<\infty$ if and only if (1.4) and (1.5) hold. We have the following explicit expression for the expected hitting time:

[TABLE]

For a derivation, see for example the proof of Proposition 2 in [5] (where $a(x)$ is a constant and denoted by $D$). Using this with (1.2)–(1.5), it is easy to see that (1.4) and (1.5) constitute necessary and sufficient conditions for the finiteness of $\int_{-\infty}^{\infty}(E_{x}\tau_{y})\mu(y)dy$. Indeed, from (2.1) and (1.2), we have

[TABLE]

By (1.2), the right hand side of (2.2) is finite if and only if

[TABLE]

and this latter expression is finite if and only if (1.4) holds. Thus, $\int_{x}^{\infty}(E_{x}\tau_{y})\mu(y)dy<\infty$ if and only if (1.4) holds. A similar analysis shows that $\int_{-\infty}^{x}(E_{x}\tau_{y})\mu(y)dy<\infty$ if and only if (1.5) holds.

We now show that if (1.4) and (1.5) hold, then $\int_{-\infty}^{\infty}(E_{x}\tau_{y})\mu(y)dy$ is independent of $x\in\mathbb{R}$. For $y\in\mathbb{R}$, define $u_{y,+}(x)=E_{x}\tau_{y}$, for $x\geq y$, and $u_{y,-}(x)=E_{x}\tau_{y}$, for $x\leq y$. (Of course, $u_{y,+}(x)$ and $u_{y,-}(x)$ respectively are equal to the first and second lines on the right hand side of (2.1).) As is well-know, it follows from an application of Ito’s formula that

[TABLE]

(Indeed, it is from this that the formulas in (2.1) were derived.) Define $F(x)=\int_{-\infty}^{\infty}(E_{x}\tau_{y})\mu(y)dy$. Then we have

[TABLE]

In light of the fact that $u_{y,+}(x)$ and $u_{y,-}(x)$ are given by (2.1), as well as the fact that (1.3)– (1.5) hold, we can differentiate freely under the integral. Using the boundary condition in (2.3), we have

[TABLE]

and differentiating again gives

[TABLE]

From (2.4) and (2.5) we obtain

[TABLE]

From (2.3) we have

[TABLE]

Using the formulas for $u_{y,+}(x)$ and $u_{y,-}(x)$ as given by the two lines on the right hand side of (2.1), and recalling (1.2), we have

[TABLE]

Thus,

[TABLE]

From (2.6)–(2.8), we conclude that $LF=0$; that is, $F$ is $L$-harmonic.

Since $L$ is a recurrent diffusion generator, it has no nonconstant positive harmonic functions [4, p.457]. Consequently, we conclude that $F(x)=\int_{-\infty}^{\infty}(E_{x}\tau_{y})\mu(y)dy$ is constant in $x$.

It remains to prove (1.6). From (2.2) and the corresponding formula for $\int_{-\infty}^{x}(E_{x}\tau_{y})\mu(y)dy$, we have

[TABLE]

Letting $x\to\infty$ in (2.9) and using (1.2) to write everything in terms of $a$ and $\mu$ gives the first alternative in (1.6). Similarly, letting $x\to-\infty$ gives the second alternative in (1.6). $\square$