Hypotheses testing and posterior concentration rates for semi-Markov processes

TL;DR

This paper develops a Bayesian framework to analyze the asymptotic behavior of semi-Markov processes, providing conditions for posterior concentration rates and constructing robust statistical tests.

Contribution

It introduces nonparametric Bayesian methods for semi-Markov processes and derives posterior concentration rates, extending existing theory to continuous time and general state spaces.

Findings

Established posterior concentration rates for semi-Markov kernels

Constructed robust tests between Hellinger balls around kernels

Applied methods to discrete-time and finite state space cases

Abstract

In this paper, we adopt a nonparametric Bayesian approach and investigate the asymptotic behavior of the posterior distribution in continuous time and general state space semi-Markov processes. In particular, we obtain posterior concentration rates for semi-Markov kernels. For the purposes of this study, we construct robust statistical tests between Hellinger balls around semi-Markov kernels and present some specifications to particular cases, including discrete-time semi-Markov processes and finite state space Markov processes. The objective of this paper is to provide sufficient conditions on priors and semi-Markov kernels that enable us to establish posterior concentration rates.

Equations

$$Z_t:=J_{N(t)},t\in {\mathbb{R}}^{+},$$

$$N(t)=\left\{\begin{array}[]{ll}0,&\mbox{if}\quad S_{1}-S_{0}>t,\\ \sup\{n\in\mathbb{N^{*}}:S_{n}\leq{t}\},&\mbox{if}\quad S_{1}-S_{0}\leq{t}.\end{array}\right.$$

$$\begin{gathered} \mathbb{P}(J_{n+1}\in B,S_{n+1}-S_n\le t|J_0,\dots ,J_n,S_0,\dots ,S_n) \\ \stackrel{\text{mbox}a.s.}{=}\mathbb{P}(J_{n+1}\in B,S_{n+1}-S_n\le t|J_n). \end{gathered}$$

$$Q_x(B,t):=\mathbb{P}(J_{n+1}\in B,S_{n+1}-S_n\le t|J_n=x),x\in E,t\in {\mathbb{R}}^{+},B\in \mathcal{E}$$

$$P^{(n)}(x,B):=\mathbb{P}(J_n\in B|J_0=x),x\in E,B\in \mathcal{E},$$

$$Q_x(B,t)={\int }_{B}P(x,dy)\mathbb{P}(S_{n+1}-S_n\le t|J_n=x,J_{n+1}=y),\forall t\in {\mathbb{R}}^{+},\forall B\in \mathcal{E}.$$

$${\int }_E\mathbf{\rho }(dx)m(x)<\infty .$$

$$\mathbb{P}(S_{n+1}-S_n\le t|J_n=x,J_{n+1}=y)\ne {\text{1}}_{{\mathbb{R}}^{+}}(t),\forall n\in \mathbb{N},\forall t\in {\mathbb{R}}^{+},\forall x,y\in E.$$

$$\tilde{\mathbf{\rho }}(A,\Gamma )={\int }_E\mathbf{\rho }(dx)Q_x(A,\Gamma ),\forall A\in \mathcal{E},\forall \Gamma \in {\mathbb{B}}^{+}.$$

$$\mathcal{Q}=\{q=q_x(y,t):x,y\in E,t\in {\mathbb{R}}^{+}\},$$

$$B_{d}(q_{0},\epsilon)=\Big{\{}q\in{\mathcal{Q}}:d({q}_{0},{q})\leq{\epsilon}\Big{\}}.$$

$$h_{\nu }^2(Q_{x;1},Q_{x;2})$$

$$d_{\mu }^2(q_1,q_2)$$

$${\mathcal{H}}_n=\{J_0,J_1,\dots ,J_n,S_0,S_1,\dots ,S_n\},$$

$${\Pi }_n^{{\mathcal{H}}_n}\left(B_d^{\complement }(q_0,{ϵ}_n)\right)\stackrel{L_1({\mathbb{P}}_0^{(n)})}{⟶}0\text{as}n\to 0,$$

$$H_0:q_0\text{against}{H}_1:q\in B_{d_{\eta \ast },n}(q_1,\xi ϵ),\text{with}{d}_{\nu \ast }(q_0,q_1)\ge ϵ,$$

$$\frac{1}{k}\sum _{u=1}^k{P}^{(u)}(x,\cdot )\ge {\nu }^{\ast }(\cdot )\text{and}{P}^{(l)}(x,\cdot )\le {\eta }^{\ast }(\cdot ),$$

$$\forall m\in \mathbb{N},\frac{1}{k}\sum _{u=1}^k{P}^{(u+m)}(x,\cdot )\ge {\nu }^{\ast }(\cdot )\text{and}{P}^{(l+m)}(x,\cdot )\le {\eta }^{\ast }(\cdot ).$$

$${\mathbb{E}}_0^{(n)}[{\psi }_1({\mathcal{H}}_n)]\le e^{-Kn{ϵ}^2}\text{and}\underset{q\in {\mathcal{Q}}_n:d_{\eta \ast }(q_1,q)<ϵ\xi }{\sup }{\mathbb{E}}_q^{(n)}[1-{\psi }_1({\mathcal{H}}_n)]\le e^{-\tilde{K}n{ϵ}^2}.$$

$$\sup_{\epsilon{>}\epsilon_{n}}\log{N}\bigl{(}{\epsilon\xi},B_{d_{\nu*},n}(q_{0},\epsilon),d_{\eta*}\bigr{)}\leq{n\epsilon_{n}^{2}}.$$

$${\mathbb{E}}_0^{(n)}[\psi ({\mathcal{H}}_n)]\le e^{-Kn{ϵ}_n^2{M}^2}$$

$$\underset{q\in {\mathcal{Q}}_n:d_{\nu \ast }(q_0,q)>{ϵ}_{n}M}{\sup }{\mathbb{E}}_q^{(n)}[1-\psi ({\mathcal{H}}_n)]\le e^{-\tilde{K}n{ϵ}_n^2{M}^2}.$$

$$q_x(y,k)=\mathbb{P}(J_{n+1}=y,X_{n+1}=k|J_n=x),$$

$$Q_x(B,k)=\mathbb{P}(J_{n+1}\in B,X_{n+1}\le k|J_n=x).$$

$$h_{\mu }^2(Q_{x;1},Q_{x;2})$$

$$q_x(y,k)=\mathbb{P}(J_{n+1}=y,X_{n+1}=k|J_n=x),$$

$$Q_x(y,k)=\mathbb{P}(J_{n+1}=y,X_{n+1}\le k|J_n=x)$$

$$h^2(Q_{x;1},Q_{x;2})$$

$$d^2(q_1,q_2)$$

$$Q_x(y,t)=\mathbb{P}(J_{n+1}=y,X_{n+1}\le t|J_n=x)$$

$$h_{{\nu }_1}^2(Q_{x;1},Q_{x;2})$$

Full text

Keywords Bayesian nonparametric statistics, posterior concentration rate, semi-Markov kernel, testing procedure, Hellinger distance

1 Introduction

Semi-Markov processes (SMPs) are stochastic processes that are widely used to model real-life phenomena encountered in seismology, biology, reliability, survival analysis, wind energy, finance and other scientific fields. SMPs ([27],[35],[37]) generalize Markov processes in the sense that they allow the sojourn times in states to follow any distribution on $[0,+\infty)$, instead of the exponential distribution in the Markov case. Since no memoryless distributions could be considered in a semi-Markov environment, duration effects could be reproduced. The duration effect firms that the time the semi-Markov system spends in a state influences its transition probabilities. Particular cases of SMPs include continuous and discrete-time Markov chains and ordinary, modified and alternating renewal processes. The foundations of the theory of SMPs were laid by Pyke ([32], [33]). Since then, further significant results were obtained by Çinlar [13], Korolyuk et al. [23] and many others. We refer the interested reader to Limnios and Oprişan [28] for an approach to SMPs and their applications in reliability. For an overview in the theory on semi-Markov chains oriented toward applications in modeling and estimation see Barbu and Limnios [4].

Although the statistical inference of SMPs has been extensively studied from a frequentist point of view, the Bayesian literature is rather limited. Except from some specific SMP models ([14],[15]), only a few papers have considered the nonparametric Bayesian theory supporting these models ([9],[31]). Here we aim to close the aforementioned gap and follow a nonparametric Bayesian approach. The key quantity in the theory of SMPs is the semi-Markov kernel (SMK), $Q$. Our objective is to draw Bayesian inference on the Radon-Nikodym derivative of the SMK, $q$. Let us denote by $\mathcal{H}_{n}$ a trajectory of the SMP of length $n$ and by $\Pi$ the prior distribution of $q$, which in all generality, could depend on $n$, and thereafter will be denoted by $\Pi_{n}$. Given $\mathcal{H}_{n}$ and $\Pi_{n},$ the knowledge on $q$ is updated by the posterior distribution, that is denoted by $\Pi_{n}^{\mathcal{H}_{n}}(\cdot)=\Pi_{n}(\cdot|{\mathcal{H}_{n}}).$ We shall stick to the last notation throughout the paper and further denote by $q_{0}$ the derivative of the “true” SMK, $Q_{0}$, which is the SMK that generated $\mathcal{H}_{n}$. The main topic of the article is the study of the asymptotic behaviour of $\Pi_{n}^{\mathcal{H}_{n}}$ in a neighbourhood of $Q_{0}.$

Most of the known results in the asymptotic behaviour of posterior distributions in infinite-dimensional models address issues of the posterior consistency and posterior concentration around the true distribution. In a nonparametric context, when the observations are i.i.d., such results were first derived in [21] and [36] with a variety of examples. Beyond the i.i.d. setup, the asymptotic behaviour of the posterior has been studied in the context of independent nonidentically distributed observations ([1], [2], [12], [17], [19], [20]).

One of the most natural extensions of the i.i.d. structure is a Markov process, where only the immediate past matters. Although, given the present, the future will not further depend on the past, the dependence propagates and may reasonably capture the dependence structure of the observations. Ghosal and van Der Vaart [17] studied the asymptotic behaviour of posterior distributions to several classes of non-i.i.d. models including Markov chains. For their purposes the authors used previous results on the existence of statistical tests ([6], [24], [25], [26]) between two Hellinger balls for a given class of models. We refer the interested reader to [8] for improved results about the existence of such tests for the relevant estimation problems. Tang and Ghosal [38] extended Schwartz’s theory of posterior consistency to ergodic Markov processes and applied it in the context of a Dirichlet mixture model for transition densities. More recently, Gassiat and Rousseau [16] studied the posterior distribution in hidden Markov chains where both the observational and the state spaces are general. For nonparametric Bayesian estimation of conditional distributions, Pati et al. [30] provided sufficient conditions on the prior under which the weak and various types of strong posterior consistency could be obtained.

For reviews on posterior consistency as well as posterior concentration in infinite dimensions, the interested reader can refer to Wasserman [40], Ghosh and Ramamoorthi [22] and Ghosal et al. [18].

This paper aims to extend previous results by studying the convergence of the posterior distribution of $q$ for SMPs. Specifically, we generalize and extend previous results on discrete-time Markov processes in finite state space [17] to continuous-time SMPs in general state space.

In order to apply the general theory to the semi-Markov framework, we demonstrate the existence of the relevant statistical tests. To this purpose, we extend the hypotheses testing results for Marov chains developed by Birgé [6] to continuous-time general state space SMPs. Such tests can also be used to distinguish Markov from semi-Markov models and decide which model could better describe the data, which is a crucial subject in real-world applications.

Very few researchers considered hypotheses testing problem in a semi-Markov context. Bath and Deshpande [5] developed a nonparametric test for testing Markov against semi-Markov processes. Banerjee and Bhattacharyya [3] considered a two-state SMP and proposed parametric tests for the equality of the sojourn time distributions, under the assumption that these distributions are absolutely continuous and belong to the Exponential family. Also in a parametric context, Malinovskii [29] considered that the probability distribution of an SMP depends on a real-valued parameter $\vartheta>0$ and studied the simple hypothesis $H_{0}:\vartheta=0$ against $H_{1}:\vartheta=hT^{-1/2}$, $0<h\leq{c}$ (the SMP is observed up to time $T$). Chang et al. ([10], [11]) considered hypotheses testing problems for semi-Markov counting processes, in a survival analysis context. Tsai [39] proposed a rank test based on semi-Markov processes in order to test whether a pair of observation $(X,Y)$ has the same distribution as $(Y,X),$ i.e., $X,$ $Y$ exchangeable. To the best of our knowledge, the present research is the first one that considers general robust hypotheses testing problems for SMPs in a nonparametric context.

We focus on SMPs since they are much more general and better adapted to applications than the Markov processes. In real-world systems, the state space of the under study processes could be $\{0,1\}^{\mathbb{N}},$ (e.g., communication systems), where $\mathbb{N}$ is the set of nonnegative integers, or $[0,\infty)$ (e.g., fatigue crack growth modelling). This is the reason why we concentrate on general SMPs. On the other side, since in physical and biological applications time is usually considered to be continuous, discrete-time processes are not always appropriate for describing such phenomena. In such situations continuous-time processes are often more suitable than the discrete-time ones. Therefore we focus our discussion on the continuous-time case rather than the discrete-time case. Nonetheless, note that our results on the robust tests are very general and could also be applied to the discrete-time case, with the corresponding modifications.

The organization of the paper is as follows. In Section $2$ the notation and preliminaries of semi-Markov processes are presented; the objectives of our paper are also presented. Section $3$ describes the hypotheses testing for the processes under study and some particular cases. Section $4$ discusses the derivation of the posterior concentration rate and the relative hypotheses. Finally, in Section $5$, we give a detailed description of the proofs and some technical lemmas.

2 The semi-Markov framework and objectives

2.1 Semi-Markov processes

We consider $(E,\mathcal{E})$ a measurable space and an $(E,\mathcal{E})-$valued semi-Markov process $\mathbf{Z}:=(Z_{t})_{t\in\mathbb{R}^{+}}$ defined on a complete probability space $(\Omega,\mathcal{F},\mathbb{P})$. The semi-Markov process $\mathbf{Z}$ corresponding to the Markov renewal process (MRP) $\mathbf{(J,S)}:=(J_{n},S_{n})_{n\in{\mathbb{N}}}$, is defined by

[TABLE]

where $0\leq{S_{0}}\leq\ldots\leq{S_{n}}\leq\ldots$ are the successive $\mathbb{R}^{+}$-valued jump times of $\mathbf{Z}$, $(J_{n})_{n\geq 0}$ denotes the successive visited states at these jump times (henceforth called the embedded Markov chain (EMC)) and

[TABLE]

$S_{0}$ may be viewed as the first non-negative time at which a jump is observed. In what follows, the EMC and MRP are considered to be homogeneous with respect to $n\in{\mathbb{N}}$. It is worth noticing that the MRP $\mathbf{(J,S)}$ satisfies the following Markov property, i.e., for any $n\in{\mathbb{N}}$, any $t\in{\mathbb{R}^{+}}$ and any $B\in{\mathcal{E}}$:

[TABLE]

In the semi-Markov framework, of central importance is the semi-Markov kernel (SMK) defined as follows:

[TABLE]

Since we suppose that the distribution of $\mathbf{Z}$ is unknown, we focus our interest on the semi-Markov kernel. In particular the stochastic behavior of the SMP Z is determined completely by its SMK and its initial distribution.

Let us denote the $n-$step transition kernel of the EMC $(J_{n})_{n\in{\mathbb{N}}}$ by

[TABLE]

and the (one-step) transition kernel by $P(x,B)=Q_{x}(B,\infty).$

It is worth mentioning that

[TABLE]

The following assumptions have to be considered in the sequel.

1. A1

The embedded Markov chain $(J_{n})_{n\in\mathbb{N}}$ is ergodic with stationary probability measure $\boldsymbol{\rho}$ (that is $\boldsymbol{\rho}{P}=\boldsymbol{\rho}$, with $P$ the transition kernel of $\mathbf{J}$ and $\boldsymbol{\rho}(E)=1$). 2. A2

The mean sojourn times $m(x)=\int_{0}^{\infty}\mathbb{P}(S_{1}-S_{0}>t\mid J_{0}=x)dt$ satisfies

[TABLE] 3. A3

[TABLE]

Note that A2 and A3 ensure that for all non negative $t$ and $B\in{\cal E}$, $\mathbb{P}(Z_{t}\in B)$ is always well-defined and non-zero. However the conditional probability in Assumption A3 may be defined as any Dirac measure on positive real numbers.

Denote also by $\mathbb{B}^{+}$ the Borelian $\sigma-$algebra on $\mathbb{R}^{+}$. We suppose that for any $x\in E$, the SMK starting from $x$ is absolutely continuous with respect to (w.r.t.) $\nu$, a $\sigma-$finite measure $(E\times\mathbb{R}^{+},\mathcal{E}\otimes{\mathbb{B}^{+}})$ and denote by $q_{x}(\cdot,\cdot)$ its Radon-Nikodym (RN) derivative, i.e., $Q_{x}(dy,dt)=q_{x}(y,t)d{\nu}(y,t)$. For $n\geq{1}$, let $X_{n}:=S_{n}-S_{n-1}$ be the successive sojourn times of $\mathbf{Z}$ and ${0}\leq{X_{0}}=S_{0}$. On $\mathcal{E}\otimes{\mathbb{B}^{+}}$, we further define the measure $\widetilde{\boldsymbol{\rho}}$ as the distribution of $\mathbf{(J,X)}:=(J_{n},X_{n})_{n\in{\mathbb{N}}}$, where

[TABLE]

Proposition 1**.**

The measure $\widetilde{\boldsymbol{\rho}}$ defined in (2) is the stationary distribution of $(J_{n},X_{n})_{n\in{\mathbb{N}}}$.

Since we are interested in obtaining asymptotic results, without loss of generality we consider as initial distribution of the process $\mathbf{(J,X)}$ its stationary distribution, $\widetilde{\boldsymbol{\rho}}$. To avoid complicated notation, we will also use $\widetilde{\boldsymbol{\rho}}$ to denote the density w.r.t. $\nu.$

In the sequel, the hypotheses A1, A2 and A3 are considered to hold true.

2.2 Objectives

Recall that we have denoted by $Q_{0}$ the true semi-Markov kernel and by ${q}_{0}$ its RN derivative w.r.t. $\nu,$ cf. Section $2.$ We suppose that $q_{0}$ belongs to a certain set of semi-Markov kernel densities $\mathcal{Q}$ defined by

[TABLE]

which is equipped with a metric $d$ that will be defined in the sequel. Next consider $\epsilon$-neighborhoods around $q_{0}$ in $\mathcal{Q}$ w.r.t. $d$, that is

[TABLE]

To allow some flexibility, it is quite common to deal with $\mathcal{Q}_{n}$, a subset of $\mathcal{Q}$, that may depend on $n,$ such that the prior distribution $\Pi_{n}$ on ${\cal Q}$ assigns most of its mass on $\mathcal{Q}_{n}$ (see Assumption H4 below). An $\epsilon$-neighborhood around $q_{0}$ in $\mathcal{Q}_{n}$ w.r.t. $d$ will be denoted by $B_{d,n}(q_{0},\epsilon)$.

As noted by Birgé [6] in the setting of Markov chains, there exists a priori no “natural” distance $d$ between two semi-Markov kernel densities. Nevertheless, a natural distance could be defined between two probability distributions $Q_{x;1}$ and $Q_{x;2}$ dominated by $\nu$ and corresponding to the same initial state $J_{0}=x\in E$. Indeed, if we further denote by ${q}_{x;1}$ and ${q}_{x;2}$ their respective RN derivatives, and following the lines of Birgé [6], $d$ could be defined in two steps. First by considering the squared Hellinger distance between ${Q}_{x;1}$ and ${Q}_{x;2}$, i.e.,

[TABLE]

and second, given a measure on $\mathcal{E}$, say $\mu$, by setting a semi-distance $d_{\mu}$ between $q_{1}$ and $q_{2}$,

[TABLE]

Given a sample path of the SMP for a given number of jumps $n\in{\mathbb{N}^{*}}$,

[TABLE]

we adopt a Bayesian point of view by considering a prior distribution $\Pi_{n}$ on $\mathcal{Q}.$ We aim to establish how fast the posterior distribution shrinks, in terms of $d$, the “true” semi-Markov kernel density, $q_{0}$. The precise definition of $d$ will be given after the statement of Assumption H1, where the measure $\mu$ is fixed. More precisely, our objective is to find the minimal positive sequence $\epsilon_{n}$ tending to zero as $n$ goes to infinity, such that under some assumptions on both $\mathcal{Q}$ and $\Pi_{n}$

[TABLE]

where $B^{\complement}_{d}$ denotes the complementary of $B_{d}$ in $\mathcal{Q}$ and $\mathbb{P}^{(n)}_{0}$ refers to the “true” distribution of ${\cal H}_{n}.$

Let us denote by $\mathbb{P}_{q}^{(n)}$ the distribution of $\mathcal{H}_{n}$, when the density of the SMK is $q$. We further denote by $\mathbb{E}_{q}^{(n)}$ the expectation and by ${\mathrm{V}}^{(n)}_{q}$ the variance w.r.t. $\mathbb{P}_{q}^{(n)}$, respectively. Every quantity (distribution, SMK, expectation, variance,$\ldots$) with an index [math] refers to the corresponding “true” quantity.

3 Hypotheses testing for semi-Markov processes

3.1 Robust tests

One of the key ingredients needed to obtain posterior concentration rates is the construction of corresponding robust hypotheses tests. For a variety of models, depending on the semi-metric $d$, some tests with exponential power do exist. For instance, in the case of density or conditional density estimation, Hellinger or $L_{1}$ tests have been introduced in [7]. Other examples of tests could be found in [17] and in [34]. However, to the best of our knowledge, no such tests exist for semi-Markov processes. Therefore it is of paramount importance to build test procedures with exponentially small errors in the semi-Markov context. Thus in the sequel we will be interested in the following testing procedure

[TABLE]

for some $\xi\in(0,1).$

In order to derive posterior concentration rates for SMK densities, one more assumption is required.

• •

H1: There exist two measures $\nu^{*}$ and $\eta^{*}$ on $\mathcal{E}$ and two positive integers $k,l$ such that for any ${x}\in{E}$,

[TABLE]

where $P^{(\cdot)}$ is defined in (1). Note that H1 implies the following inequalities which serve to prove Proposition 2:

[TABLE]

Proposition 2**.**

Under Hypothesis H1, for any $n\in\mathbb{N}^{*}$, there exist universal positive constants $\xi\in(0,1),$ $K$ and ${\tilde{K}}$ such that for any $\epsilon>0$ and any $q_{1}\in\mathcal{Q}_{n}$ such that $d_{\nu*}(q_{1},q_{0})>\epsilon$, there exists a test $\psi_{1}(\mathcal{H}_{n})$ satisfying

[TABLE]

The next corollary generalizes Proposition 2 to any $q_{1}\in{\cal Q}_{n}$ which is $\epsilon-$distant from $q_{0}$ w.r.t. $d_{\nu*}$. It requires an additional assumption (see hereafter H2) to control the complexity of ${\tilde{\cal Q}_{n}}\subseteq{{\cal Q}_{n}}$. This assumption is based on the minimum number of $d_{\nu*}$-balls of radius ${\tilde{\epsilon}}$ needed to cover ${\tilde{\cal Q}_{n}}$, which is denoted by ${N}({\tilde{\epsilon}},{\tilde{\cal Q}_{n}},d_{\nu*})$.

Note that the case where the null hypothesis is composite could also be considered; the first type error in (6) would be written similarly to the second type error, with straightforward modifications.

Corollary 1**.**

Under Hypothesis H1, assume that for a sequence $\epsilon_{n}$ of positive numbers such that $\displaystyle{\lim_{n\rightarrow+\infty}}\epsilon_{n}=0$ and $\displaystyle{\lim_{n\rightarrow+\infty}}n\epsilon_{n}^{2}=0$, the following assumption holds true.

• •

H2* For ${\xi}$ in $(0,1),$*

[TABLE]

Then, there exists a test $\psi(\mathcal{H}_{n})$ satisfying

[TABLE]

3.2 Particular cases

In this paper the results are rather generic in the sense that they refer to continuous-time and general state space SMPs. In the sequel, we focus on some particular cases that could be of special interest either from an applicative point of view, or as a starting point for further research.

First, note that the state space is considered to be finite in most of the applicative articles. Second, we would like to stress out that in some applications the state space is intrinsically continuous, due to the fact that the scale of the measures is continuous.

3.2.1 Discrete-time SMPs

• •

General state space

Let us first denote by

[TABLE]

the RN derivative of the SMK. Then for any $k\in{\mathbb{N}}$ and any $B\in{\mathcal{E}}$, the respective cumulative semi-Markov kernel is given by

[TABLE]

It should be noted that in this case $\nu$ in (3) is the product measure between a finite-measure $\mu$ on $(E,\mathcal{E})$ used in (4) and the counting measure on $\mathbb{N}.$ Thus in this framework, the squared Hellinger distance becomes

[TABLE]

while the semi-distance $d_{\mu}$ between $q_{1}$ and $q_{2}$ is given in Equation (4).

• •

Finite state space

For any $k\in{\mathbb{N}}$ and any $y\in{E}$, we define by

[TABLE]

the semi-Markov kernel and by

[TABLE]

the cumulative semi-Markov kernel, respectively.

Since in this framework $\mu$ is the counting measure on $(E,\mathcal{E}),$ the squared Hellinger distance becomes

[TABLE]

and the semi-distance $d$ between $q_{1}$ and $q_{2}$ is given by

[TABLE]

3.2.2 Continuous-time SMPs

• •

Finite state space

Let us first denote by

[TABLE]

the semi-Markov kernel, for any $y\in{E}$ and any $t\in\mathbb{R}^{+}$.

In this context, the squared Hellinger distance becomes

[TABLE]

where $\nu_{1}$ is the marginal on $(\mathbb{R}^{+},\mathbb{B}^{+})$ of the measure $\nu$ defined on $E\times\mathbb{R}^{+},$ and the semi-distance $d$ between $q_{1}$ and $q_{2}$ is defined as in Eq. (9).

3.3 Specification to the Markov case

Note that the previously obtained results on robust tests for SMPs could be adapted to the particular case of Markov processes. These tests are of great interest and could be used for real-life applications. In particular, they enable us to decide if an observed dataset would be better described by a Markov (null hypothesis) or a semi-Markov process (alternative hypothesis). More precisely suppose we are interested in the following testing problem

[TABLE]

Note that $\tilde{H}_{1}$ could be extended to any $\xi\epsilon-$ball around $Q_{1}$ with $\xi\in]0,1[$.

In this section, we are going to explain how the hypothesis testing problem $\tilde{H}_{0}$ versus $\tilde{H}_{1}$ can directly be handled from solving the hypothesis problem ${H}_{0}$ versus ${H}_{1}$ stated in (5).

First, for the discrete-time and finite state space case, assume that we have a Markov process with Markov transition matrix $\widetilde{p}=(\widetilde{p}_{xy})_{x,y\in E},$ $\widetilde{p}_{xx}\neq 1$ for all states $x\in E.$

Note that a Markov process could be represented as a semi-Markov process with semi-Markov kernel given in (7) and expressed as

[TABLE]

Consequently, we can define the corresponding squared Hellinger distance as in (8) and construct the corresponding testing procedure.

Second, for the continuous-time and finite state space case, consider a regular jump Markov process with continuous transition semigroup $\widetilde{P}=\left(\widetilde{P}(t)\right)_{t\in\mathbb{R}^{+}}$ and infinitesimal generator matrix $A=(a_{xy})_{x,y\in E}.$

In this context, we can represent the Markov process as a semi-Markov process with semi-Markov kernel given in (10) and expressed as

[TABLE]

where $a_{x}:=-a_{xx}<\infty,x\in E.$

Note that one can also consider the case where the null hypothesis is composite or the case where the alternative hypothesis is simple, with straightforward modifications.

4 Posterior concentration rates for semi-Markov kernels

In this part, we present the key assumptions and state our main result.

First note that the likelihood function of the sample path $\mathcal{H}_{n}$ evaluated at $q\in{\cal Q}$ is given by

[TABLE]

Let us introduce the tools that play a central role in asymptotic Bayesian nonparametrics: the Kullback-Liebler (KL) divergence between any two distributions $\mathbb{P}^{(n)}_{q_{1}}$ and $\mathbb{P}^{(n)}_{q_{2}}$ and the centered second moment of the integrand of the corresponding KL divergence, which are defined by

[TABLE]

where $\mathbb{E}_{0}^{(n)}$ and $\mathbb{V}_{0}^{(n)}$ denote respectively the expectation and the variance w.r.t. $\mathbb{P}^{(n)}_{0}$.

Then, consider the subspace of $\mathcal{Q}$, ${\cal U}(q_{0},\epsilon)$, which represents the following Kullback-Liebler $\epsilon$-neighborhood of $\mathbb{P}^{(n)}_{0}$, that is, for positive $\epsilon$,

[TABLE]

It is worth mentioning that although $\widetilde{\boldsymbol{\rho}}$ is not of primary interest, since it is unknown it should require a prior. But since any prior on $\widetilde{\boldsymbol{\rho}}$ that is independent of the prior on $q$ would disappear upon marginalization of the posterior of $(\widetilde{\boldsymbol{\rho}},q)$ relatively to $\widetilde{\boldsymbol{\rho}},$ in the sequel it will be dropped. Thus, it suffices to consider only a prior distribution on $q.$

Let us now state the main result. We recall that $\Pi_{n}$ denotes a prior distribution on $\mathcal{Q}.$

Theorem 1**.**

Assume that H1 holds true and suppose that for a sequence of positive numbers $\epsilon_{n}$ such that $\displaystyle{\lim_{n\rightarrow+\infty}}\epsilon_{n}=0$, $\displaystyle{\lim_{n\rightarrow+\infty}}n\epsilon_{n}^{2}=0$, H2 and H3-H4 defined hereafter, hold true.

• –

H3 $\exists\;c>0$, \Pi_{n}\bigl{(}{\cal U}(q_{0},\epsilon_{n})\bigr{)}>{e^{-cn\epsilon_{n}^{2}}},

• –

H4 $\mathcal{Q}_{n}\subset\mathcal{Q}$ is such that \Pi_{n}\bigl{(}\mathcal{Q}_{n}^{\complement}\bigr{)}\leq{e^{-2n(c+1)\epsilon^{2}_{n}}}.

Then for $M$ large enough,

[TABLE]

Some comments on the result of Theorem 1 as well as the hypotheses we deal with:

• –

Under H1, Theorem 1 guarantees that, for both a particular set of semi-Markov kernels ${\mathcal{Q}}$ containing some subset ${\mathcal{Q}}_{n}$ such that H2 holds true for a sequence of positive numbers $\epsilon_{n}$ and a prior distribution $\Pi_{n}$ on ${\mathcal{Q}}$ satisfying assumptions H3-H4 with $\epsilon_{n}$, the posterior distribution shrinks towards $q_{0}\in{\cal Q}$ at a rate proportional to $\epsilon_{n}$.

• –

Assumption H3 is classical in Bayesian Nonparametrics; it states that the prior distribution puts enough mass around KL neighborhoods of $q_{0}$.

• –

As mentioned in Section 3.1, ${\cal Q}_{n}$ has to be almost the support of $\Pi_{n}$: it is guaranteed by Assumption H4, which in addition quantifies how $\Pi_{n}$ covers ${\cal Q}_{n}$. If H2 holds true with $B_{d_{\nu*}}(q_{0},\epsilon)$ instead of $B_{d_{\nu*},n}(q_{0},\epsilon)$, then ${\cal Q}_{n}$ coincides with ${\cal Q}$ and Assumption H4 is no more needed.

• –

Although our semi-Markov framework differs from the Markov one, it is worth noticing that Assumption H1 is similar to the one stated as Equation (4.1) in Ghosal and van Der Vaart [17]. In particular, for Markov chains, this assumption is related to the transition probabilities of the Markov chain, whereas in our context, H1 is concerned with the SMK density.

Note also that Assumption H1 could be replaced by the following:

• –

$\widetilde{\textbf{H1}}$: There exists a strictly positive constant $C$ and a strictly positive integer $k$ such that for any ${x}\in{E}$,

[TABLE]

5 Proofs

5.1 Proof of Proposition 1

In order to prove Proposition 1, we prove that the right-hand side of Eq (2) satisfies the two relevant conditions. First, for any $A\in\mathcal{E}$, any $\Gamma\in\mathbb{B}^{+}$, we have

[TABLE]

Second,

[TABLE]

5.2 Proof of Proposition 2

Our proof is constructive; indeed, we are going to construct a suitable testing procedure, namely $\psi_{1}({\cal H}_{n})$, for the hypotheses testing problem given in (5), i.e.,

[TABLE]

To control exponentially both the type I and type II errors of $\psi_{1}({\cal H}_{n})$, we first fix some $x\in E$ for which we construct the “least favorable” pair of RN derivatives of semi-Markov kernels associated to the following auxiliary testing problem

[TABLE]

where $\lambda$ is any value in $]0,1/4[$ and $\alpha_{x}$ belongs to $]0,\pi/2[$ such that

[TABLE]

Based on this least favorable pair of $q_{x}$’s, we will then derive the construction of $\psi_{1}({\cal H}_{n})$ for the testing problem (5).

For the sake of simplicity, let us denote by $q_{x}$ and $q_{x;j}$ for $j\in{\mathrm{I}\!N}$ the probability density functions $q_{x}(\cdot,\cdot)$ and $q_{x;j}(\cdot,\cdot)$, respectively.

Least favorable pair of $q_{x}$’s for the testing problem (14)

For our purposes, we adapt the construction of Birgé [6] for Markov chains to the semi-Markov framework. Whatever is $x$ in ${E}$, we attach to $x$ a particular probability density function $q_{x;2}\in{\tilde{H}}_{1,x}$ defined by

[TABLE]

By construction, the following relations hold:

[TABLE]

Construction of the test procedure for the testing problem (5)

Next, we set $\kappa=k+l$ and $N=[n/{\kappa}]$, where $l$ and $k$ are issued from Assumption H1 and \big{[}\cdot\big{]} denotes the integer part. We consider $N$ i.i.d. random variables $Y_{1},Y_{2},\ldots,Y_{N}$, which are generated independently from $\mathcal{H}_{n}$ according to the discrete uniform distribution $\mathcal{U}_{\{1,\ldots,k\}}$.

We further define the test statistic

[TABLE]

where

[TABLE]

Our test procedure for the hypotheses problem (5) is then defined as follows

[TABLE]

• •

Test simple hypothesis vs simple hypothesis

Let us focus on the general SMPs and consider the following statistical test:

[TABLE]

To construct the testing procedure, the test statistic defined in (19), should be modified as follows:

[TABLE]

where

[TABLE]

In this case, Hypothesis H1 reduces to $\textbf{H1}^{\sharp}:$

• –

$\textbf{H1}^{\sharp}$: There exist a measure $\nu^{*}$ on $\mathcal{E}$ and a positive integer $k$ such that for any ${x}\in{E}$,

[TABLE]

Then following the steps of the proof of the Proposition 2 and replacing the Assumption H1 by $\textbf{H1}^{\sharp}$ lead us to the desired result. It is worth mentioning that in this case the inequalities (16), (17), (18) and Lemma 1 are not used.

Note also that in Proposition 2, the upper-bound of both errors is the same, equal to $\exp(-Kn\epsilon^{2}).$

Type I error probability

By means of the Markov property we obtain that

[TABLE]

where $\mathcal{H}_{\kappa(N-1)}=(J_{0},\ldots,J_{\kappa(N-1)},X_{0},\ldots,X_{\kappa(N-1)},).$

• •

Step 1

Set $T_{1}:=\mathbb{E}_{0}({\Phi_{J_{\tau_{N}-1}}(J_{\tau_{N}},X_{\tau_{N}})}|J_{\kappa(N-1)})$. Since $\tau_{i}\sim{U_{\{\kappa(i-1)+l+1,\ldots,\kappa{i}\}}}$, we obtain

[TABLE]

Next set \Gamma_{u}:=\mathbb{E}_{0}\big{[}{\Phi_{J_{\kappa(N-1)+l+u-1}}{(J_{\kappa(N-1)+l+u},X_{\kappa(N-1)+l+u})}|J_{\kappa(N-1)}}\big{]} and rewrite $\Gamma_{u}$ as follows,

[TABLE]

where the last equality is due to

[TABLE]

Assumption H1 and Eq. (17) lead us to the following upper bound of $T_{1}$:

[TABLE]

This latter inequality provides a first upper bound of $\mathbb{E}_{0}(\psi_{1}(\mathcal{H}_{n}))$ via the relation (20).

• •

Then, by setting $T_{i}:=\mathbb{E}_{0}({\Phi_{J_{\tau_{N-i+1}-1}}(J_{\tau_{N-i+1}},X_{\tau_{N-i+1}})}|J_{\kappa(N-i)})$ for $i=2,\ldots,N$, and by repeating Step 1 for the successive $T_{i}$, we finally obtain

[TABLE]

Type II error probability

To bound from above the type II error probability, we need an additional result stated as Lemma 1. This lemma provides upper bounds for a quantity which is similar to the $T_{1}$-term appearing in the first type error probability. The main difference here is that this quantity should be bounded from above uniformly over $q$ in $B_{d_{\eta^{*}},n}(q_{1},\xi\epsilon)$.

This requires the definition of the subset $G_{q}$ of $E$ by

[TABLE]

and the notation of its complementary into $E$ by $G_{q}^{\complement}$.

Lemma 1**.**

For any $\lambda\in]0,1/4[$, there exists $\iota\in[0,\frac{3}{4}[$, such that for all $q\in B_{d_{\eta^{*}},n}(q_{1},\xi\epsilon)$,

• •

if $x\in{G_{q}}$, then

[TABLE]

• •

if $x\in{G_{q}^{\complement}}$, then

[TABLE]

The proof of Lemma 1 is postponed to Section 5.3.

Consider $\Phi^{-1}$ equal to one over $\Phi$, that is $\displaystyle{\Phi^{-1}=\sqrt{\frac{q_{0}}{q_{2}}}}$. Similarly to the calculations of the type I error probability, we obtain that for any $q\in B_{d_{\eta*},n}(q_{1},\xi\epsilon)$,

[TABLE]

Similarly to $T_{1}$, we further define $W_{1}$ by

[TABLE]

• •

Step 2 Taking into account the partition of $E$ into $G_{q}$ and $G_{q}^{\complement}$, we obtain

[TABLE]

Combining with $(1-\lambda)^{2}>\displaystyle{\frac{1-3\lambda}{1-\lambda}}$ , Assumption H1 and Lemma 1 lead to,

[TABLE]

where $K(\lambda)$ is positive since there exists $\xi>0$ such that $\displaystyle{\frac{1-3\lambda}{1-\lambda}[1-\iota]}>8\displaystyle{{\frac{(1-\lambda)}{\lambda}\xi^{2}}}$.

• •

To complete the proof, we consider $W_{i}:=\mathbb{E}_{q}({\Phi^{-1}_{J_{\tau_{N-i+1}-1}}(J_{\tau_{N-i+1}},X_{\tau_{N-i+1}})}|J_{\kappa(N-i)})$ for $i=2,\ldots,N$. We then repeat Step 2 for the successive $W_{i}$, and finally deduce that for any $q\in B_{d_{\eta*},n}(q_{1},\xi\epsilon)$,

[TABLE]

with $\tilde{K}(\lambda)=K(\lambda)/\kappa$.

5.3 Proof of Lemma 1

We define the Hellinger affinity between two distributions $P_{1}$ and $P_{2}$, absolutely continuous w.r.t. $\nu$ ,with derivatives $p_{1}$ and $p_{2}$ respectively, by

[TABLE]

In the sequel, let $q$ be an arbitrary element of $B_{d_{\eta^{*}},n}(q_{1},\xi\epsilon)$.

When $x$ belongs to $G_{q}$, the proof of (21) results directly from Theorem 2 in Birgé [8].

When $x$ belongs to $G_{q}^{\complement}$, i.e., $x\in E$ such that $h_{\nu}(Q_{x},Q_{x;1})>\lambda{h_{\nu}(Q_{x;0},Q_{x;1})}$, let us prove the statement (22).

We follow the lines of Birgé [6] and consider a real number $A$ such that $A\geq\displaystyle{\frac{2}{1-\lambda}}$. We then decompose the term $\mathbb{E}_{q}[\Phi^{-1}_{J_{0}}(J_{1},X_{1})|J_{0}=x]$ into four terms:

[TABLE]

where

[TABLE]

For the sake of simplicity, set $r_{x}=\displaystyle{\frac{q_{x}}{q_{x;1}}}$ and start with $T_{0}$. Due to the definition of $\Phi^{-1}_{x}(\cdot,\cdot)$, to Equation (15) and to the concavity of the function $\displaystyle{y\rightarrow\frac{\sin(\alpha_{x})y}{\sin(\alpha_{x}\lambda)y+\sin(\alpha_{x}(1-\lambda))}}$ , we deduce that

[TABLE]

where the last inequality results from both the convexity of the $tan$ function on $]0,\pi/2[$ and $\lambda<1/4$.

Let us now turn to $T_{1}$. First note that

[TABLE]

where (24) results from the following inequality

[TABLE]

On ${\cal A}_{x;1}$, since $\displaystyle{r_{x}-1<\frac{A}{A-2}(\sqrt{r_{x}}-1)^{2}}$, then from (24) we obtain,

[TABLE]

where ${\cal A}_{x}$ is a subset of ${{\cal A}_{x;1}^{\complement}}$.

Second we study the last two terms $T_{2}$ and $T_{3}$. On ${\cal A}_{x;2}$ and ${\cal A}_{x;3}$, we first apply the Cauchy-Schwarz inequality, i.e., $\forall i\in\{2,3\}$,

[TABLE]

Second we note that

[TABLE]

where $\beta$, the upper bound of $\displaystyle{\frac{q_{x;1}}{q_{x;2}}}$, is given by \beta=\left\{\begin{array}[]{ccll}1&\mbox{{\rm on }}&{\cal A}_{x;2}&\mbox{{\rm since} }\displaystyle{\frac{q_{x;1}}{q_{x;0}}}<1,\\ \displaystyle{\frac{1}{(1-\lambda)^{2}}}&\mbox{{\rm on }}&{\cal A}_{x;3}&\mbox{{\rm due to \eqref{use-ine-sin}}}.\end{array}\right. We further note that

[TABLE]

The latter combined with (27) and since $A>2/(1-\lambda)$, entails

[TABLE]

From (23), (26) and (28), it follows that

[TABLE]

At a next step we consider the following function of $z_{x}$

[TABLE]

whose maximum is reached at

[TABLE]

Hence we obtain a new upper bound of $\mathbb{E}_{1}[\Phi^{-1}_{J_{0}}(J_{1},X_{1})|J_{0}=x]$, that is

[TABLE]

where the penultimate inequality results from the increase of the function $x\in]0,\pi/2[\rightarrow\displaystyle{\frac{\sin(\lambda x/2)}{\lambda\sin(x/2)}}$ for any $\lambda\in]0,1]$.

Finally, by setting $A=8/3$ that satisfies $A\geq 2/(1-\lambda)$ and using both inequalities \displaystyle{\sin^{2}\Big{(}(1-\lambda)\frac{\pi}{4}\Big{)}}<(1-\lambda)^{2}\Big{(}\frac{\pi}{4}\Big{)}^{2} $\forall\lambda\in]0,1/4[$ and $\displaystyle{\frac{\lambda(1-\lambda)^{2}}{1-3\lambda}<\frac{9}{16}}$ $\forall\lambda\in]0,1/4[$, Lemma 1 is proved with $\iota=\frac{\pi^{2}}{16}<3/4$. $\Box$

5.4 Proof of Corollary 1

The proof of Corollary 1 is similar to the proof of Lemma $9$ in [17]. However, we sketch it in order to define the statistical test procedure $\psi({\cal H}_{n})$. First, consider the partition:

[TABLE]

For $\xi\in]0,1[$, and any $j\geq 1$, we consider ${\widetilde{H}}_{j}$, a $j\epsilon_{n}\xi{M}$-net on ${H}_{j}$ for the distance $d_{\eta*}$ satisfying three conditions:

• •

$\forall q\in{\widetilde{H}}_{j}$, $d_{\nu*}(q_{0},q)\geq j\epsilon_{n}M$;

• •

$\forall{q}\in{H}_{j}$, $\exists q_{j}\in{\widetilde{H}}_{j}$ such that $d_{\eta*}(q,q_{j})\leq j\epsilon_{n}\xi{M}$;

• •

\log{N}\bigl{(}{\epsilon_{n}M\xi},{\widetilde{H}}_{j},d_{\eta*}\bigr{)}\leq{n\epsilon_{n}^{2}} (due to H2).

For $j\geq 1$ and any $q_{j,i}\in\widetilde{H}_{j}$, we then apply Proposition 2 with $\epsilon=jM\epsilon_{n}$ and $q_{1}=q_{j,i}$; this implies the existence of a statistical procedure $\psi_{j,i}({\cal H}_{n})$ that satisfies (6).

We then define our test procedure

[TABLE]

We further combine Assumption H2 and Proposition 2 to obtain for $M$ large enough

[TABLE]

and

[TABLE]

5.5 Proof of Theorem 1

Let $M$ be a positive constant. We first decompose the right-hand side of (13) in two parts

[TABLE]

In the sequel, each term in the right-hand side of (30) is separately bounded from above: for $A_{1}$, we apply Corollory 1, whereas to upper bound $A_{2}$ we use H3 and H4.

First, let us focus on $A_{1}$. Recall that $\mathcal{L}_{n}(q)$, the likelihood function of the sample path $\mathcal{H}_{n}$ evaluated at $q\in{\cal Q}$, is given by

[TABLE]

Then, $A_{1}$ could be written as follows:

[TABLE]

Moreover consider ${\cal D}_{n}$ as the following event:

[TABLE]

By means of the test procedure defined in (29), $\psi(\mathcal{H}_{n})$, $\mathbb{E}_{0}^{(n)}(A_{1})$ could be written as follows

[TABLE]

To bound from above $\mathbb{E}_{0}^{(n)}(A_{1})$, it is sufficient to upper bound every term in the right-hand side of (31).

• •

Term $T_{1}$. We apply Corollary 1 and obtain that there exists $K>0$ such that

[TABLE]

• •

Term $T_{2}$. We apply once again Corollary 1, which combined with H3 entails that there exists ${\tilde{K}}>0$ such that

[TABLE]

where $\kappa:={\tilde{K}}M^{2}-1-c$ is positive under the condition that $M$ is sufficiently large.

• •

Term $T_{3}$. Consider the following subspace of ${\cal Q}$

[TABLE]

and observe that

[TABLE]

It then follows from Fubini’s theorem and Markov’s inequality that

[TABLE]

Third, let us turn to $A_{2}$ which is rewritten as follows

[TABLE]

Then, using Equation (34) and from Assumptions H3 and H4, we obtain

[TABLE]

Finally, Inequalities (32)–(35) lead to the desired result (13).

Acknowledgments

The authors would like to thank the Fédération de Recherche Mathématiques des Pays de Loire (FR 2962) for their support.