Stationary Markovian Arrival Processes, Results and Open Problems

TL;DR

This paper investigates properties of Markovian arrival processes, especially MMPPs, exploring overdispersion, hazard rates, and stochastic orders, and highlights open problems and conjectures in the field.

Contribution

It provides new results and counterexamples for properties of Markovian arrival processes, and formulates conjectures about their stochastic behavior.

Findings

(I) Overdispersion holds for general MSPPs and two-state MMPPs.

(II) Non-increasing hazard rate is false for general MMPPs due to counterexamples.

(III) Variance-related property follows from stochastic order, but its validity is open for MMPPs.

Abstract

We consider two classes of irreducible Markovian arrival processes specified by the matrices $C$ and $D$. The Markov Modulated Poison Process (MMPP) and the Markovian Switched Poison Process (MSPP). The former exhibits a diagonal $D$ while the latter exhibits a diagonal $C$. For these two classes, we consider the following statements: (I) Overdispersion of the counts process. (II) A non-increasing hazard rate of the stationary inter-event time. (III) The squared coefficient of variation of the event stationary process is greater or equal to unity. (IV) A stochastic order showing that the time stationary inter-arrival time dominates the event-stationary time. For general MSPPs and two-state MMPPs, we show that (I)-(IV) hold. Then for general MMPPs, it is easy to establish (I), while (II) is false due to a counter-example of Miklos Telek and Illes Horvath. For general simple point processes, (III) follows from (IV). For MMPPs we conjecture and numerically test that (IV) and thus (III) hold. Importantly, modeling folklore has often treated MMPPs as ``bursty'' and implicitly assumed that (III) holds. However, this is still an open question.

Equations

$$c^2=\frac{\mathrm{Var}(T_1^{\mathbf{\alpha }})}{{\mathbb{E}}^2[T_1^{\mathbf{\alpha }}]},\text{mbox}and{d}^2=\underset{t\to \infty }{\lim }\frac{\mathrm{Var}(N(t))}{\mathbb{E}[N(t)]},$$

$$h(t)=\frac{f(t)}{1-F(t)}.$$

$$d^{2}=c^{2}\Big{(}1+2\sum_{j=1}^{\infty}\frac{\mathrm{Cov}(T_{0}^{\boldsymbol{\alpha}},T_{j}^{\boldsymbol{\alpha}})}{\mathrm{Var}(T_{0}^{\boldsymbol{\alpha}})}\Big{)}.$$

$$\mathbb{E}[T_{1}^{\boldsymbol{\pi}}]=\frac{1}{2}\lambda^{*}\mathbb{E}\big{[}\big{(}T_{1}^{\boldsymbol{\alpha}}\big{)}^{2}\big{]},$$

$$\lambda^{*}=\lim_{t\to\infty}\frac{E\big{[}N[0,t]\big{]}}{t}=\frac{1}{\mathbb{E}[T_{1}^{\boldsymbol{\alpha}}]}.$$

$$c^{2}=\frac{\mathbb{E}[\big{(}T_{1}^{\boldsymbol{\alpha}}\big{)}^{2}]-\big{(}\mathbb{E}[T_{1}^{\boldsymbol{\alpha}}]\big{)}^{2}}{\big{(}\mathbb{E}[T_{1}^{\boldsymbol{\alpha}}]\big{)}^{2}}=2\frac{\mathbb{E}[T_{1}^{\boldsymbol{\pi}}]}{\mathbb{E}[T_{1}^{\boldsymbol{\alpha}}]}-1,$$

$$\mathbf{\alpha }=\frac{\mathbf{\pi }D}{\mathbf{\pi }D1}\text{and}\mathbf{\pi }=\frac{\mathbf{\alpha }(-C{)}^{-1}}{\mathbf{\alpha }(-C{)}^{-1}1}={\lambda }^{\ast }\mathbf{\alpha }(-C{)}^{-1},$$

$$\mathbb{P}(T_1>t)=\mathbf{\eta }{e}^{Ct}1.$$

$$f(t)=\mathbf{\eta }{e}^{Ct}D1,h(t)=\frac{\mathbf{\eta }{e}^{Ct}D1}{\mathbf{\eta }{e}^{Ct}1}.$$

$$h^{\prime }(t)=\frac{\mathbf{\eta }C{e}^{Ct}(-C)1\mathbf{\eta }{e}^{Ct}1-\mathbf{\eta }C{e}^{Ct}1\mathbf{\eta }{e}^{Ct}(-C)1}{(\mathbf{\eta }{e}^{Ct}1{)}^2}.$$

$$M_{k}=\mathbb{E}[T_{n}^{k}]=k!\boldsymbol{\alpha}(-C)^{-k}\mathbf{1}=(-1)^{k+1}\,k!\,\frac{1}{\lambda^{*}}\boldsymbol{\pi}\big{(}C^{-1}\big{)}^{k-1}{\mathbf{1}},$$

$$M_1=\frac{1}{{\lambda }^{\ast }}\mathbf{\pi }1=\frac{1}{{\lambda }^{\ast }},M_2=2\frac{1}{{\lambda }^{\ast }}\mathbf{\pi }(-C{)}^{-1}1.$$

$$c^2+1=\frac{M_2}{M_1^2}=\frac{-2(1/{\lambda }^{\ast })\mathbf{\pi }{C}^{-1}1}{(1/{\lambda }^{\ast }{)}^2}=2\mathbf{\pi }C1\mathbf{\pi }{C}^{-1}1.$$

$$\mathbb{E}[N(t)]$$

$$\displaystyle\mathrm{Var}\big{(}N(t)\big{)}$$

$$D_Q^{♯}=\underset{t\to \infty }{\lim }{D}_Q^{♯}(t)={\int }_0^{\infty }(e^{Qu}-1\mathbf{\pi })du.$$

$$d^2=1+\frac{2}{{\lambda }^{\ast }}\mathbf{\pi }D{D}_Q^{♯}D1.$$

$$\mathbf{\pi }D1=\sum _{i=1}^p{\pi }_i{\lambda }_i,D1=\mathbf{\lambda }=({\lambda }_1,\cdots ,{\lambda }_p{)}^{\prime },\mathbf{\pi }D=({\pi }_1{\lambda }_1,\cdots ,{\pi }_p{\lambda }_p).$$

$$(I)$$

$$(II)$$

$$(III)$$

$$(IV)$$

$$\mathbf{\pi }D{D}_Q^{♯}D1\ge 0,\text{for any diagonal non-negative matrix D.}$$

$$-(\sum _{i=1}^p{\alpha }_i{c}_i^2{u}_i)(\sum _{i=1}^p{\alpha }_i{u}_i)+(\sum _{i=1}^p{\alpha }_i{c}_i{u}_i{)}^2.$$

$$-(\sum _{i=1}^p{c}_i^2{p}_i)+(\sum _{i=1}^p{c}_i{p}_i{)}^2.$$

$$Q=\left(\begin{array}[]{cccc}-1&1&0&0\\ 0&-1&1&0\\ 0&0&-1&1\\ 1&0&0&-1\end{array}\right)\qquad\mbox{and}\qquad D=\left(\begin{array}[]{cccc}\displaystyle 0.01&0&\,\,0&\,\,\,\,0\\ 0&0.01&\,\,0&\,\,\,\,0\\ 0&0&\,\,1&\,\,\,\,0\\ 0&0&\,\,0&\,\,\,\,1\end{array}\right).$$

$$(\mathbf{\pi }-\mathbf{\alpha })e^{Ct}1\ge 0,\forall t\ge 0.$$

$$(\pi -\mathbf{\alpha })e^{Ct}1$$

$$=\underset{\text{non-negative}}{\underbrace{\sum _{i=1}^{k-1}({\pi }_i-{\alpha }_i)e^{-c_{i}t}}}-\underset{\text{non-negative}}{\underbrace{\sum _{i=k}^p({\alpha }_i-{\pi }_i)e^{-c_{i}t}}}.$$

$$\sum _{i=1}^{k-1}({\pi }_i-{\alpha }_i)e^{-c_{i}t}<\sum _{i=k}^p({\alpha }_i-{\pi }_i)e^{-c_{i}t}.$$

$$\sum _{i=1}^{k-1}({\pi }_i-{\alpha }_i)e^{-c_{k-1}t}<\sum _{i=k}^p({\alpha }_i-{\pi }_i)e^{-c_{k}t}.$$

$$e^{-c_{k-1}t}\sum _{i=1}^{k-1}({\pi }_i-{\alpha }_i)<e^{-c_{k}t}\sum _{i=1}^{k-1}({\pi }_i-{\alpha }_i),$$

$$1\le c^2\le 2\frac{{\kappa }^2}{{\gamma }^2}-1,$$

Full text

Keywords: Markovian Arrival Processes (MAPs), Markov Modulated Poisson Process (MMPP), Markov switched Poisson process, overdispersion, hazard rate, squared coefficient of variation, inter-event times, stationary process.

1 Introduction

Point processes on the line, generated by transitions of Continuous Time Markov Chains (CTMCs) have been studied intensely by the applied probability community over the past few decades under the umbrella of Matrix Analytic Methods (MAM), see e.g. [16]. These have been applied to teletraffic [1], business networks [14], social operations research [25], and biological systems [20]. The typical model referred to as the Markovian Arrival Process (MAP) is comprised of a finite state irreducible CTMC which generates events at selected instances of state change and/or according to Poisson processes modulated by the CTMC. MAPs have been shown to be dense in the class of point processes so that they can essentially approximate any point process, [5]. Yet at the same time, they are analytically tractable and may often be incorporated effectively within more complex stochastic models [19].

In general, treating point processes as stationary often yields a useful mathematical perspective which matches scenarios when there is no known dependence on time. In describing a point process we use $N(t)$ to denote the number of events during $[0,t]$ and further use the sequence $\{T_{n}\}$ to denote the sequence of inter-event times. Two notions of stationarity are useful in this respect. Roughly, a point process is time-stationary if the distribution of the number of events within a given interval does not depend on the location of the interval; that is if $N(t_{1}+s)-N(t_{1})$ is distributed as $N(t_{2}+s)-N(t_{2})$ for any non-negative $t_{1},t_{2}$ and $s$. A point process is event-stationary if the joint distribution of $T_{k_{1}},\ldots,T_{k_{n}}$ is the same as that of $T_{k_{1}+\ell},\ldots,T_{k_{n}+\ell}$ for any integer sequence of indices $k_{1},\ldots,k_{n}$ and any integer shift $\ell$. For a given model of a point process, one may often consider either the event-stationary or the time-stationary case. The probability laws of both cases agree in the case of the Poisson process. However, this is not true in general. For MAPs, time-stationarity and event-stationarity are easily characterized by the initial distribution of the background CTMC. Starting it at its stationary distribution yields time-stationarity and starting at the stationary distribution of the embedded Markov chain (jump chain) yields event-stationarity.

A common way to parameterize MAPs is by considering the generator, $Q$, of an irreducible finite state CTMC and setting $Q=C+D$. Roughly speaking, the matrix $C$ determines state transitions without event counts and the matrix $D$ determines event counts. Such parameterization hints at considering two special cases: Markov Modulated Poisson Processes (MMPP) arising from a diagonal matrix $D$, and Markovian Switched Poisson Processes (MSPP) arising from a diagonal matrix $C$.

MMPPs are a widely used class of processes in modelling and are a typical example of a Cox process, also known as a doubly stochastic Poisson process, [11] and [23]. For a detailed outline of a variety of classic MMPP results, see [10] and references therein. MSPPs were introduced in [9] and to date, have not been as popular for modeling. However, the duality of diagonal $D$ vs. diagonal $C$ motivates us to consider and contrast both these processes. We also note that hyper-exponential renewal processes are special cases of MSPPs as well as Markovian Transition Counting Processes (as introduced in [3]).

Our focus in this paper is on second order properties of MMPPs and MSPPs and related traits. Consider the squared coefficient of variation and the limiting index of dispersion of counts given by,

[TABLE]

where $T_{1}^{\boldsymbol{\alpha}}$ is the time of the first event, taken from the event stationary version. Modelling folklore of MMPP sometimes assumes that $c^{2}\geq 1$. This is perhaps due to the fact that $d^{2}\geq 1$ is straightforward to verify and the similarity between these measures (for example for a renewal process, $c^{2}=d^{2}$). However, as we highlight in this paper, establishing such “burstiness” properties is not straightforward.

A related property is having $T_{1}^{\alpha}$ exhibit Decreasing Hazard Rate (DHR), where for a random variable with PDF $f(t)$ and CDF $F(t)$ the hazard rate is,

[TABLE]

A further related property is the stochastic order, $T_{1}^{\pi}\geq_{\mbox{st}}T_{1}^{\alpha}$ where $T_{1}^{\pi}$ is the first event time in the time-stationary version. We denote the properties as follows:

(I) $d^{2}\geq 1$.

(II) $T_{1}^{\boldsymbol{\alpha}}$ exhibits DHR.

(III) $c^{2}\geq 1$.

(IV) The stochastic order $T_{1}^{\boldsymbol{\pi}}\geq_{\mbox{st}}T_{1}^{\boldsymbol{\alpha}}$.

All these properties are related and in this paper we highlight relationships between (I), (II), (III) and (IV) and establish the following: For MSPPs and MMPPs of order $2$ we show that (I)–(IV) holds. For general MMPPs it is known that (I) holds however, a counter-example of Miklos Telek and Illes Horvath shows that (II) does not hold and we conjecture (and numerically test) that (III) and (IV) holds.

Our interest in this class of problems stemmed from relationships between different types of MAPs as in [24] and [3]. Once it became evident that $c^{2}\geq 1$ for MMPPs is an open problem even though it is acknowledged as a modeling fact in folklore, we searched for alternative proof avenues. This led to the stochastic order in (IV) as well as to considering DHR properties (the latter via communication with Miklos Telek and Illes Horvaths).

The remainder of the paper is structured as follows. In Section 2 we present preliminaries, focusing on the relationships between properties (I) – (IV) as well as defining MMPPs and MSPPs. In Section 3 we present our main results and the conjecture. We close in Section 4.

2 Preliminaries

Consider first properties (I)–(IV) and their relationships. With an aim of establishing property (III), $c^{2}\geq 1$, there are several possible avenues based on properties (I), (II) and (IV). We now explain these relationships.

Using property (I):

First, from the theory of simple point processes on the line, note the relationship between $d^{2}$ and $c^{2}$:

[TABLE]

However, the autocorrelation structure is typically intractable and hence does not yield results. If we were focusing on a renewal process where $T_{i}$ and $T_{j}$ are independent for $i\neq j$ then this immediately shows that $d^{2}=c^{2}$. Our focus is broader and hence property (I) indicating that $d^{2}\geq 1$ does not appear to be of use.

Using property (II):

An alternative way is to consider property (II) and use the fact that for any DHR random variable we have $c^{2}\geq 1$ (see [22]). Hence if property (II) holds then (III) holds.

Using property (IV):

We have the following Lemma, implying that (III) is a consequence of the stochastic order (IV).

Lemma 2.1**.**

Consider a simple non-transient point process on the line, and let $T_{1}^{\boldsymbol{\pi}}$, $T_{1}^{\boldsymbol{\alpha}}$ represent the first inter-event time in the time-stationary case and event-stationary case respectively. Then $c^{2}\geq 1$ if and only if $\mathbb{E}[T_{1}^{\boldsymbol{\pi}}]\geq\mathbb{E}[T_{1}^{\boldsymbol{\alpha}}]$.

Proof.

From point process theory (see for example, Eq. (3.4.17) of [8]), it holds

[TABLE]

where,

[TABLE]

Now,

[TABLE]

and we obtain the result. ∎

MAPs:

We now describe Markovian Arrival Process (MAPs). A MAP of order $p$ (MAPp) is generated by a two-dimensional Markov process $\{(N(t),X(t));t\geq 0\}$ on state space $\{0,1,2,\cdots\}\times\{1,2,\cdots,p\}$. The counting process $N(\cdot)$ counts the number of “events” in $[0,t]$ with $\mathbb{P}(N(0)=0)=1$. The phase process $X(\cdot)$ is an irreducible CTMC with state space $\{1,\ldots,p\}$, initial distribution $\boldsymbol{\eta}$ and generator matrix $Q$. A MAP is characterized by parameters $(\boldsymbol{\eta},C,D)$, where the matrix $C$ has negative diagonal elements and non-negative off-diagonal elements and records the rates of phase transitions which are not associated with an event. The matrix $D$ has non-negative elements and describes the changes of the phase process with an event (increase of $N(t)$ by 1). Moreover, we have $Q=C+D$. More details are in [4] (Chapter XI) and [12] (Chapter 2).

MAPs are attractive due to the tractability of many of their properties, including distribution functions, generating functions, and moments of both $N(\cdot)$ and the sequence of inter-event times $\{T_{n}\}$. Since $Q$ is assumed irreducible and finite, it has a unique stationary distribution $\boldsymbol{\pi}$ satisfying $\boldsymbol{\pi}Q=\mathbf{0}^{\prime}$, $\boldsymbol{\pi}\mathbf{1}=1$. Note that from $Q{\mathbf{1}}=\mathbf{0}^{\prime}$ we have $-C{\mathbf{1}}=D{\mathbf{1}}$. Of further interest is the embedded discrete-time Markov chain with irreducible stochastic matrix $P=(-C)^{-1}D$ and stationary distribution $\boldsymbol{\alpha}$, where $\boldsymbol{\alpha}P=\boldsymbol{\alpha}$ and $\boldsymbol{\alpha}\mathbf{1}=1$.

Observe the relation between the stationary distributions $\boldsymbol{\pi}$ and $\boldsymbol{\alpha}$:

[TABLE]

where $\lambda^{*}=\boldsymbol{\pi}D{\mathbf{1}}=-\boldsymbol{\pi}C{\mathbf{1}}$.

The following known proposition, as distilled from the literature (see for example [4], Chapter XI) provides the key results of MAPs that we use in this paper. It shows that $T_{1}$ is a Phase Type (PH) random variable with parameters $\boldsymbol{\eta}$ for the initial distribution of the phase and $C$ for the sub-generator matrix. It further shows that the initial distribution of the phase process may render the MAP as time stationary or event stationary.

Proposition 2.2**.**

Consider a MAP with parameters ($\boldsymbol{\eta}$,$C$,$D$), then

[TABLE]

Further, if $\boldsymbol{\eta}=\boldsymbol{\pi}$ then the MAP is time-stationary and if $\boldsymbol{\eta}=\boldsymbol{\alpha}$ it is event stationary, where $\boldsymbol{\pi}$ and $\boldsymbol{\alpha}$ are associated stationary distributions.

Note that for such a $PH(\boldsymbol{\eta},C)$ random variable the density $f(t)$ and the hazard rate $h(t)$, are respectively,

[TABLE]

Further, as may be used for showing DHR, the derivative of the hazard rate is,

[TABLE]

We now describe second-order properties associated with each case.

Event-Stationary Case:

The MAP is event-stationary333Sometimes an event-stationary MAP is referred to as an interval-stationary MAP, see for instance [10]. if $\boldsymbol{\eta}=\boldsymbol{\alpha}$. In this case, the (generic) inter-event time is phase-type distributed, $PH(\boldsymbol{\alpha},C)$ and thus has $k$-th moment:

[TABLE]

with the first and second moments (here represented in terms of $\boldsymbol{\pi}$ and $C$):

[TABLE]

The squared coefficient of variation (SCV) of events (intervals) has a simple formula:

[TABLE]

Time-Stationary Case:

A MAP with parameters $(\boldsymbol{\eta},C,D)$ is time-stationary if $\boldsymbol{\eta}=\boldsymbol{\pi}$. In the time-stationary case ($\boldsymbol{\eta}=\boldsymbol{\pi}$), we have (see [4]):

[TABLE]

where $D_{Q}^{\sharp}$ is the deviation matrix associated with $Q$ defined by the following formula.

[TABLE]

Note that in some sources, for instance [4] and [17], the variance formula (8) is presented in terms of the matrix $Q^{-}:=(\mathbf{1}{\boldsymbol{\pi}}-Q)^{-1}$. The relation between these two matrices is $Q^{-}=D_{Q}^{\sharp}+\mathbf{1}{\boldsymbol{\pi}}$, see [7].

Applying (7) and (8), we can write $d^{2}$ in terms of a MAP parameters as:

[TABLE]

MMPP:

A MAP with a diagonal matrix $D$ is an MMPP. MMPPs correspond to doubly-stochastic Poisson processes (also known as Cox processes) where the modulating process is driven by a CTMC. MMPPs have been used extensively in stochastic modelling and analysis, see for example [10]. The parameters of an MMPPp are $D=\text{diag}(\lambda_{i})$, where $\lambda_{i}\geq 0$ for $i=1,\ldots,p$, and $C=Q-D$. Here, $Q$ is the generator matrix of a CTMC. For MMPPs, (7) and (8) can be simplified by using the following relations:

[TABLE]

MSPP:

A MAP with a diagonal matrix $C$ is is an MSPP. For MSPPp events switch between $p$ Poisson processes with rates $\lambda_{1},\cdots,\lambda_{p}$, where each switch also incurs an event. Here as in MMPPs we denote the diagonal elements of $D$ via $\lambda_{1},\cdots,\lambda_{p}$. However, unlike MMPPs, (irreducible) MSPPs don’t have a diagonal $D$. We also remark that the modulation in the MSPP is of a discrete nature and it occurs at certain event epochs of the counting process, whereas the modulation of the MMPP is performed at epochs without events. See [2] and [12].

As our research attempts have shown, analyzing MSPPs is considerably easier than MMPPs, because a diagonal $C$ is much easier to handle than a non-diagonal $C$ and in an (irreducible) MMPP, $C$ must be non-diagonal.

Properties (I)-(IV) for MAPs:

Using the results above, for any irreducible MAP with matrices $C$ and $D$ we have that the main properties (I)-(IV) of this paper can be formulated as follows:

[TABLE]

3 Main Results

We now present results for MSPP and MMPP2 for properties (I)-(IV) as presented in the introduction. Establishing property (I), $d^{2}\geq 1$ is not a difficult task for both MMPPs and MSPPs:

Proposition 3.1**.**

MMPP and MSPP processes have $d^{2}\geq 1$.

Proof.

This is a well-known result that for all doubly stochastic Poisson processes (Cox processes), $d^{2}\geq 1$. So, we have the proof for an MMPP, for instance see Chapter 6 of [15].

For an MSPP, using the fact that for a given MMPP, we have $d^{2}\geq 1$, results in:

[TABLE]

On the other hand, all MAPs satisfy ${\boldsymbol{\pi}}DD_{Q}^{\sharp}D\mathbf{1}={\boldsymbol{\pi}}(-C)D_{Q}^{\sharp}(-C)\mathbf{1}$. Since for an MSPP, $-C$ is a diagonal non-negative matrix, from (15) we have (11). ∎

It isn’t difficult to show that property (II), DHR holds for MSPP:

Proposition 3.2**.**

For an MSPP the hazard rate of the stationary inter-event time is non-increasing.

Proof.

Denote the diagonal matrix $C$ with $C=diag(-c_{i})$ and the positive elements of the column vector $e^{Ct}\mathbf{1}$ with $\mathbf{u}$. So, Eq. (12) can be written element-wise as:

[TABLE]

Denoting $v_{i}=\alpha_{i}u_{i}$ and assuming $p_{i}=\frac{v_{i}}{\sum_{i=1}^{p}v_{i}}$ results in:

[TABLE]

The above expression can be viewed as the minus variance of a random variable that takes values $c_{i}$ with probability $p_{i}$. Therefore, we have (12).

∎

However, somewhat surprisingly, MMPPs don’t necessarily possess DHR. An exception is MMPP2 as shown in Proposition 3.6. However for higher order MMPPs DHR doesn’t always hold. The gist of the following example was communicated to us by Milkos Telek and Illes Horvath. Set

[TABLE]

As shown in Figure 1, the hazard rate function for an MMPP with the above matrices is not monotone. Hence at least for general MMPPs, trying to show (III), $c^{2}\geq 1$, via hazard rates is not a viable avenue.

Since hazards rates don’t appear to be a viable paths for establishing (III) for MMPPs, an alternative may be to consider the stochastic order (IV). Starting with MSPPs, we see that this property holds.

Proposition 3.3**.**

For an MSPP $T_{1}^{\boldsymbol{\pi}}\geq_{\mbox{st}}T_{1}^{\boldsymbol{\alpha}}$.

Proof.

Using (14), the claim is,

[TABLE]

Without loss of generality we assume that there is an order $0<c_{1}\leq c_{2}\leq\cdots\leq c_{p}$ (with $c_{i}\neq c_{j}$ for some $i,j$) for diagonal elements of matrix $(-C)$. There is a possibility that for $1<p^{\prime}<p$, $0=c_{1}=c_{2}=\ldots=c_{p^{\prime}-1}$, and $0<c_{p^{\prime}}$, however, in the rest of the proof, we assume that $p^{\prime}=1$, meaning that all $c_{i}$ are strictly positive. Adapting to the case of $p^{\prime}>1$ is straightforward.

Now, $\{\lambda^{*}-c_{i}\}_{i=1,\cdots,p}$ is a non-increasing sequence and therefore in the sequence $\{\pi_{i}-\alpha_{i}\}=\{\frac{\pi_{i}}{\lambda^{*}}(\lambda^{*}-c_{i})\}$ when an element $\pi_{k}-\alpha_{k}$ is negative, all the elements $\pi_{i}-\alpha_{i}$ for $i\geq k$ are negative. Moreover, both $\boldsymbol{\pi}$ and $\boldsymbol{\alpha}$ are probability vectors, so $(\boldsymbol{\pi}-\boldsymbol{\alpha})\mathbf{1}=\sum_{i}(\pi_{i}-\alpha_{i})=0$. Therefore, at least the first element in the sequence $\{\pi_{i}-\alpha_{i}\}=\{\frac{\pi_{i}}{\lambda^{*}}(\lambda^{*}-c_{i})\}$ is positive. Hence, there exists an index $1<k\leq p$ such that $\pi_{i}-\alpha_{i}$ for $i=1,\cdots,k-1$ is non-negative and for $i=k,\cdots,p$ is negative. Therefore, we have:

[TABLE]

Assume: $({\pi}-{\alpha})e^{Ct}\mathbf{1}<0$ :

[TABLE]

Then, since $0<c_{1}\leq c_{2}\leq\cdots\leq c_{p}\,$, we have $e^{-c_{1}t}\geq e^{-c_{2}t}\geq\cdots\geq e^{-c_{p}t}$. Now from (19) we can conclude that:

[TABLE]

Using the fact that $\sum_{i=k}^{p}({\alpha_{i}}-{\pi_{i}})=\sum_{i=1}^{k-1}({\pi_{i}}-{\alpha_{i}})$, results in:

[TABLE]

which is not true. Consequently, the assumption $({\pi}-{\alpha})e^{Ct}\mathbf{1}<0$ is not true and hence (14) holds.

∎

Hence via Lemma 2.1 or alternatively via the DHR property in Proposition 3.2 we have:

Corollary 3.4**.**

For an MSPP $c^{2}\geq 1$.

In fact, for MSPPs this is an easy result and it can also be proved independently by using the Cauchy-Schwarz inequality. Further, we can find an upper bound:

Proposition 3.5**.**

An MSPP with diagonal matrix $C=-\text{diag}(c_{i})$ for $i=1,\cdots,p$ satisfies

[TABLE]

where $\kappa=\frac{\min{c_{i}}+\max{c_{i}}}{2}$ and $\gamma=\sqrt{(\min{c_{i}})(\max{c_{i}})}$.

Proof.

From Eq. (6), we have $c^{2}+1=2(\boldsymbol{\pi}C{\mathbf{1}}\boldsymbol{\pi}C^{-1}{\mathbf{1}})$. For $C=-\text{diag}(c_{i})$, we have $C^{-1}=-\text{diag}(\frac{1}{c_{i}})$ and so,

[TABLE]

On the other hand, from the Cauchy-Schwarz inequality and the Kantorovich’s Inequality (see [21]), we have:

[TABLE]

Combination of the above two equations results in:

[TABLE]

which completes the proof.

∎

However, for MMPPs while we believe the result is true (see numerical evidence in the next section) we don’t have a general proof for properties (III) or (IV). Still, for two state MMPPs (MMPP2) things are easier and we are able to show that all properties (I)-(IV) hold:

Proposition 3.6**.**

For a two-state MMPP2, $c^{2}>1$ and $d^{2}>1$, $h(t)$ is DHR and the stochastic order $T_{1}^{\boldsymbol{\pi}}\geq_{\mbox{st}}T_{1}^{\boldsymbol{\alpha}}$ holds.

Proof.

Consider an MMPP2 with parameters

[TABLE]

Then, $\boldsymbol{\pi}=\frac{1}{\sigma_{1}+\sigma_{2}}(\sigma_{2},\,\sigma_{1})$. As in [13], evaluation of the transient deviation matrix through (for e.g.) Laplace transform inversion yields:

[TABLE]

Therefore from (1), we have

[TABLE]

Further, explicit computation yields,

[TABLE]

Thus it is evident that the MMPP2 has $d^{2}>1\,,c^{2}>1$ as long as $\lambda_{1}\neq\lambda_{2}$ and $d^{2}=c^{2}=1$ when $\lambda_{1}=\lambda_{2}$.

For DHR and the stochastic order, first we note that for an MMPP2 with the above parameters, $\boldsymbol{\alpha}=\frac{1}{\sigma_{1}\lambda_{1}+\sigma_{2}\lambda_{2}}(\sigma_{1}\lambda_{1},\sigma_{2}\lambda_{2})$. By setting $B=\sigma_{1}+\sigma_{2}+\lambda_{1}+\lambda_{2}$ and $A=\sigma_{2}\lambda_{1}+\lambda_{2}(\sigma_{1}+\lambda_{1})$, after some simplification, Eq. (12) is given by:

[TABLE]

which is strictly negative for $\lambda_{1}\neq\lambda_{2}$ and is zero for $\lambda_{1}=\lambda_{2}$. For the stochastic order, from Eq. (14), we have:

[TABLE]

which is strictly positive for $\lambda_{1}\neq\lambda_{2}$ and is zero for $\lambda_{1}=\lambda_{2}$.

∎

4 Conjectures for MMPP

We embarked on this research due to the folklore assumption that for MMPP, $c^{2}\geq 1$ (III). Initially we believed that it is easy to verify, however to date there isn’t a known proof for an arbitrary irreducible MMPP. Still, we conjecture that both (III) and (IV) hold for MMPPs:

Conjecture 4.1**.**

For an irreducible MMPP, $c^{2}~{}\geq~{}1$.

Conjecture 4.2**.**

For an irreducible MMPP, $T_{1}^{\boldsymbol{\pi}}\geq_{\mbox{st}}T_{1}^{\boldsymbol{\alpha}}$.

In an attempt to disprove these conjectures or alternatively gain confidence in their validity, we carried out an extensive numerical experiment. Our experiment works by generating random instances of MMPPs . Each instance is generated by first generating a matrix $Q$ with uniform$(0,1)$ off-diagonal entries and diagonal entries that ensure row sums are [math]. We then generate a matrix $D$ with diagonal elements that are exponentially distributed with rate $1$. Such a $(Q,D)$ pair then implies ${\boldsymbol{\pi}}$ and ${\boldsymbol{\alpha}}$. For each such MMPP we calculate $\boldsymbol{\pi}C{\mathbf{1}}\boldsymbol{\pi}C^{-1}{\mathbf{1}}-1$ as in (13) and $(\boldsymbol{\pi}-\boldsymbol{\alpha})e^{Ct}\mathbf{1}$ as in (14), where we take $t\in\{0,0.2,0.4,\ldots,9.8,10.0\}$. We then ensure that both of these quantities are non-negative.

We repeated this experiment for $10^{6}$ random MMPP instances of orders $3,4,5$ and $6$. In all cases the calculated quantities were greater that $-10^{-15}$. Note that in certain cases, the quantity associated with (IV) was negative and lying in the range $(-10^{-15},-10^{-16}]$. We attribute this to numerical error stemming from the calculation of the matrix exponential $e^{Ct}$. We ran our experiments with the Julia programming language, V1.0. The calculation time was about 1.5 hours.

This provides some evidence for the validity of Conjectures 1 and 2, although it is clearly not a proof. Further, we note that it is possible that some extreme cases exist that are not likely to come up by uniformly and randomly generating entries of $Q$. For example the cyclic matrix $Q$ in (17). For this we have also considered random cyclic $Q$ matrices with non-zero entries similar to (17). We generated $10^{6}$ such (order $4$) examples and all agreed with (III) and (IV).

5 Conclusion

We have highlighted various related properties for point processes on the line and MAPs exhibiting diagonal matrices ($C$ or $D$) in particular. Showing that $c^{2}\geq 1$ for MMPP and establishing the stochastic order $T_{1}^{\boldsymbol{\pi}}\geq_{\mbox{st}}T_{1}^{\boldsymbol{\alpha}}$ remains an open problem. We have shown this for MMPPs of order $2$ and using a similar technique to our MSPP proof, we can also show it for MMPPs with symmetric $C$ matrices. However for general MMPPs this remains an open problem.

We note, that stepping outside of the matrix analytic paradigm and considering general Cox processes is also an option. In fact, since any Cox process can be approximated by an MMPP, we believe that versions of conjectures $1$ and $2$ also hold for Cox processes under suitable regularity conditions.

There is also a related branch of questions dealing with characterizing the Poisson process via $c^{2}=1$ and considering when an MMPP is Poisson. For example, for the general class of MAPs, the authors of [6] provide a condition for determining if a given MAP is Poisson. It is not hard to construct a MAP with $c^{2}=1$ that is not Poisson. But, we believe that all MMPPs with $c^{2}=1$ are Poisson. Yet, we don’t have a proof. Further, we believe that for an MMPP, if $c^{2}=1$ then all $\lambda_{i}$ are equal (the converse is trivially true). We don’t have a proof of this either. Related questions also hold for the more general Cox processes.

We also note that the MSPP class of process that we considered generalized hyper-exponential renewal processes as well as a class of processes called Markovian Transition Counting Processes (MTCP) as in [3].

Acknowledgement

Azam Asanjarani’s research is supported by the Australian Research Council Centre of Excellence for the Mathematical and Statistical Frontiers (ACEMS). Yoni Nazarathy is supported by Australian Research Council Grant DP180101602. We thank Soren Asmussen, Qi-Ming He, Illes Horvath, Peter Taylor and Miklos Telek for useful discussions and insights related to this problem.