On the Informativeness of Measurements in Shiryaev's Bayesian Quickest Change Detection

TL;DR

This paper uncovers a super-martingale phenomenon in Shiryaev's Bayesian quickest change detection, showing it can hinder detection when measurements lack sufficient informativeness, especially for subtle changes.

Contribution

It is the first to describe a weak super-martingale phenomenon in Shiryaev's Bayesian QCD and links it to measurement informativeness and relative entropy.

Findings

Super-martingale phenomenon can occur under certain entropy conditions.

The phenomenon indicates limitations of Shiryaev's test for subtle changes.

Insufficiently informative measurements impair detection performance.

Abstract

This paper provides the first description of a weak practical super-martingale phenomenon that can emerge in the test statistic in Shiryaev's Bayesian quickest change detection (QCD) problem. We establish that this super-martingale phenomenon can emerge under a condition on the relative entropy between pre and post change densities when the measurements are insufficiently informative to overcome the change time's geometric prior. We illustrate this super-martingale phenomenon in a simple Bayesian QCD problem which highlights the unsuitability of Shiryaev's test statistic for detecting subtle change events.

Equations

$$J(\tau)=\ cE_{\pi}\bigg{[}(\tau-\nu)^{+}\bigg{]}+P_{\pi}(\tau<\nu),$$

$$\tau =\inf \left\{k\ge 1:{\hat{X}}_k^1<1-h\right\},$$

$${\hat{X}}_k^1=N_k(1-\rho )b^1(y_k){\hat{X}}_{k-1}^1$$

$$N_k^{-1}=b^2(y_k)+(1-\rho )\left(b^1(y_k)-b^2(y_k)\right){\hat{X}}_{k-1}^1.$$

$$A=\left[\begin{array}{cc}1-\rho & 0\\ \rho & 1\end{array}\right],$$

$${\hat{X}}_k=N_k\text{mbox}diag([b^1(y_k),b^2(y_k)])A{\hat{X}}_{k-1}$$

$$N_k^{-1}$$

$$E_{\pi}\left[\log(M_{k})\Big{|}\hat{X}_{k-1}^{1}\right]<\log(1-\rho)+D\left(b^{1}(y_{k})\big{|}\big{|}{b^{2}(y_{k})}\right)+\delta.$$

$${\gamma }_k\triangleq \log \left(b^2(y_k)\right)$$

$$-\log \left(b^2(y_k)+(1-\rho )(b^1(y_k)-b^2(y_k)){\hat{X}}_{k-1}^1\right).$$

$$\displaystyle E_{\pi}\left[\log(N_{k})\Big{|}\hat{X}_{k-1}^{1}\right]$$

$$E_{\pi }[\log (M_k)|{\hat{X}}_{k-1}^1]$$

$$\displaystyle E_{\pi}\left[\log\left(\frac{b^{1}(y_{k})}{b^{2}(y_{k})}\right)\Big{|}\hat{X}_{k-1}^{1}\right]=$$

$$P_{\pi }\left(\text{mbox}foralln\ge k,E_{\pi }[\log ({\hat{X}}_{n+1}^1)|\log ({\hat{X}}_n^1)]<\log ({\hat{X}}_n^1)\right)$$

$$>1-{\delta }_p.$$

$$D\left(b^{1}(y_{k})\big{|}\big{|}{b^{2}(y_{k})}\right)<\log(1/(1-\rho))$$

$$E_{\pi }[\log ({\hat{X}}_k^1)|\log ({\hat{X}}_{k-1}^1)]<\log ({\hat{X}}_{k-1}^1)$$

$$P_{\pi }\left(\underset{n\ge k}{\max }{Z}_n\ge (\log (h_{\delta }/h_m))\right)\le \frac{E_{\pi }[Z_k]}{\log (h_{\delta }/h_m)}.$$

$$P_{\pi }\left(\underset{n\ge k}{\max }\log ({\hat{X}}_n^1)\ge \log (h_{\delta })\right)<\frac{\log (h_s/h_m)}{\log (h_{\delta }/h_m)}.$$

$$P_{\pi }\left(\underset{n\ge k}{\max }\log ({\hat{X}}_n^1)<\log (h_{\delta })\right)>1-{\delta }_l$$

$$\mathcal{M}(\rho )\triangleq \left\{m:\frac{m^2}{2}<\log (1/(1-\rho ))\right\}.$$

Full text

, ,

1 Introduction

Quickly detecting a change in the statistics of a process is an important signal processing problem with application in a diverse range of areas including: automatic control [1], quality control[2, 3, 1], statistics [4], target detection[5, 6] and many more [7, Ch, 1.3]. In the classic Bayesian quickest change detection (QCD) problem, it is assumed that a permanent change in the statistics of an observed process occurs at some random time (see [7, Ch. 1.2] for a comparison with non-Bayesian QCD). The classic Bayesian QCD objective is to minimize the average detection delay subject to a constraint on the probability of a false alarm. When the change time has a geometric prior, Shiryaev established the optimal stopping rule as a test of whether the change posterior probability is above a threshold [8]. This paper investigates the properties of Shiryaev’s famous test statistic in weak measurement environments.

The main contribution here is to provide the first report and characterization of a super-martingale phenomenon in Shiryaev’s Bayesian QCD problem (see [3] and [9] for extensive investigations of martingale phenomenon in other QCD rules). This paper introduces a new weak practical super-martingale concept and exploits the maximal inequality for non-negative supermartingales to characterise of conditions under which the Bayesian QCD measurements are not sufficiently informative and Shiryaev’s test statistic is dominated by the change time’s prior. Interestingly, the identified super-martingale phenomenon appears suddenly once an information theoretic requirement on the pre and post change densities holds (rather than emerging as a graceful degradation). Practically, in applications with weak measurements, these observations motivate consideration of subtle problem adjustments, such as in the quickest intermittent signal detection problem [10] which generalizes Shiryaev’s problem for use in a vision-based aircraft detection application, or using non-Bayesian QCD such as the Lorden criterion [11].

The specific contributions are:

1. (i)

Establishing a condition in terms of the change time’s geometric prior and the relative entropy between pre and post change densities that identifies when measurements are insufficiently informative. 2. (ii)

Establishing that when measurements are insufficiently informative, Shiryaev’s test statistic can exhibit a super-martingale phenomenon; that is, the log of no change posterior is a weak practical super-martingale. 3. (iii)

Providing an example exhibiting this super-martingale phenomenon to illustrate a situation where Shiryaev’s Bayesian QCD approach is potentially unsuitable for detecting subtle change events.

We would expect similar phenomenon to emerge in recent Bayesian QCD generalizations involving non-ergodic models [12].

2 Shiryaev’s Bayesian Quickest Change Detection Problem and Optimal Solution

For $k>0$, let $y_{k}\in\mathcal{Y}$ be an independent and individually distributed (i.i.d.) sequence of random variables taking values in the set $\mathcal{Y}\subseteq\mathbb{R}^{M}$. Initially, the random variables $y_{k}$ have a pre change (marginal) probability density $b^{1}(\cdot)$ before, at some random change time $\nu\geq 1$, switching to having a post change (marginal) probability density $b^{2}(\cdot)$. We will assume for $i\in\{1,2\}$, that $b^{i}(.)<B$ for some finite $B<\infty$. For $k\geq 0$, let random variable $X_{k}\in\{e_{1},e_{2}\}$ denote a change event process in the sense that $X_{k}=e_{1}$ for $k<\nu$ and $X_{k}=e_{2}$ for $k\geq\nu$. Here $e_{i}\in R^{2}$ are indicator vectors with 1 in the $i$th element, and zero elsewhere. Let $y_{[1,k]}\triangleq\{y_{1},\ldots,y_{k}\}$ be shorthand for measurement sequences.

Before we formally state Shiryaev’s Bayesian QCD problem, let us first introduce a probability measure space. Let $\mathcal{F}_{k}=\sigma(y_{[1,k]})$ denote the filtration generated by $y_{[1,k]}$. We will assume the existence of a probability space $(\Omega,\mathcal{F},P_{\nu})$ where $\Omega$ is a sample space of sequences $y_{[1,\infty]}$, $\sigma$-algebra $\mathcal{F}=\cup_{k=1}^{\infty}\mathcal{F}_{k}$ with the convention that $\mathcal{F}_{0}=\{0,\Omega\}$, and $P_{\nu}$ is the probability measure constructed using Kolmogorov’s extension on the joint probability density function of the observations $p_{\nu}(y_{[1,k]})=\Pi_{i=1}^{\nu-1}b^{1}(y_{i})\Pi_{j=\nu}^{k}b^{2}(y_{j})$ where we define $\Pi_{j=\nu}^{k}b^{2}(y_{j})=1$ when $k<\nu$. We will let $E_{\nu}$ denote expectation under $P_{\nu}$ and use the probability measure $P_{\infty}$ and expectation $E_{\infty}$ to denote the special case when there is no change event. We let D\left(b^{1}(y_{k})\big{|}\big{|}{b^{2}(y_{k})}\right)\triangleq E_{\infty}\left[\log\left(\frac{b^{1}(y_{k})}{b^{2}(y_{k})}\right)\right] denote the relative entropy between pre and post change densities.

In Bayesian QCD problem the change time $\nu\geq 1$ that $X_{k}$ transitions from $e_{1}$ to $e_{2}$ is considered to be an unknown random variable with prior distribution $\pi_{k}\triangleq P(\nu=k)$ for $k\geq 1$. This allows us to construct a new averaged measure $P_{\pi}(G)=\sum_{k=1}^{\infty}\pi_{k}P_{k}(G)$ for all $G\in\mathcal{F}$ and we let $E_{\pi}$ denote the corresponding expectation operation. In Shiryaev’s problem we consider the special case of the geometric prior $\pi_{k}=(1-\rho)^{k-1}\rho$ for some $\rho\in(0,1)$ (and set $\pi_{k}\triangleq 0$, $k<1$).

Let $\tau>0$ be a stopping time with respect to filtration $\mathcal{F}_{k}$. We can now introduce the Shiryaev cost criterion [8] to trade-off average detection delay with probability of false alarm as

[TABLE]

where $(\tau-\nu)^{+}\triangleq\max(0,\tau-\nu)$, $c$ is the delay penalty and the problem is to minimise $\inf_{\tau}J(\tau)$.

For $i\in\{1,2\}$, let the no change and change posterior probabilities be denoted $\hat{X}_{k}^{i}\triangleq P_{\pi}(X_{k}=e_{i}|y_{[1,k]})$, respectively. Noting that we can write $\hat{X}_{k}^{2}=1-\hat{X}_{k}^{1}$, allows us to write Shiryaev’s optimal stopping rule for this cost criterion in terms of the no change posterior probability as

[TABLE]

where $h$ is a threshold selected to control the probability of false alarm, as it can be shown that the probability of false alarm satisfies $P_{\pi}(\tau<\nu)<1-h$ [10].

3 The Emergence of the Super-Martingale Phenomenon

To develop conditions under which the test statistic $\hat{X}^{1}_{k}$ of Shiryaev’s rule exhibits rapid decrease even in the no change regime, we first introduce the following result that establishes how to efficiently calculate it.

Lemma 1**.**

For $k>0$, given a sequence of measurements $y_{[1,k]}$ the no change posterior probability $\hat{X}^{1}_{k}$ is given by the scalar recursion

[TABLE]

with $\hat{X}_{0}^{1}=1$ and the normalization factor

[TABLE]

{pf}

As defined above, $X_{k}$ is a first order time-homogeneous Markov chain whose transition probabilities at each time instant are given by $A^{i,j}\triangleq P(X_{k+1}=e_{i}|X_{k}=e_{j})$ for $i,j\in\{1,2\}$ as

[TABLE]

where $X_{0}=e_{1}$, and $X_{k}$ is observed via the random variables $y_{k}$. Hence, the no change posterior $\hat{X}_{k}^{1}$ can efficiently be calculated by hidden Markov model filter [13], where $\hat{X}_{k}=[\hat{X}_{k}^{1},\hat{X}_{k}^{2}]^{\prime}$,

[TABLE]

where $N_{k}=\langle\mbox{diag}([b^{1}(y_{k}),b^{2}(y_{k})])A\hat{X}_{k-1},\underline{1}\rangle^{-1}$ and with $\hat{X}_{0}=e_{1}$. Noting that $\hat{X}_{k}^{2}=1-\hat{X}_{k}^{1}$, then simple algebra lets us write (2). Then we note that

[TABLE]

giving (3). This completes the proof.

To facilitate characterization of our test statistic’s behaviour let us introduce $M_{k}\triangleq N_{k}(1-\rho)b^{1}(y_{k})\in R$, noting that we can write $\log(\hat{X}_{k}^{1})=\log(M_{k})+\log(\hat{X}_{k-1}^{1})$, and establish the following bound on $\log(M_{k})$.

Lemma 2**.**

($\hat{X}_{k}$ dependent bound on $\log(M_{k})$) For any $\delta>0$, there is a $h_{\delta}>0$ such that for any $\hat{X}_{k-1}^{1}<h_{\delta}$ we have

[TABLE]

{pf}

We define

[TABLE]

Using (3) lets us write

[TABLE]

It then follows from (5) and the definition of $M_{k}$ that

[TABLE]

Noting that $\log(x)$ is a continuous (monotonic increasing) in $x$ and that $b^{i}(.)<B$ are finite gives that for any $\delta>0$, there is a $h_{\delta}>0$ such that for any $\hat{X}_{k-1}^{1}<h_{\delta}$ we have E_{\pi}\left[\gamma_{k}\Big{|}\hat{X}_{k-1}^{1}\right]\leq\delta, and therefore (6) gives that

[TABLE]

Then using $P_{\pi}(y_{k}|\hat{X}_{k-1}^{1})=P_{\pi}(X_{k}=e_{1}|\hat{X}_{k-1}^{1})P_{\pi}(y_{k}|X_{k}=e_{1},\hat{X}_{k-1}^{1})+P_{\pi}(X_{k}=e_{2}|\hat{X}_{k-1}^{1})P_{\pi}(y_{k}|X_{k}=e_{2},\hat{X}_{k-1}^{1})$ and $P_{\pi}(X_{k}=e_{1}|\hat{X}_{k-1}^{1})=(1-\rho)\hat{X}_{k-1}^{1}$ gives

[TABLE]

because $(1-\rho)\hat{X}_{k-1}^{1}<1$, $(1-(1-\rho)\hat{X}_{k-1}^{1})\geq 0$ and $E_{0}\left[\log\left(\frac{b^{2}(y_{k})}{b^{1}(y_{k})}\right)\right]\geq 0$. Substitution of (8) into (7) gives the lemma result.

Recall that we can write $\log(\hat{X}_{k}^{1})=\log(M_{k})+\log(\hat{X}_{k-1}^{1})$. Hence Lemma 2 provides a bound on the test statistic increment $\log(M_{k})$ which allows us to investigate conditions under which the measurements are insufficient to overcome the geometric prior information $\log(1-\rho)<0$, and $\log(\hat{X}_{k}^{1})$ becomes a weak practical super-martingale in the following sense:

Definition 3**.**

*(Weak practical super-martingale) If for any arbitrarily small $\delta_{p}>0$ there exists a $h_{s}>0$ such that if $\hat{X}_{k}^{1}<h_{s}$ then *

[TABLE]

and the log of the no change posterior probability $\log(\hat{X}_{k}^{1})$ is called a weak practical super-martingale.

We now establish our theorem which provides conditions under which measurements are insufficiently informative and this super-martingale phenomenon emerges.

Theorem 4**.**

(Insufficiently informative measurements) If the relative entropy between probability densities ${b^{1}(\cdot)}$ and ${b^{2}(\cdot)}$ is sufficient small, namely

[TABLE]

*then the measurements are insufficiently informative in the sense that $\log(\hat{X}_{k}^{1})$ is a weak practical super-martingale (cf. Definition 3). *

{pf}

From Lemma 2, the bound (9) gives that there exists a $h_{\delta}>0$ such that for $\hat{X}_{k-1}^{1}<h_{\delta}$ we have $E_{\pi}[\log(M_{k})|\hat{X}_{k-1}^{1}]<0$ and hence that $\log(\hat{X}_{k}^{1})$ satisfies the super-martingale property

[TABLE]

noting $E_{\pi}[\log(\hat{X}_{k}^{1})|\log(\hat{X}_{k-1}^{1})]=E_{\pi}[\log(M_{k})|\log(\hat{X}_{k-1}^{1})]+\log(\hat{X}_{k-1}^{1})$ and that conditioning on $\log(\hat{X}_{k-1}^{1})$ and $\hat{X}_{k-1}^{1}$ are equivalent. It remains to establish if $\hat{X}_{k-1}^{1}$ remains trapped in $[0,h_{\delta})$ or escapes.

Let us introduce $h_{s}\triangleq\beta h_{\delta}$ and $h_{m}\triangleq\beta\eta h_{\delta}$, with some $\beta,\eta<1$ as bounding parameters to manage our possibly unbounded $\log(\hat{X}_{k-1}^{1})$ super-martingale process. We define a new process $Z_{k}\triangleq\max(\log(\hat{X}_{k}^{1}/h_{m}),0)$. We now note that (10) gives that $Z_{k}$ is a non-negative super-martingale and hence by the maximal inequality for non-negative super-martingales (cf. [14, Lemma 1]) we have, for any $k$. that

[TABLE]

Noting that $Z_{n}\geq\log(\hat{X}_{n}^{1}/h_{m})$ and that if $\hat{X}_{k}^{1}<h_{s}$ then $E_{\pi}[Z_{k}]<\log(h_{s}/h_{m})$ gives

[TABLE]

Rewriting in terms of the complimentary set for the maximal event gives, if $\hat{X}_{k}^{1}<h_{s}$ that

[TABLE]

where $\delta_{l}\triangleq\log(h_{s}/h_{m}))/\log(h_{\delta}/h_{m})$ can be written as $\delta_{l}=\log(\eta)/\log(\beta\eta)$. We note that the event $\max_{n\geq k}\log(\hat{X}_{n}^{1})<\log(h_{\delta})$ implies for all $n\geq k$ that $\hat{X}_{n}^{1}<h_{\delta}$ and hence by Lemma 2 that (10) holds for all $n\geq k$. The theorem result follows by noting that for any $\delta_{p}>0$ we can find a $\beta$ (or equivalently a $h_{s}>0$) so that $\delta_{l}\leq\delta_{p}$ and the Definition 3 property holds.

Theorem 4 establishes that unless the relative entropy between pre and post change densities D\left(b^{1}(y_{k})\big{|}\big{|}{b^{2}(y_{k})}\right) is sufficiently large, the no change posterior $\hat{X}_{k}^{1}$ is a weak practical super-martingale under $E_{\pi}$ and hence there exists a trap defined by the interval $\hat{X}_{k}^{1}<h_{\delta}$ where Shiryaev’s test statistic becomes increasingly confident that the change has occurred even if it has not. Further, we note that on sufficiently long sequence of measurements there is non zero probability of entering the interval $\hat{X}_{k}^{1}<h_{\delta}$. A test statistic that can exhibit such incorrect increasing confidence on non-pathological sequences is problematic in a practical setting and hence we interpret the existence of this interval trap under the condition of Theorem 4 as meaning the measurements are insufficiently informative. To understand the behaviour of Shiryaev’s rule and the role of the relative entropy D\left(b^{1}(y_{k})\big{|}\big{|}{b^{2}(y_{k})}\right) first consider the limit case $b^{1}(\cdot)=b^{2}(\cdot)$. In this case, D\left(b^{1}(y_{k})\big{|}\big{|}{b^{2}(y_{k})}\right) is zero, the posterior is given by $\hat{X}_{k}^{1}=(1-\rho)^{k}$, Shiryaev’s rule becomes the deterministic rule to stop at the earliest time at or after $\log(1-h)/\log(1-\rho)$ and $h_{\delta}=1$. Informally, a similar geometric prior $(1-\rho)^{k}$ mechanism is driving the super-martingale phenomenon that occurs when D\left(b^{1}(y_{k})\big{|}\big{|}{b^{2}(y_{k})}\right) is non-zero but less than $\log(1/(1-\rho))$, with $h_{\delta}\in(0,1)$. Finally, we note that as D\left(b^{1}(y_{k})\big{|}\big{|}{b^{2}(y_{k})}\right) increases towards the critical value of $\log(1/(1-\rho))$ then $h_{\delta}$ decreases towards [math], and the probability of entering the trap interval decays.

4 Example: Bayesian Quickest Change Detection With Gaussian Densities

Proposition 5**.**

Consider Shiryaev’s Bayesian quickest change detection problem with pre and post change (marginal) probabilities densities $b^{1}(y_{k})=\frac{1}{2\pi}\exp(-y_{k}^{2}/2)$ and $b^{2}(y_{k})=\frac{1}{2\pi}\exp(-(y_{k}-m)^{2}/2)$. Consider the set

[TABLE]

$\mathcal{M}(\rho)$* is non-empty. Further, there exists a $m_{c}$ such that $\mathcal{M}(\rho)$ has a threshold structure in the sense of $\mathcal{M}(\rho)=\left\{m:0<m<m_{c}\right\}$, where $m_{c}=\sqrt{2\log(1/(1-\rho))}$. Finally, when $m\in\mathcal{M}(\rho)$, then the measurements are insufficiently informative in that $\log(\hat{X}_{k}^{1})$ is a weak practical super-martingale (cf. Definition 3). *

{pf}

To establish that $\mathcal{M}(\rho)$ is non-empty we note that for any $\rho>0$ there exist a $\epsilon>0$ such that $\log\left(1/(1-\rho)\right)>\epsilon$. As $\lim_{m\rightarrow 0}\left[\frac{m^{2}}{2}\right]=0$ then for any $\epsilon>0$, there must be at least one $m>0$ such that $\frac{m^{2}}{2}<\epsilon$ and hence this $m>0$ as an element of the non-empty $\mathcal{M}(\rho)$. The interval result follows by noting that if $m\in\mathcal{M}(\rho)$ then $\beta m\in\mathcal{M}(\rho)$ for all $0<\beta\leq 1$, and this means the set $\mathcal{M}(\rho)$ can be described as the interval $\left\{m:0<m<m_{c}\right\}$ with some critical largest element $m_{c}$. Algebra and the monotonic increasing nature of $m^{2}/2$ gives that $m_{c}=\sqrt{2\log(1/(1-\rho))}$. The final result follows from Lemma 2 and noting that the relative entropy between these two Gaussians is given by $\frac{m^{2}}{2}$ [1, Example 4.1.9] .

Simulation:

Consider a geometric prior $\rho=0.05$ and note from Proposition 5 that the phenomenon emerges below $m_{c}=0.32$. Figure 1 illustrates two simulated examples of the posterior’s behaviour on a sequence prior to the change time ($m=0.40$ and $m=0.23$ representing examples of informative and non-informative measurements). The significantly different behaviour seen is an illustration of the super-martingale phenomenon discussed in this paper.

To illustrate the transition in the behaviour of Shiryaev’s test statistic, for each value of $m=0.1,0.15,\ldots,0.6$ we conducted a Monte-Carlo study of 1000 trials of 5000 long random variable sequences with no change. In Figure 2 the mean value of $\hat{X}^{1}_{5000}$ ($\rho=0.05$) illustrates that below the critical value $m_{c}=0.32$ the test statistic exhibits the super-martingale phenomenon and becomes incorrectly convinced that a change has occurred, when it has not.

5 Discussion

The super-martingale phenomenon emerges in Bayesian QCD as a consequence of the non-ergodic nature of the underlying signal model. That the class of post-change densities exhibiting the phenomenon can by parameterized by an interval set suggests this is a systemic issue of the problem rather than the result of a pathological noise realisation. Potential remedies in applications with weak measurements include using quickest intermittent signal detection [10] or using non-Bayesian QCD such as the Lorden criterion [11]. Finally, we would expect similar phenomenon to arise in more complex Bayesian QCD or filter problems involving non-ergodic models with weak observations.

Acknowledgment.

The authors express many thanks to an anonymous reviewer who helped correct the proof of Lemma 2.