# A Bayesian Theory of Change Detection in Statistically Periodic Random Processes

## TL;DR

This paper introduces a Bayesian framework for change detection in a new class of stochastic processes with periodic statistical behavior, using optimal control of Markov decision processes to derive effective stopping rules.

## Contribution

It develops a Bayesian change detection theory for i.p.i.d. processes and formulates it as an optimal control problem for periodic MDPs, providing algorithms and asymptotic optimality results.

## Key findings

- Optimal change detection policy is nonstationary and periodic.
- Single-threshold policy is asymptotically optimal as false alarm probability approaches zero.
- Numerical results show the asymptotic policy is not strictly optimal.

## Abstract

A new class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes is defined to capture periodically varying statistical behavior. A novel Bayesian theory is developed for detecting a change in the distribution of an i.p.i.d. process. It is shown that the Bayesian change point problem can be expressed as a problem of optimal control of a Markov decision process (MDP) with periodic transition and cost structures. Optimal control theory is developed for periodic MDPs for discounted and undiscounted total cost criteria. A fixed-point equation is obtained that is satisfied by the optimal cost function. It is shown that the optimal policy for the MDP is nonstationary but periodic in nature. A value iteration algorithm is obtained to compute the optimal cost function. The results from the MDP theory are then applied to detect changes in i.p.i.d. processes. It is shown that while the optimal change point algorithm is a stopping rule based on a periodic sequence of thresholds, a single-threshold policy is asymptotically optimal, as the probability of false alarm goes to zero. Numerical results are provided to demonstrate that the asymptotically optimal policy is not strictly optimal.