Sequential Tracking of a Hidden Markov Chain Using Point Process Observations

TL;DR

This paper develops a method for optimally tracking hidden Markov states using Poisson process observations, with applications in economics and staffing, providing explicit strategies and numerical solutions.

Contribution

It introduces a novel approach for finite horizon optimal switching in hidden Markov models with Poisson observations, including explicit strategy characterization and numerical methods.

Findings

Regularity of the value function established

Explicit optimal strategies derived

Numerical scheme demonstrated with examples

Abstract

We study finite horizon optimal switching problems for hidden Markov chain models under partially observable Poisson processes. The controller possesses a finite range of strategies and attempts to track the state of the unobserved state variable using Bayesian updates over the discrete observations. Such a model has applications in economic policy making, staffing under variable demand levels and generalized Poisson disorder problems. We show regularity of the value function and explicitly characterize an optimal strategy. We also provide an efficient numerical scheme and illustrate our results with several computational examples.