Divergence metrics in the study of Markov and hidden Markov processes

TL;DR

This paper reviews divergence metrics in Markov processes, focusing on their applications in stochastic stability, thermodynamics, and hidden Markov models, providing a unified formalism for researchers.

Contribution

It offers a comprehensive review and unified formalism for f-divergences in both Markov and hidden Markov processes, linking theoretical foundations to practical applications.

Findings

Analysis of f-divergences in continuous-time Markov processes

Application of divergence measures to stochastic thermodynamics

Insights into filter stability and Maxwell demon concepts

Abstract

This paper is divided into two parts. The first part reviews the formulae for f-divergences in the study of continuous-time Markov processes and explores their applications in areas such as stochastic stability, the second law of thermodynamics, and its non-equilibrium extensions. This sets the foundation for the second part, which focuses on f-divergence in the study of hidden Markov processes. In this context, we present analyses of filter stability and stochastic thermodynamics, with the latter being used to illustrate the concept of a Maxwell demon in an over-damped Langevin model with white noise observations. The paper's expository style and unified formalism for both Markov and hidden Markov processes aim to serve as a valuable resource for researchers working across related fields.