R\'enyi Divergence in General Hidden Markov Models

TL;DR

This paper investigates the properties and convergence of Rényi divergence in general hidden Markov models, providing theoretical insights, characterizations, and practical computation methods, including for neural networks and switching models.

Contribution

It introduces a novel analysis of Rényi divergence in general HMMs, including convergence results, characterizations, and a non-Monte Carlo computational method.

Findings

Convergence of Rényi divergence under regularity conditions

Characterization of Rényi divergence and its limit as α approaches 1

Development of a non-Monte Carlo method for Markov switching models

Abstract

In this paper, we examine the existence of the R\'enyi divergence between two time invariant general hidden Markov models with arbitrary positive initial distributions. By making use of a Markov chain representation of the probability distribution for the general hidden Markov model and eigenvalue for the associated Markovian operator, we obtain, under some regularity conditions, convergence of the R\'enyi divergence. By using this device, we also characterize the R\'enyi divergence, and obtain the Kullback-Leibler divergence as {\alpha} \rightarrow 1 of the R\'enyi divergence. Several examples, including the classical finite state hidden Markov models, Markov switching models, and recurrent neural networks, are given for illustration. Moreover, we develop a non-Monte Carlo method that computes the R\'enyi divergence of two-state Markov switching models via the underlying invariant probability measure, which is characterized by the Fredholm integral equation.