Exact Computation of Kullback-Leibler Distance for Hidden Markov Trees and Models

TL;DR

This paper introduces recursive formulas for exact computation of the Kullback-Leibler distance between Hidden Markov Trees, including models conditioned on observations, validated through numerical examples.

Contribution

It provides the first recursive formulas for exact KLD calculation in HMTs and HMMs, including conditioned cases, advancing precise divergence measurement methods.

Findings

Recursive formulas accurately compute KLD in HMTs and HMMs.

Closed-form KLD expression for homogeneous HMTs without evidence.

Validation confirms the formulas match Monte Carlo estimations.

Abstract

We suggest new recursive formulas to compute the exact value of the Kullback-Leibler distance (KLD) between two general Hidden Markov Trees (HMTs). For homogeneous HMTs with regular topology, such as homogeneous Hidden Markov Models (HMMs), we obtain a closed-form expression for the KLD when no evidence is given. We generalize our recursive formulas to the case of HMMs conditioned on the observable variables. Our proposed formulas are validated through several numerical examples in which we compare the exact KLD value with Monte Carlo estimations.