Probability Bracket Notation: Markov Sequence Projector of Visible and Hidden Markov Models in Dynamic Bayesian Networks

TL;DR

This paper introduces Probability Bracket Notation (PBN) and the Markov Sequence Projector (MSP) to unify and extend the evolution formulas of various Markov models within Dynamic Bayesian Networks, enhancing understanding and analysis in ML and AI.

Contribution

The paper develops PBN and MSP to unify and extend Markov model evolution formulas, including VMMs, HMMs, and factorial HMMs, within the framework of DBNs.

Findings

Verified results with Elvira software package.

Unified evolution formulas for multiple Markov models.

Extended HMM addressing feedback issues.

Abstract

With the symbolic framework of Probability Bracket Notation (PBN), the Markov Sequence Projector (MSP) is introduced to expand the evolution formula of Homogeneous Markov Chains (HMCs). The well-known weather example, a Visible Markov Model (VMM), illustrates that the full joint probability of a VMM corresponds to a specifically projected Markov state sequence in the expanded evolution formula. In a Hidden Markov Model (HMM), the probability basis (P-basis) of the hidden Markov state sequence and the P-basis of the observation sequence exist in the sequential event space. The full joint probability of an HMM is the product of the (unknown) projected hidden sequence of Markov states and their transformations into the observation P-bases. The Viterbi algorithm is applied to the famous Weather-Stone HMM example to determine the most likely weather-state sequence given the observed stone-state sequence. Our results are verified using the Elvira software package. Using the PBN, we unify the evolution formulas for Markov models like VMMs, HMMs, and factorial HMMs (with discrete time). We briefly investigated the extended HMM, addressing the feedback issue, and the continuous-time VMM and HMM (with discrete or continuous states). All these models are subclasses of Dynamic Bayesian Networks (DBNs) essential for Machine Learning (ML) and Artificial Intelligence (AI).