Two-step Nonnegative Matrix Factorization Algorithm for the Approximate Realization of Hidden Markov Models

TL;DR

This paper introduces a two-step nonnegative matrix factorization algorithm to construct Hidden Markov Models that approximate a given distribution, focusing on model size and divergence minimization.

Contribution

The paper presents a novel two-step NMF-based algorithm for HMM realization, improving model approximation and order reduction techniques.

Findings

Effective in HMM order reduction

Produces models with minimal divergence from target distribution

Demonstrated through numerical simulations

Abstract

We propose a two-step algorithm for the construction of a Hidden Markov Model (HMM) of assigned size, i.e. cardinality of the state space of the underlying Markov chain, whose $n$-dimensional distribution is closest in divergence to a given distribution. The algorithm is based on the factorization of a pseudo Hankel matrix, defined in terms of the given distribution, into the product of a tall and a wide nonnegative matrix. The implementation is based on the nonnegative matrix factorization (NMF) algorithm. To evaluate the performance of our algorithm we produced some numerical simulations in the context of HMM order reduction.