On the Realization of Hidden Markov Models and Tensor Decomposition

TL;DR

This paper investigates the minimum realization problem for hidden Markov models, revealing conditions under which tensor decomposition can effectively determine the minimal number of states, especially when the observation function is deterministic.

Contribution

It extends tensor decomposition methods to handle deterministic observation functions in HMMs, identifying conditions that allow for minimal realization beyond generic cases.

Findings

Tensor decomposition can reduce the number of rank-one tensors needed for certain HMMs.

The rank of the constructed tensor is bounded by the effective subspace dimension.

The approach applies when the reachable subspace is not the entire space or the null space is non-zero.

Abstract

The minimum realization problem of hidden Markov models (HMM's) is a fundamental question of stationary discrete-time processes with a finite alphabet. It was shown in the literature that tensor decomposition methods give the hidden Markov model with the minimum number of states generically. However, the tensor decomposition approach does not solve the minimum HMM realization problem when the observation is a deterministic function of the state, which is an important class of HMM's not captured by a generic argument. In this paper, we show that the reduction of the number of rank-one tensors necessary to decompose the third-order tensor constructed from the probabilities of the process is possible when the reachable subspace is not the whole space or the null space is not the zero space. In fact, the rank of the tensor is not greater than the dimension of the effective subspace or the rank of the generalized Hankel matrix.