Ambiguity Rate of Hidden Markov Processes

TL;DR

This paper introduces the ambiguity rate for hidden Markov processes, linking it with the Shannon entropy rate to determine the growth of predictive features, and provides a method to calculate the statistical complexity dimension of complex processes.

Contribution

It defines the ambiguity rate and demonstrates its role in quantifying the growth of predictive features, enabling the calculation of the statistical complexity dimension for complex processes.

Findings

Ambiguity rate corrects the Lyapunov dimension of an IFS's attractor.

Provides a method to compute the statistical complexity dimension.

Links ambiguity rate with the growth of predictive features.

Abstract

The $\epsilon$-machine is a stochastic process' optimal model -- maximally predictive and minimal in size. It often happens that to optimally predict even simply-defined processes, probabilistic models -- including the $\epsilon$-machine -- must employ an uncountably-infinite set of features. To constructively work with these infinite sets we map the $\epsilon$-machine to a place-dependent iterated function system (IFS) -- a stochastic dynamical system. We then introduce the ambiguity rate that, in conjunction with a process' Shannon entropy rate, determines the rate at which this set of predictive features must grow to maintain maximal predictive power. We demonstrate, as an ancillary technical result which stands on its own, that the ambiguity rate is the (until now missing) correction to the Lyapunov dimension of an IFS's attractor. For a broad class of complex processes and for the first time, this then allows calculating their statistical complexity dimension -- the information dimension of the minimal set of predictive features.