# Nearly Maximally Predictive Features and Their Dimensions

## TL;DR

This paper investigates the limits of predictive features in stochastic processes, deriving bounds based on fractal dimensions, and demonstrates how mixed-state features outperform finite-order Markov models in prediction accuracy.

## Contribution

It introduces bounds on nearly maximally predictive features' growth and shows the advantages of mixed-state features over traditional Markov models.

## Key findings

- Bounds on the growth rate of predictive features based on fractal dimensions
- Mixed-state features outperform finite-order Markov models
- Finite-order Markov models can fail as predictors

## Abstract

Scientific explanation often requires inferring maximally predictive features from a given data set. Unfortunately, the collection of minimal maximally predictive features for most stochastic processes is uncountably infinite. In such cases, one compromises and instead seeks nearly maximally predictive features. Here, we derive upper-bounds on the rates at which the number and the coding cost of nearly maximally predictive features scales with desired predictive power. The rates are determined by the fractal dimensions of a process' mixed-state distribution. These results, in turn, show how widely-used finite-order Markov models can fail as predictors and that mixed-state predictive features offer a substantial improvement.