Signatures of Infinity: Nonergodicity and Resource Scaling in Prediction, Complexity, and Learning

TL;DR

This paper analyzes the complexity of infinite-memory processes derived from stationary, ergodic finite-memory processes, offering new insights into predictability and learning in complex systems with hierarchical structures.

Contribution

It provides an alternative framework contrasting computation-theoretic and statistical approaches to understanding process complexity and resource scaling.

Findings

Highlights differences in informational and correlational divergences.

Elucidates resource divergences in ergodic and nonergodic processes.

Explores hierarchical structures in complex processes.

Abstract

We introduce a simple analysis of the structural complexity of infinite-memory processes built from random samples of stationary, ergodic finite-memory component processes. Such processes are familiar from the well known multi-arm Bandit problem. We contrast our analysis with computation-theoretic and statistical inference approaches to understanding their complexity. The result is an alternative view of the relationship between predictability, complexity, and learning that highlights the distinct ways in which informational and correlational divergences arise in complex ergodic and nonergodic processes. We draw out consequences for the resource divergences that delineate the structural hierarchy of ergodic processes and for processes that are themselves hierarchical.