Time's Barbed Arrow: Irreversibility, Crypticity, and Stored Information

TL;DR

This paper explores the relationship between excess entropy and statistical complexity in dynamical systems, revealing conditions under which they differ and introducing a classification scheme based on causal irreversibility and crypticity.

Contribution

It develops closed-form expressions for excess entropy using epsilon-machines and introduces a classification scheme for dynamical systems based on irreversibility and crypticity.

Findings

Excess entropy and statistical complexity can differ in dynamical systems.

Causal irreversibility and crypticity determine the relationship between these quantities.

The paper provides formulas linking excess entropy to causal predictors and retrodictors.

Abstract

We show why the amount of information communicated between the past and future--the excess entropy--is not in general the amount of information stored in the present--the statistical complexity. This is a puzzle, and a long-standing one, since the latter is what is required for optimal prediction, but the former describes observed behavior. We layout a classification scheme for dynamical systems and stochastic processes that determines when these two quantities are the same or different. We do this by developing closed-form expressions for the excess entropy in terms of optimal causal predictors and retrodictors--the epsilon-machines of computational mechanics. A process's causal irreversibility and crypticity are key determining properties.