Thermodynamic Depth of Causal States: When Paddling around in Occam's Pool Shallowness Is a Virtue

TL;DR

This paper critiques thermodynamic depth as a measure of structural complexity, showing it only captures randomness, and proposes using causal states via epsilon-machines to define a more meaningful, minimal-depth complexity measure.

Contribution

It introduces a new definition of depth based on causal states, ensuring the measure reflects true structure rather than randomness.

Findings

Thermodynamic depth only measures randomness, not structure.

Causal states via epsilon-machines provide an optimal, minimal-depth complexity measure.

Depth defined through epsilon-machines is consistent with accurate prediction.

Abstract

Thermodynamic depth is an appealing but flawed structural complexity measure. It depends on a set of macroscopic states for a system, but neither its original introduction by Lloyd and Pagels nor any follow-up work has considered how to select these states. Depth, therefore, is at root arbitrary. Computational mechanics, an alternative approach to structural complexity, provides a definition for a system's minimal, necessary causal states and a procedure for finding them. We show that the rate of increase in thermodynamic depth, or {\it dive}, is the system's reverse-time Shannon entropy rate, and so depth only measures degrees of macroscopic randomness, not structure. To fix this we redefine the depth in terms of the causal state representation---$\epsilon$-machines---and show that this representation gives the minimum dive consistent with accurate prediction. Thus, $\epsilon$-machines are optimally shallow.