Properties of Persistent Mutual Information and Emergence

TL;DR

This paper rigorously defines the persistent mutual information (PMI), explores its relation to other complexity measures, and evaluates its effectiveness as a formal measure of emergence in stochastic processes.

Contribution

It mathematically formalizes PMI, proves its relation as an upper bound to excess entropy, and assesses its suitability for measuring emergence.

Findings

Excess entropy bounds PMI from above.

Explicit calculations of PMI for example processes.

Discussion on PMI's effectiveness as an emergence measure.

Abstract

The persistent mutual information (PMI) is a complexity measure for stochastic processes. It is related to well-known complexity measures like excess entropy or statistical complexity. Essentially it is a variation of the excess entropy so that it can be interpreted as a specific measure of system internal memory. The PMI was first introduced in 2010 by Ball, Diakonova and MacKay as a measure for (strong) emergence. In this paper we define the PMI mathematically and investigate the relation to excess entropy and statistical complexity. In particular we prove that the excess entropy is an upper bound of the PMI. Furthermore we show some properties of the PMI and calculate it explicitly for some example processes. We also discuss to what extend it is a measure for emergence and compare it with alternative approaches used to formalize emergence.