Redundancy in Systems which Entertain a Model of Themselves: Interaction Information and the Self-organization of Anticipation

TL;DR

This paper explores how systems that model themselves process information and generate anticipation, using measures of interaction information and redundancy, and empirically tests these concepts through textual analysis.

Contribution

It introduces a novel measure of interaction information based on maximum entropy approximations and applies it to analyze self-modeling systems in textual data.

Findings

Interaction information can be distinguished from redundancy in self-modeling systems.

Empirical analysis of textual data reveals patterns of intellectual organization.

Second-order observation influences information processing and uncertainty.

Abstract

Mutual information among three or more dimensions (mu-star = - Q) has been considered as interaction information. However, Krippendorff (2009a, 2009b) has shown that this measure cannot be interpreted as a unique property of the interactions and has proposed an alternative measure of interaction information based on iterative approximation of maximum entropies. Q can then be considered as a measure of the difference between interaction information and redundancy generated in a model entertained by an observer. I argue that this provides us with a measure of the imprint of a second-order observing system -- a model entertained by the system itself -- on the underlying information processing. The second-order system communicates meaning hyper-incursively; an observation instantiates this meaning-processing within the information processing. The net results may add to or reduce the prevailing uncertainty. The model is tested empirically for the case where textual organization can be expected to contain intellectual organization in terms of distributions of title words, author names, and cited references.