Synchronization and Control in Intrinsic and Designed Computation: An Information-Theoretic Analysis of Competing Models of Stochastic Computation

TL;DR

This paper applies information theory to analyze how observers synchronize with hidden states of stochastic processes, revealing the influence of internal structure and observer models on synchronization and process classification.

Contribution

It introduces a hierarchy of information quantifiers and a duality between synchronization and controllability, providing new insights into process representations.

Findings

Synchronization depends on process structure and observer model

Hierarchy of information quantifiers parallels block entropy hierarchy

New classification of process representations based on minimality and synchronizability

Abstract

We adapt tools from information theory to analyze how an observer comes to synchronize with the hidden states of a finitary, stationary stochastic process. We show that synchronization is determined by both the process's internal organization and by an observer's model of it. We analyze these components using the convergence of state-block and block-state entropies, comparing them to the previously known convergence properties of the Shannon block entropy. Along the way, we introduce a hierarchy of information quantifiers as derivatives and integrals of these entropies, which parallels a similar hierarchy introduced for block entropy. We also draw out the duality between synchronization properties and a process's controllability. The tools lead to a new classification of a process's alternative representations in terms of minimality, synchronizability, and unifilarity.