Stripping syntax from complexity: An information-theoretical perspective on complex systems

TL;DR

This paper advocates applying information theory to complexity science to decouple domain-specific details from universal phenomena, potentially leading to a new understanding and universality of complex systems.

Contribution

It proposes a novel perspective of using information theory to analyze complex systems independently of their specific models, highlighting initial successes and future potential.

Findings

Information theory can describe complex systems through storage, transfer, and modification of information.

Decoupling model details reveals universal properties across different complex systems.

Early applications show promise for a new paradigm in complexity science.

Abstract

Claude Shannons information theory (1949) has had a revolutionary impact on communication science. A crucial property of his framework is that it decouples the meaning of a message from the mechanistic details from the actual communication process itself, which opened the way to solve long-standing communication problems. Here we argue that a similar impact could be expected by applying information theory in the context of complexity science to answer long-standing, cross-domain questions about the nature of complex systems. This happens by decoupling the domain-specific model details (e.g., neuronal networks, ecosystems, flocks of birds) from the cross-domain phenomena that characterize complex systems (e.g., criticality, robustness, tipping points). This goes beyond using information theory as a non-linear correlation measure, namely it allows describing a complex system entirely in terms of the storage, transfer, and modification of informational bits. After all, a phenomenon that does not depend on model details should best be studied in a framework that strips away all such details. We highlight the first successes of information-theoretic descriptions in the recent complexity literature, and emphasize that this type of research is still in its infancy. Finally we sketch how such an information-theoretic description may even lead to a new type of universality among complex systems, with a potentially tremendous impact. The goal of this perspective article is to motivate a paradigm shift in the young field of complexity science using a lesson learnt in communication science.