Information Flows? A Critique of Transfer Entropies

TL;DR

This paper critiques the use of transfer entropy as a measure of information flow in complex systems, highlighting its limitations and proposing the need for alternative measures to better capture system dynamics.

Contribution

It identifies fundamental issues with transfer entropy and causation entropy in quantifying information flow, and discusses the implications for network-based analyses of complex systems.

Findings

Transfer entropy can overestimate or underestimate information flow.

Current measures fail to account for polyadic dependencies.

Network models may overlook complex organizational structures.

Abstract

A central task in analyzing complex dynamics is to determine the loci of information storage and the communication topology of information flows within a system. Over the last decade and a half, diagnostics for the latter have come to be dominated by the transfer entropy. Via straightforward examples, we show that it and a derivative quantity, the causation entropy, do not, in fact, quantify the flow of information. At one and the same time they can overestimate flow or underestimate influence. We isolate why this is the case and propose several avenues to alternate measures for information flow. We also address an auxiliary consequence: The proliferation of networks as a now-common theoretical model for large-scale systems, in concert with the use of transfer-like entropies, has shoehorned dyadic relationships into our structural interpretation of the organization and behavior of complex systems. This interpretation thus fails to include the effects of polyadic dependencies. The net result is that much of the sophisticated organization of complex systems may go undetected.