Higher-Order Interactions and Their Duals Reveal Synergy and Logical Dependence beyond Shannon-Information

TL;DR

This paper introduces a new interpretation of higher-order interactions among binary variables using M"obius inversion on surprisal, revealing their ability to distinguish systems beyond Shannon information, including logic gates and causal dynamics.

Contribution

It provides a novel information-theoretic framework for higher-order interactions and their duals, extending mutual information concepts beyond Shannon entropy.

Findings

Dual mutual information relates to differential mutual information.

Dual interactions can distinguish all six 2-input logic gates.

Higher-order interactions identify systems indistinguishable by Shannon information.

Abstract

Information-theoretic quantities reveal dependencies among variables in the structure of joint, marginal, and conditional entropies, but leave some fundamentally different systems indistinguishable. Furthermore, there is no consensus on how to construct and interpret a higher-order generalisation of mutual information (MI). In this manuscript, we show that a recently proposed model-free definition of higher-order interactions amongst binary variables (MFIs), like mutual information, is a M\"obius inversion on a Boolean algebra, but of surprisal instead of entropy. This gives an information-theoretic interpretation to the MFIs, and by extension to Ising interactions. We study the dual objects to MI and MFIs on the order-reversed lattice, and find that dual MI is related to the previously studied differential mutual information, while dual interactions (outeractions) are interactions with respect to a different background state. Unlike mutual information, in- and outeractions uniquely identify all six 2-input logic gates, the dy- and triadic distributions, and different causal dynamics that are identical in terms of their Shannon-information content.