Estimating mutual information and multi--information in large networks

TL;DR

This paper presents practical methods for estimating mutual information and multi-information in large networks, demonstrating their effectiveness across diverse real-world systems like gene expression and financial markets.

Contribution

It introduces computationally feasible techniques for estimating complex information measures in large networks, extending neural code analysis methods to broader applications.

Findings

Reliable mutual information estimates are achievable with manageable computation.

Higher order multi-information can be estimated using the proposed methods.

Information measures correlate with intuitive system structures.

Abstract

We address the practical problems of estimating the information relations that characterize large networks. Building on methods developed for analysis of the neural code, we show that reliable estimates of mutual information can be obtained with manageable computational effort. The same methods allow estimation of higher order, multi--information terms. These ideas are illustrated by analyses of gene expression, financial markets, and consumer preferences. In each case, information theoretic measures correlate with independent, intuitive measures of the underlying structures in the system.