Null models for comparing information decomposition across complex systems

TL;DR

This paper introduces NuMIT, a null model-based normalization method for information-theoretic measures, enabling meaningful cross-system comparisons and significance testing in complex datasets like neuroimaging.

Contribution

NuMIT offers a novel, robust normalization approach based on null models that improves upon existing entropy-based methods for comparing information decomposition across systems.

Findings

NuMIT outperforms standard normalization techniques.

It provides reliable significance testing for PID analyses.

Demonstrated on synthetic and neuroimaging data.

Abstract

A key feature of information theory is its universality, as it can be applied to study a broad variety of complex systems. However, many information-theoretic measures can vary significantly even across systems with similar properties, making normalisation techniques essential for allowing meaningful comparisons across datasets. Inspired by the framework of Partial Information Decomposition (PID), here we introduce Null Models for Information Theory (NuMIT), a null model-based non-linear normalisation procedure which improves upon standard entropy-based normalisation approaches and overcomes their limitations. We provide practical implementations of the technique for systems with different statistics, and showcase the method on synthetic models and on human neuroimaging data. Our results demonstrate that NuMIT provides a robust and reliable tool to characterise complex systems of interest, allowing cross-dataset comparisons and providing a meaningful significance test for PID analyses.