Higher-order Common Information

TL;DR

This paper introduces a new measure of higher-order common information among multiple random variables, providing bounds and practical estimation methods, with applications to EEG data and neural response analysis.

Contribution

It proposes a novel higher-order common information measure $R_ extell$, with analytical bounds and a practical estimation approach, applied to real-world EEG data.

Findings

$R_3$ captures unique properties not present in $R_2$.

Linear relationship observed between $R_3$ and neural tracking of stimuli.

Bounds provided for Gaussian and arbitrary sources.

Abstract

We present a new notion $R_\ell$ of higher-order common information, which quantifies the information that $\ell\geq 2$ arbitrarily distributed random variables have in common. We provide analytical lower bounds on $R_3$ and $R_4$ for jointly Gaussian distributed sources and provide computable lower bounds for $R_\ell$ for any $\ell$ and any sources. We also provide a practical method to estimate the lower bounds on, e.g., real-world time-series data. As an example, we consider EEG data acquired in a setup with competing acoustic stimuli. We demonstrate that $R_3$ has descriptive properties that is not in $R_2$. Moreover, we observe a linear relationship between the amount of common information $R_3$ communicated from the acoustic stimuli and to the brain and the corresponding cortical activity in terms of neural tracking of the envelopes of the stimuli.