Partial information decomposition for mixed discrete and continuous random variables

TL;DR

This paper introduces a novel partial information decomposition framework for mixed discrete and continuous variables, enabling better analysis of complex interactions in systems like neuroscience and machine learning.

Contribution

It develops a new PID scheme for mixed variables, expressing mutual information as a KL divergence and estimating it with a data-efficient nearest-neighbor method.

Findings

Effective in simulated mixed-variable systems

Demonstrated in a physiological application

Applicable to sensory coding and feature selection

Abstract

The framework of Partial Information Decomposition (PID) unveils complex nonlinear interactions in network systems by dissecting the mutual information (MI) between a target variable and several source variables. While PID measures have been formulated mostly for discrete variables, with only recent extensions to continuous systems, the case of mixed variables where the target is discrete and the sources are continuous is not yet covered properly. Here, we introduce a PID scheme whereby the MI between a specific state of the discrete target and (subsets of) the continuous sources is expressed as a Kullback-Leibler divergence and is estimated through a data-efficient nearest-neighbor strategy. The effectiveness of this PID is demonstrated in simulated systems of mixed variables and showcased in a physiological application. Our approach is relevant to many scientific problems, including sensory coding in neuroscience and feature selection in machine learning.