Computing the Unique Information

TL;DR

This paper introduces an efficient algorithm to quantify the unique information that predictor variables have about a response, decomposing total information into unique, redundant, and synergistic parts.

Contribution

It proposes a novel iterative divergence minimization algorithm with convergence guarantees for computing the unique information component.

Findings

Algorithm outperforms existing methods in efficiency

Provides accurate decomposition of information components

Demonstrates applicability to real-world data sets

Abstract

Given a pair of predictor variables and a response variable, how much information do the predictors have about the response, and how is this information distributed between unique, redundant, and synergistic components? Recent work has proposed to quantify the unique component of the decomposition as the minimum value of the conditional mutual information over a constrained set of information channels. We present an efficient iterative divergence minimization algorithm to solve this optimization problem with convergence guarantees and evaluate its performance against other techniques.