Properties of Unique Information

TL;DR

This paper analyzes the properties of the unique information measure in information decomposition, providing conditions for solution non-uniqueness, reformulating optimization conditions, and improving computational methods especially for binary variables.

Contribution

It offers new insights into the uniqueness and support of solutions for the unique information measure, with reformulations that facilitate faster computation.

Findings

Identified conditions for non-uniqueness of solutions.

Reformulated first order conditions as rank constraints.

Provided a complete characterization for binary variables.

Abstract

We study the measure of unique information $UI(T:X\setminus Y)$ defined by Bertschinger et al. (2014) within the framework of information decompositions. We study uniqueness and support of the solutions to the optimization problem underlying the definition of $UI$. We identify sufficient conditions for non-uniqueness of solutions with full support in terms of conditional independence constraints and in terms of the cardinalities of $T$, $X$ and $Y$. Our results are based on a reformulation of the first order conditions on the objective function as rank constraints on a matrix of conditional probabilities. These results help to speed up the computation of $UI(T:X\setminus Y)$, most notably when $T$ is binary. In the case that all variables are binary, we obtain a complete picture of where the optimizing probability distributions lie.