A Novel Approach to the Partial Information Decomposition

TL;DR

This paper introduces a new, general framework for partial information decomposition (PID) that uses set-theoretic analogies and the Blackwell order, improving upon previous methods and applicable across various information theories.

Contribution

It proposes a novel, axiomatic framework for multivariate PID based on set theory and the Blackwell order, enhancing flexibility and operational interpretation.

Findings

Framework is algebraically and axiomatically motivated.

Overcomes drawbacks of previous PID proposals.

Demonstrated on numerous examples.

Abstract

We consider the "partial information decomposition" (PID) problem, which aims to decompose the information that a set of source random variables provide about a target random variable into separate redundant, synergistic, union, and unique components. In the first part of this paper, we propose a general framework for constructing a multivariate PID. Our framework is defined in terms of a formal analogy with intersection and union from set theory, along with an ordering relation which specifies when one information source is more informative than another. Our definitions are algebraically and axiomatically motivated, and can be generalized to domains beyond Shannon information theory (such as algorithmic information theory and quantum information theory). In the second part of this paper, we use our general framework to define a PID in terms of the well-known Blackwell order, which has a fundamental operational interpretation. We demonstrate our approach on numerous examples and show that it overcomes many drawbacks associated with previous proposals.