Continuity and Additivity Properties of Information Decompositions

TL;DR

This paper investigates the continuity and additivity properties of various information decomposition methods, finding that only one method satisfies both, thus highlighting key theoretical constraints for such decompositions.

Contribution

It introduces the importance of continuity and additivity in information decompositions and identifies the unique method satisfying both properties.

Findings

Most decompositions satisfy continuity.

Only one decomposition is both continuous and additive.

Highlights theoretical constraints for information decompositions.

Abstract

Information decompositions quantify how the Shannon information about a given random variable is distributed among several other random variables. Various requirements have been proposed that such a decomposition should satisfy, leading to different candidate solutions. Curiously, however, only two of the original requirements that determined the Shannon information have been considered, namely monotonicity and normalization. Two other important properties, continuity and additivity, have not been considered. In this contribution, we focus on the mutual information of two finite variables $Y,Z$ about a third finite variable $S$ and check which of the decompositions satisfy these two properties. While most of them satisfy continuity, only one of them is both continuous and additive.