Secret Sharing and Shared Information

TL;DR

This paper explores the use of secret sharing schemes as models for information decomposition, revealing limitations in current frameworks and suggesting the need for lattice extensions to better capture redundancy.

Contribution

It identifies the incompleteness of the partial information lattice and proposes that extending it can improve the understanding of information redundancy.

Findings

Partial information lattice may be incomplete.

Negative partial information terms arise in certain decompositions.

Extensions to the lattice could better represent higher-order redundancy.

Abstract

Secret sharing is a cryptographic discipline in which the goal is to distribute information about a secret over a set of participants in such a way that only specific authorized combinations of participants together can reconstruct the secret. Thus, secret sharing schemes are systems of variables in which it is very clearly specified which subsets have information about the secret. As such, they provide perfect model systems for information decompositions. However, following this intuition too far leads to an information decomposition with negative partial information terms, which are difficult to interpret. One possible explanation is that the partial information lattice proposed by Williams and Beer is incomplete and has to be extended to incorporate terms corresponding to higher order redundancy. These results put bounds on information decompositions that follow the partial information framework, and they hint at where the partial information lattice needs to be improved.

Equations

$$\{A,B\},\{A,C\},\{B,C\},\{A,B,C\}.$$

$$\underline{\mathcal{A}}:=\big{\{}A\in\mathcal{A}:\text{if }B\subseteq A\text{ and }B\neq A,\text{ then }B\notin\mathcal{A}\big{\}}.$$

$$(A_1,\dots ,A_k)⪯(B_1,\dots ,B_l):⟺\text{for any }{B}_i\text{ there exists }{A}_j\text{ with }{A}_j\subseteq B_i.$$

$$I_{\cap }(S;A_1,\dots ,A_k)=\sum _{(B_1,\dots ,B_l)⪯(A_1,\dots ,A_k)}{I}_{\partial }(S;B_1,\dots ,B_l).$$

$$I_{\cap }(S;A_1,\dots ,A_k)=\{\begin{array}{ll}H(S),& \text{ if }{A}_1,\dots ,A_k\text{ are all authorized},\\ 0,& \text{ otherwise,}\end{array}$$

$$I_{\cap }(S;A_1,\dots ,A_k)\le I(S;A_i)=0$$

$$I_{\cap }(S;A_1,\dots ,A_k)\ge I_{\cap }(S;A_1,\dots ,A_k,S)=I_{\cap }(S;S)=H(S).\text{qed}$$

$$I_{\partial }(S;A_1,\dots ,A_k)=\{\begin{array}{ll}H(S),& \text{ if }\underline{\mathcal{A}}=\{A_1,\dots ,A_k\},\\ 0,& \text{ otherwise,}\end{array}$$

$$J_{\cap }(S;A_1,\dots ,A_k)=\sum _{(B_1,\dots ,B_l)⪯(A_1,\dots ,A_k)}{J}_{\partial }(S;B_1,\dots ,B_l).$$

$$J_{\cap }(S;A_1,\dots ,A_k)=\{\begin{array}{ll}H(S),& \text{ if }{A}_1,\dots ,A_k\text{ are all authorized},\\ 0,& \text{ otherwise,}\end{array}$$

$$H(S_i|S_1,\dots ,S_{i-1},S_{i+1},\dots ,S_l,X_A)=H(S_i)\text{ if }A\notin {\mathcal{A}}_i.$$

$$H(S_i|S_1,\dots ,S_{i-1},S_{i+1},\dots ,S_l)\ge H(S_i|S_1,\dots ,S_{i-1},S_{i+1},\dots ,S_l,X_A)=H(S_i).$$

$$I_{\cap}\big{(}(S_{1},\dots,S_{l});A_{1},\dots,A_{k}\big{)}=H\big{(}\{S_{i}:A_{1},\dots,A_{k}\in\mathcal{A}_{i}\}\big{)}$$

$$I_{\text{MMI}}(S;A_1,\dots ,A_k):=\underset{i=1,\dots ,k}{\min }I(S;A_i).$$

$$I_{\partial}\big{(}(S_{1},\dots,S_{l});A_{1},\dots,A_{k}\big{)}=\begin{cases}H(S_{i}),&\text{ if }\underline{\mathcal{A}_{i}}=\{A_{1},\dots,A_{k}\}\\ &\qquad\qquad\text{ for some }i\in\{1,\dots,l\},\\ 0,&\text{ otherwise,}\end{cases}$$

$$S=(S_{\mathcal{A}}{)}_{\mathcal{A}},X_1=(X_{1,\mathcal{A}}{)}_{\mathcal{A}},\dots ,X_n=(X_{n,\mathcal{A}}{)}_{\mathcal{A}},$$

$$I_{\cap}\big{(}(X_{1},X_{2});X_{1},X_{2}\big{)}=0.$$

$$I_{\cap}\big{(}S;(X_{1}X_{2}),(X_{1}X_{3}),(X_{2}X_{3})\big{)}\leq$2\text{\,}\mathrm{b}\mathrm{i}\mathrm{t}$,$$

$$I_{\partial}\big{(}S;(X_{1}X_{2}),(X_{1}X_{3}),(X_{2}X_{3})\big{)}=I_{\cap}\big{(}S;(X_{1}X_{2}),(X_{1}X_{3}),(X_{2}X_{3})\big{)}\\ -I_{\cap}\big{(}S;X_{1},(X_{2}X_{3})\big{)}-I_{\cap}\big{(}S;X_{2},(X_{1}X_{3})\big{)}-I_{\cap}\big{(}S;X_{3},(X_{1}X_{2})\big{)}\pm 0\\ \leq$2\text{\,}\mathrm{b}\mathrm{i}\mathrm{t}$-$3\text{\,}\mathrm{b}\mathrm{i}\mathrm{t}$=$-1\text{\,}\mathrm{b}\mathrm{i}\mathrm{t}$,$$

$$P(X_3=x_3|X_1=x_1,X_2=x_2)=\{\begin{array}{ll}1-ϵ,& \text{ if }{x}_3=x_1\oplus x_2,\\ ϵ,& \text{ otherwise,}\end{array}$$

$$I\big{(}(S_{1},\dots,S_{l});X_{A}\big{)}=H\big{(}\{S_{i}:A\in\mathcal{A}_{i}\}\big{)}.$$

$$\begin{gathered} H(S_1,\dots ,S_l|X_A)=H(S_1,\dots ,S_m|X_A)+H(S_{m+1},\dots ,S_l|S_1,\dots ,S_m,X_A) \\ =H(S_{m+1},\dots ,S_l|S_1,\dots ,S_m,X_A)\le H(S_{m+1},\dots ,S_l)\le \sum _{i=m+1}^{l}H(S_i). \end{gathered}$$

$$\begin{gathered} H(S_{m+1},\dots ,S_l|S_1,\dots ,S_m,X_A)=\sum _{i=m+1}^{l}H(S_i|S_1,\dots ,S_{i-1},X_A) \\ \ge \sum _{i=m+1}^{l}H(S_i|S_1,\dots ,S_{i-1},S_{i+1},\dots ,S_l,X_A)=\sum _{i=m+1}^{l}H(S_i). \end{gathered}$$

$$I\big{(}(S_{1},\dots,S_{l});X_{A}\big{)}=H(S_{1},\dots,S_{l})-H(S_{1},\dots,S_{l}|X_{A})=H(S_{1},\dots,S_{m}).\qed$$

$$\widetilde{SI}\big{(}(S_{1},\dots,S_{l});X_{A_{1}},X_{A_{2}}\big{)}=H\big{(}\{S_{i}:A\in\mathcal{A}_{1}\cap\mathcal{A}_{2}\}\big{)}.$$

$$Q^{\ast }(s_1,\dots ,s_l,x_1,x_2)=P(s_1,\dots ,s_l)P(x_{A_1}=x_1|s_1,\dots ,s_l)P(x_{A_2}=x_2|s_1,\dots ,s_l).$$

$$H_{Q^{\ast }}(S_1,\dots ,S_l|X_{A_1},X_{A_2})=H_{Q^{\ast }}(S_{m+1},\dots ,S_l)=H_P(S_{m+1},\dots ,S_l).$$

$$H_Q(S_1,\dots ,S_l|X_{A_1},X_{A_2})\le H_Q(S_{m+1},\dots ,S_l)=H_P(S_{m+1},\dots ,S_l).$$

$$\displaystyle I_{Q^{*}}\big{(}(S_{1},\dots,S_{l});X_{A_{1}}\big{|}X_{A_{2}}\big{)}$$

$$\displaystyle I_{Q^{*}}\big{(}(S_{1},\dots,S_{l});X_{A_{1}}\big{)}$$

$$\widetilde{SI}\big{(}(S_{1},\dots,S_{l});X_{A_{1}},X_{A_{2}}\big{)}\\ =I_{Q^{*}}\big{(}(S_{1},\dots,S_{l});X_{A_{1}}\big{)}-I_{Q^{*}}\big{(}(S_{1},\dots,S_{l});X_{A_{1}}\big{|}X_{A_{2}}\big{)}\\ =\sum_{i=s}^{r}H(S_{i})=H(S_{s},\dots,S_{r}).\qed$$

Full text

1. Introduction

Williams and Beer (2010) have proposed a general framework to decompose the multivariate mutual information $I(S;X_{1},\dots,X_{n})$ between a target random variable $S$ and predictor random variables $X_{1},\dots,X_{n}$ into different terms (called partial information terms) according to different ways in which combinations of the variables $X_{1},\dots,X_{n}$ provide unique, shared or synergistic information about $S$. Williams and Beer argue that such a decomposition can be based on a measure of shared information. The underlying idea is that any information can be classified according to “who knows what.” But is this true?

A situation where the question “who knows what” is easy to answer very precisely is secret sharing, a part of cryptography in which the goal is to distribute information (the secret) over a set of participants such that the secret can only be reconstructed if certain authorized combinations of the participants join their information (see Beimel (2011) for a survey). The set of authorized combinations is called the access structure. Formally, the secret is modelled as a random variable $S$, and a secret sharing scheme assigns a random variable $X_{i}$ to each participant $i$ in such a way that, if $\{i_{1},\dots,i_{k}\}$ is an authorized set of participants, then $S$ is a function of $X_{i_{1}},\dots,X_{i_{k}}$; that is, $H(S|X_{i_{1}},\dots,X_{i_{k}})=0$; and, conversely, if $\{i_{1},\dots,i_{k}\}$ is not authorized, then $H(S|X_{i_{1}},\dots,X_{i_{k}})>0$. It is assumed that the participants know the scheme, and so any authorized combination of participants can reconstruct the secret if they join their information. A secret sharing scheme is perfect if non-authorized sets of participants know nothing about the secret; i.e., $H(S|X_{i_{1}},\dots,X_{i_{k}})=H(S)$. Thus, in a perfect secret sharing scheme, it is very clearly specified “who knows what.” In this sense, perfect secret sharing schemes provide model systems for which it should be easy to write down an information decomposition.

One connection between secret sharing and information decompositions is that the set of access structures of secret sharing schemes with $n$ participants is in one-to-one correspondence with the partial information terms of Williams and Beer. This correspondence makes it possible to give another interpretation to all partial information terms: Namely, the partial information term is a measure of how similar a given system of random variables is to a secret sharing scheme with a given access structure.

This correspondence also allows to introduce the secret sharing property that makes precise the above intuition: An information decomposition satisfies this property if and only if any perfect secret sharing scheme has just a single partial information term (which corresponds to its access structure). Lemma 2 states the secret sharing property is implied by the Williams and Beer axioms, which shows that the secret sharing property plays well together with the ideas of Williams and Beer. Proposition 1 shows that in an information decomposition that satisfies a natural generalization of this property, it is possible to prescribe arbitrary nonnegative values to all partial information terms.

These results suggest that perfect secret sharing schemes fit well together with the ideas of Williams and Beer. However, following this intuition too far leads to inconsistencies. As Theorem 4 shows, extending the secret sharing property to pairs of perfect secret sharing schemes leads to negative partial information terms. While other authors have started to build an intuition for negative partial terms and argue that they may be unavoidable in information decompositions, the concluding section collects arguments against such claims and proposes as another possible solutions that the Williams and Beer framework is incomplete and is missing nodes that represent higher order redundancy.

Cryptography, where the goal is not only to transport information (as in coding theory) but also to keep it concealed from unauthorized parties, has initiated many interesting developments in information theory, for example, by introducing new information measures and re-interpreting older ones; see, for example, Maurer and Wolf (1997); Csiszar and Narayan (2004). This manuscript focuses on another contribution of cryptography: probabilistic systems with well-defined distribution of information.

The remainder of this article is organized as follows: Section 2 summarizes definitions and results about secret sharing schemes. Section 3 introduces different secret sharing properties that fix the values that a measure of shared information assigns to perfect secret sharing schemes and combinations thereof. The main result of Section 4 is that the pairwise secret sharing property leads to negative partial information terms. Section 5 discusses the implications of this incompatibility result.

2. Perfect secret sharing schemes

We consider $n$ participants among whom we want to distribute information about a secret in such a way that we can control which subsets of participants together can decrypt the secret.

Definition 1**.**

An access structure $\mathcal{A}$ is a family of subsets of $\{1,\dots,n\}$, closed to taking supersets. Elements of $\mathcal{A}$ are called authorized sets.

A secret sharing scheme with access structure $\mathcal{A}$ is a family of random variables $S$, $X_{1},\dots,X_{n}$ such that:

• •

$H(X_{A},S)=H(X_{A})$, whenever $A\in\mathcal{A}$.

Here, $X_{A}=(X_{i})_{i\in A}$ for all subsets $A\subseteq\{1,\dots,n\}$. A secret sharing scheme is perfect if

• •

$H(X_{A},S)=H(X_{A})+H(S)$, whenever $A\notin\mathcal{A}$.

The condition for perfection is equivalent to $H(S|X_{A})=H(S)$. See Beimel (2011) for a survey on secret sharing.

Theorem 1**.**

For any access structure $\mathcal{A}$ and any $h>0$, there exists a perfect secret sharing scheme with access structure $\mathcal{A}$ for which the entropy of the secret $S$ equals $H(S)=h$.

Proof.

Perfect secret sharing schemes for arbitrary access structures were first constructed by Ito et al. (1987). In this construction, the entropy of the secret equals $1\text{\,}\mathrm{b}\mathrm{i}\mathrm{t}$. Combining $n$ copies of such a secret sharing scheme gives a secret sharing scheme with a secret of $n$ bit. As explained in (Beimel, 2011, Claim 1), the distribution of the secret may be perturbed arbitrarily (as long as the support of the distribution remains the same). In this way it is possible to prescribe the entropy of the secret in a perfect secret sharing scheme. ∎

Example 1*.*

Let $Y_{1},Y_{2},Y_{3},S$ be independent uniform binary random variables, and let $A=(Y_{1},Y_{2}\oplus S)$, $B=(Y_{2},Y_{3}\oplus S)$, $C=(Y_{3},Y_{1}\oplus S)$, where $\oplus$ denotes addition modulo 2 (or the $\operatorname{XOR}$ operation). Then $(S,A,B,C)$ is a perfect secret sharing scheme with access structure

[TABLE]

It may be of little surprise that integer addition modulo $k$ is an important building block in many secret sharing schemes.

While existence of perfect secret sharing schemes is solved, there remains the problem of finding efficient secret sharing schemes in the sense that the variables $X_{1},\dots,X_{n}$ should be as small as possible (in the sense of a small entropy), given a fixed entropy of the secret. For instance, in Example 1, $H(X_{i})/H(S)=2$ for all $i$. See Beimel (2011) for a survey.

Since an access structure $\mathcal{A}$ is closed to taking supersets, it is uniquely determined by its inclusion-minimal elements

[TABLE]

For instance, in Example 1, the first three elements belong to $\underline{\mathcal{A}}$. The set $\underline{\mathcal{A}}$ has the property that no element of $\underline{\mathcal{A}}$ is a subset of another element of $\underline{\mathcal{A}}$. Such a collection of sets is called an antichain. Conversely, any such antichain equals the set of inclusion-minimal elements of a unique access structure.

The antichains have a natural lattice structure, which was used by Williams and Beer to order the different values of shared information and organize them into what they call the partial information lattice. The same lattice also has a description in terms of secret sharing.

Definition 2**.**

Let $(A_{1},\dots,A_{k})$ and $(B_{1},\dots,B_{l})$ be antichains. Then

[TABLE]

The partial information lattice for the case $n=3$ is depicted in Figure 1.

Lemma 1**.**

Let $\mathcal{A}$ be an access structure on $\{1,\dots,n\}$, and let $(B_{1},\dots,B_{l})$ be an antichain. Then $B_{1},\dots,B_{l}$ are all authorized for $\mathcal{A}$ if and only if $\underline{\mathcal{A}}\preceq(B_{1},\dots,B_{l})$.

Proof.

The statement directly follows from the definitions. ∎

3. Information decompositions of secret sharing schemes

Williams and Beer (2010) proposed to decompose the total mutual information $I(S;X_{1},\dots,X_{n})$ between a target random variable $S$ and predictor random variables $X_{1},\dots,X_{n}$ according to different ways in which combinations of the variables $X_{1},\dots,X_{n}$ provide unique, shared or synergistic information about $S$. One of their main ideas is to base such a decomposition on a single measure of shared information $I_{\cap}$, which is a function $I(S;Y_{1},\dots,Y_{k})$ that takes as arguments a list of random variables, of which the first, $S$, takes a special role. To arrive at a decomposition of $I(S;X_{1},\dots,X_{n})$, the variables $Y_{1},\dots,Y_{k}$ are taken to be combinations $X_{A}=(X_{i})_{i\in A}$ of $X_{1},\dots,X_{n}$, corresponding to subsets $A$ of $\{1,\dots,n\}$. For simplicity, $I_{\cap}(S;X_{A_{1}},\dots,X_{A_{k}})$ is denoted by $I_{\cap}(S;A_{1},\dots,A_{k})$ for all $A_{1},\dots,A_{k}\subseteq\{1,\dots,n\}$.

Williams and Beer proposed a list of axioms that such a measure $I_{\cap}$ should satisfy. It follows from these axioms that it suffices to consider the function $I_{\cap}(S;A_{1},\dots,A_{k})$ in the case that $(A_{1},\dots,A_{k})$ is an antichain. Moreover, $I_{\cap}(S;\;\cdot)$ is a monotone function on the partial information lattice (Definition 2). Thus it is natural to write each value $I_{\cap}(S;A_{1},\dots,A_{k})$ on the lattice as a sum of local terms $I_{\partial}$ corresponding to the antichains that lie below $(A_{1},\dots,A_{k})$ in the lattice:

[TABLE]

The terms $I_{\partial}$ are called partial information terms. This representation always exists, and the partial information terms are uniquely defined (using a Möbius inversion). However, it is not guaranteed that $I_{\partial}$ is always nonnegative. If $I_{\partial}$ is nonnegative, then $I_{\cap}$ is called locally positive.

Williams and Beer also defined a function denoted by $I_{\min}$ that satisfies their axioms and that is locally positive. While the framework is intriguing and has attracted a lot of further research (as this special issue illustrates), the function $I_{\min}$ has been critiziced as not measuring the right thing. The difficulty of finding a reasonable measure of shared information that is locally positive (Bertschinger et al., 2013; Rauh et al., 2014) has led some to argue that maybe local positivity is not a necessary requirement for an information decomposition. This issue is discussed further in Section 5.

The goal of this section is to present additional natural properties for a measure of shared information that relate secret sharing with the intuition behind information decompositions. In a perfect secret sharing scheme, any combination of participants knows either nothing or everything about $S$. This motivates the following definition:

Definition 3**.**

A measure of shared information $I_{\cap}$ has the secret sharing property if and only if for any access structure $\mathcal{A}$ and any perfect secret sharing scheme $(X_{1},\dots,X_{n},S)$ with access structure $\mathcal{A}$, the following holds:

[TABLE]

for any $A_{1},\dots,A_{k}\subseteq\{X_{1},\dots,X_{n}\}$.

Lemma 2**.**

The secret sharing property is implied by the Williams and Beer axioms.

Proof.

The Williams and Beer axioms imply that

[TABLE]

whenever $A_{i}$ is not authorized. On the other hand, when $A_{1},\dots,A_{k}$ are all authorized, then the monotonicity axiom implies

[TABLE]

Perfect secret sharing schemes lead to information decompositions with a single nonzero partial information term:

Lemma 3**.**

If $I_{\cap}$ has the secret sharing property and if $(X_{1},\dots,X_{n},S)$ is a perfect secret sharing scheme with access structure $\mathcal{A}$, then

[TABLE]

for any $A_{1},\dots,A_{k}\subseteq\{X_{1},\dots,X_{n}\}$.

Proof.

Suppose that $\underline{\mathcal{A}}=\{A^{\prime}_{1},\dots,A^{\prime}_{k^{\prime}}\}$, and let $J_{\partial}(S;A_{1},\dots,A_{k})$ be the right hand side of (1). We need to show that $I_{\partial}=J_{\partial}$. Since the Möbius inversion is unique, it suffices to show that $J_{\cap}=I_{\cap}$, where

[TABLE]

By Lemma 1,

[TABLE]

for any $A_{1},\dots,A_{k}\subseteq\{X_{1},\dots,X_{n}\}$, from which the claim follows. ∎

What happens when we have several secret sharing schemes involving the same participants? In order to have a clear intuition, assume that the secret sharing schemes satisfy the following definition:

Definition 4**.**

Let $\mathcal{A}_{1},\dots,\mathcal{A}_{l}$ be access structures on $\{1,\dots,n\}$. A combination of (perfect) secret sharing schemes with access structures $\mathcal{A}_{1},\dots,\mathcal{A}_{l}$ consists of random variables $S_{1},\dots,S_{l}$, $X_{1},\dots,X_{n}$ such that $(S_{i},X_{1},\dots,X_{n})$ is a (perfect) secret sharing scheme with access structure $\mathcal{A}_{i}$ for $i=1,\dots,l$ and such that

[TABLE]

This definition ensures that the secrets are independent in the sense that knowing some of the secrets provides no information about the other secrets. Formally, one can see that the secrets are probabilistically independent as follows: For any $A\notin\mathcal{A}_{i}$ (for example, $A=\emptyset$),

[TABLE]

In Definition 4, if two access structures $\mathcal{A}_{i},\mathcal{A}_{j}$ are identical, then we can replace $S_{i}$ and $S_{j}$ by a single random variable $(S_{i},S_{j})$ and obtain a smaller combination of (perfect) secret sharing schemes.

In a combination of perfect secret sharing schemes, it is very clear who knows what: Namely, a group of participants knows all secrets for which it is authorized, while it knows nothing about the remaining secrets. This motivates the following definition:

Definition 5**.**

A measure of shared information $I_{\cap}$ has the combined secret sharing property if and only if for any combination of perfect secret sharing schemes with access structures $\mathcal{A}_{1},\dots,\mathcal{A}_{l}$,

[TABLE]

(the entropy of those secrets for which $A_{1},\dots,A_{k}$ are all authorized). $I_{\cap}$ has the pairwise secret sharing property if and only if the same holds true in the special case $l=2$.

The combined secret sharing property implies the pairwise secret sharing property. The pairwise secret sharing property does not follow from the Williams and Beer axioms. For example, $I_{\min}$ satisfies the Williams and Beer axioms, but not the pairwise secret sharing property (as will become apparent in Theorem 2). So one can ask whether the pairwise and combined secret sharing properties are compatible with the Williams and Beer axioms. This question is difficult to answer, since currently there are only two proposed measures of shared information that satisfy the Williams and Beer axioms, namely $I_{\min}$ and the minimum of mutual informations (Barrett, 2014)

[TABLE]

Both measures do not satisfy the pairwise secret sharing property.

While there has been no further proposal for a function that satisfies the Williams and Beer axioms for arbitrarily many arguments, several measures have been proposed for the “bivariate case” $k=2$, notably $I_{\text{red}}$ of Harder et al. (2013) and $\widetilde{SI}$ of Bertschinger et al. (2014). The appendix shows that $\widetilde{SI}$ at least satisfies the combined secret sharing property “as far as possible.”

Combinations of $l$ perfect secret sharing schemes lead to information decompositions with at most $l$ nonzero partial information terms.

Lemma 4**.**

Assume that $I_{\cap}$ has the combined secret sharing property. If $(S_{1},\dots,S_{l}$, $X_{1},\dots,X_{n})$ is a combination of perfect secret sharing schemes with pairwise different access structures $\mathcal{A}_{1},\dots,\mathcal{A}_{l}$, then

[TABLE]

for any $A_{1},\dots,A_{k}\subseteq\{X_{1},\dots,X_{n}\}$.

The proof is similar to the proof of Lemma 3 and omitted.

The combined secret sharing property implies that any combination of nonnegative values can be prescribed as partial information values.

Proposition 1**.**

Suppose that a nonnegative number $h_{\mathcal{A}}$ is given for any antichain $\mathcal{A}$. For any measure of shared information that satisfies the combined secret sharing property, there exist random variables $S,X_{1},\dots,X_{n}$ such that the corresponding partial measure $I_{\partial}$ satisfies $I_{\partial}(S;A_{1},\dots,A_{k})=h_{A_{1},\dots,A_{k}}$ for all antichains $\mathcal{A}=(A_{1},\dots,A_{k})$.

Proof.

By Theorem 1, for each antichain $\mathcal{A}$ there exists a perfect secret sharing scheme $S_{\mathcal{A}}$, $X_{1,\mathcal{A}},\dots,X_{n,\mathcal{A}}$ with $H(S_{\mathcal{A}})=h_{\mathcal{A}}$. Combine independent copies of these perfect secret sharing schemes and let

[TABLE]

where $\mathcal{A}$ runs over all antichains. Then $S,X_{1},\dots,X_{n}$ is an independent combination of perfect secret sharing schemes, and the statement follows from Lemma 4. ∎

Unfortunately, not every random variable $S$ can be decomposed in such a way as a combination of secret sharing schemes. However, Proposition 1 suggests that, given a measure $I_{\cap}$ of shared information that satisfies the combined secret sharing property, $I_{\partial}(S;\underline{\mathcal{A}})$ can informally be interpreted as a measure that quantifies how much $(X_{1},\dots,X_{n},S)$ looks like a perfect secret sharing scheme with access structure $\mathcal{A}$.

Lemma 5**.**

Suppose that $I_{\cap}$ is a measure of shared information that satisfies the pairwise secret sharing property. If $X_{1}$ and $X_{2}$ are independent, then

[TABLE]

In the language of Ince (2017), the lemma says that the pairwise secret sharing property implies the independent identity property.

Proof.

Let $S_{1}=X_{1}$, $S_{2}=X_{2}$. Then $S_{1},S_{2},X_{1},X_{2}$ is a pair of perfect secret sharing schemes with access structures \mathcal{A}_{1}=\big{\{}\{1\}\big{\}} and \mathcal{A}_{2}=\big{\{}\{2\}\big{\}}. The statement follows from Definition 5, since $X_{1}$ is not authorized for $\mathcal{A}_{2}$ and $X_{2}$ is not authorized for $\mathcal{A}_{1}$. ∎

4. Incompatibility with local positivity

Unfortunately, although the combined secret sharing property very much fits the intuition behind the axioms of Williams and Beer, it is incompatible with a nonnegative decomposition according to the partial information lattice:

Theorem 2**.**

Let $I_{\cap}$ be a measure of shared information that satisfies the Williams-Beer axioms and has the pairwise secret sharing property. Then $I_{\partial}$ is not nonnegative.

Proof.

The XOR example, which was already used by Bertschinger et al. (2013) and Rauh et al. (2014) to prove incompatibility results for properties of information decompositions, can also be used here.

Let $X_{1},X_{2}$ be independent binary uniform random variables, let $X_{3}=X_{1}\oplus X_{2}$, and let $S=(X_{1},X_{2},X_{3})$. Observe that the situation is symmetric in $X_{1},X_{2},X_{3}$. In particular, $X_{2},X_{3}$ are also independent, and $X_{1}=X_{2}\oplus X_{3}$. The following values of $I_{\cap}$ can be computed from the assumptions:

• •

I_{\cap}\big{(}S;X_{1},(X_{2}X_{3})\big{)}=I_{\cap}\big{(}S;X_{1},(X_{1}X_{2}X_{3})\big{)}=I_{\cap}(S;X_{1})=1\text{,}\mathrm{b}\mathrm{i}\mathrm{t}$$, since $X_{1}$ is a function of $(X_{2},X_{3})$ and by the monotonicity axiom.

• •

I_{\cap}(S;X_{1},X_{2})=I_{\cap}\big{(}(X_{1}X_{2}X_{3});X_{1},X_{2}\big{)}=I_{\cap}\big{(}(X_{1}X_{2});X_{1},X_{2}\big{)}=0 by Lemma 5.

By monotonicity, $I_{\cap}(S;X_{1},X_{2},X_{3})=0$. Moreover,

[TABLE]

since $2\text{\,}\mathrm{b}\mathrm{i}\mathrm{t}$ is the total entropy in the system. But then

[TABLE]

where $\pm 0$ denotes values of $I_{\cap}$ that vanish. Thus, $I_{\partial}$ is not nonnegative. ∎

Note that the random variables $(S=(X_{1},X_{2},X_{3}),X_{1},X_{2},X_{3})$ from the proof of Theorem 2 form three perfect secret sharing schemes that do not satisfy the definition of a combination of perfect secret sharing schemes. The three secrets $X_{1},X_{2},X_{3}$ are not independent, but they are pair-wise independent (and so Lemma 4 does not apply).

Remark 1*.*

The XOR example from the proof of Theorem 2 (which was already used by Bertschinger et al. (2013) and Rauh et al. (2014)) was criticized by Chicharro and Panzeri (2017) on the grounds that it involves random variables that stand in a deterministic functional relation (in the sense that $X_{3}=X_{1}\oplus X_{2}$). Chicharro and Panzeri argue that in such a case it is not appropriate to use the full partial information lattice. Instead, the functional relationship should be used to eliminate (or identify) nodes from the lattice. Thus, while the monotonicity axiom of Williams and Beer implies $I_{\cap}(S;X_{3},(X_{2},X_{3}))=I_{\cap}(S;X_{3})$ (and so $\{3;23\}$ is not part of the partial information lattice), the same axiom also implies that $I_{\cap}(S;X_{3},(X_{1},X_{2}))=I_{\cap}(S;X_{3})$ in the XOR example, and so $\{3;12\}$ should similarly be excluded from the lattice when analyzing this particular example. But note that the first argument is a formal argument that is valid for all joint distributions of $S,X_{1},X_{2},X_{3}$, while the second argument takes into account the particular underlying distribution.

It is easy to work around this objection. The deterministic relationship disappears when an arbitrarily small stochastic noise is added to the joint distribution. To be precise, let $X_{1},X_{2}$ be independent binary random variables, and let $X_{3}$ be binary with

[TABLE]

for $0\leq\epsilon\leq 1$. For $\epsilon=0$, the example from the proof is recovered. Assuming that the partial information terms depend continuously on this joint distribution, the partial information term I_{\partial}\big{(}S;(X_{1}X_{2}),(X_{1}X_{3}),(X_{2}X_{3})\big{)} will still be negative for small $\epsilon>0$. Thus, assuming continuity, the conclusion of Theorem 2 still holds true when the information decomposition according to the full partial information lattice is only considered for random variables that do not satisfy any functional deterministic constraint.

Remark 2*.*

Analyzing the proof of Theorem 2, one sees that the independent identity axiom (Lemma 5) is the main ingredient to arrive at the contradiction. The same property also arises in the other uses of the XOR example (Bertschinger et al., 2013; Rauh et al., 2014).

5. Discussion

Perfect secret sharing schemes correspond to systems of random variables in which it is very clearly specified “who knows what.” In such a system, it is easy to assign intuitive values to the shared information nodes in the partial information lattice, and one may conjecture that the intuition behind this assignment is the same intuition that underlies the Williams and Beer axioms, which define the partial information lattice. Moreover, following the same intuition, independent combinations of perfect secret sharing schemes can be used as a tool to construct systems of random variables with prescribable (nonnegative) values of partial information.

Unfortunately, this extension to independent combinations of perfect secret sharing schemes is not without problems: By Theorem 2, it leads to decompositions with negative partial information terms. But what does it mean that the examples derived from the same intuition as the Williams and Beer axioms contradict the same axioms in this way? Is this an indication that the whole idea of information decomposition does not work (and that the question posed in the first paragraph of the introduction cannot be answered affirmatively)?

There are several ways out of this dilemma. The first solution is to assign different values to combinations of perfect secret sharing schemes. This solution will not be pursued further in this text, as it would change the interpretation of the information decomposition as measuring “who knows what.”

The second solution is to accept negative partial values in the information decomposition. It has been argued that negative values of information can be given an intuitive interpretation in terms of confusing or misleading information. For event-wise (also called “local”) information quantities, such as the event-wise mutual information $i(s;x)=\log(p(s)/p(s|x))$, this interpretation goes back to the early days of information theory Fano (1961). Sometimes, this phenomenon is called “misinformation” (Ince, 2017; Wibral et al., 2015). However, in the usual language, misinformation refers to “false or incorrect information, especially when it is intended to trick someone” Macmillan Publishers Limited (retrieved on 2017/10/05), which is not the effect that is modelled here. Thus, the word misinformation should be avoided, in order not to mislead the reader into the wrong intuition.

While negative event-wise information quantities are well-understood, the situation is more problematic for average quantities. When an agent receives side-information in the form of the value $x$ of a relevant random variable $X$, she changes her strategy. While the prior strategy should be based on the prior distribution $p(S)$, the new strategy should be based on the posterior $p(S|X=x)$. Clearly, in a probabilistic setting, any change of strategy can lead to a better or worse result in a single instance. On average, though, side-information never hurts (and it is never advantageous on average to ignore side-information), which is why the mutual information is never negative. Similarly, it is natural to expect non-negativity of other information quantities. It is difficult to imagine how correct side-information (or an aspect thereof) can be misleading on average. The situation is different for incorrect information, where the interpretation of a negative value is much easier.

More conceptually, I would suspect that an (averaged) information quantity that may change its sign actually conflates different aspects of information111One can argue whether the same should be true for event-wise quantities. Recently, Ince (2017) suggested to also write the event-wise mutual information as a difference of non-negative quantities., just as the interaction information (or co-information) conflates synergy and redundancy Williams and Beer (2010).

In any case, allowing negative partial values alters the interpretation of an information decomposition to a point where it is questionable whether the word “decomposition” is still appropriate. When decomposing an object into parts, the parts should in some reasonable way be sub-objects. For example, in a Fourier decomposition of a function, the Fourier components are never larger than the function (in the sense of the $L^{2}$-norm), and the sum of the squared $L^{2}$-norms of the Fourier coefficients equals the squared $L^{2}$-norm of the original function. As another example, given a (positive) amount of money and two investment options, it may indeed be possible to invest a negative share of the total amount into one of the two options in order to increase the funds that can be invested in the second option. However, such short selling is regulated in many countries with much stronger rules than ordinary trading.

I do not claim that an information decomposition with negative partial information terms cannot possibly make sense. However, it has to be made clear precisely how to interpret negative terms, and it is important to distinguish between correct information that leads to a suboptimal decision due to unlikely events happening (“bad luck”) and incorrect information that leads to decisions being based on the wrong posterior probabilities (as opposed to the “correct” conditional probabilities).

A third solution is to change the underlying lattice structure of the decomposition. A first step in this direction was done by Chicharro and Panzeri (2017) who propose to decompose mutual information according to subsets of the partial information lattice. However, it is also conceivable that the lattice has to be enlarged.

Williams and Beer derived the partial information lattice from their axioms together with the assumption that everything can be expressed in terms of shared information (that is, according to “who knows what”). Shared information is sometimes equivalently called redundant information, but it may be necessary to distinguish the two. Information that is shared by several random variables is information that is accessible to each single random variable, but redundancy can also arise at higher orders. An example is the infamous XOR example from the proof of Theorem 2: In this example, each pair $X_{i},X_{j}$ is independent and contains of two bits, but the total system $X_{1},X_{2},X_{3}$ has only two bits. Therefore, there is one bit of redundancy. However, this redundancy bit is not located anywhere specifically: It is not contained in either of $X_{1},X_{2},X_{3}$, and thus it is not shared information. Since the redundant bit is not part of $X_{1}$, it is not “shared” by $X_{1}$ in this sense. This phenomenon corresponds to the fact that random variables can be pairwise independent without being independent.

This kind of higher order redundancy does not have a place in the partial information lattice, so it may be that nodes corresponding to higher order redundancy have to be added. When the lattice is enlarged in this way, the structure of the Möbius inversion is changed, and it is possible that the resulting lattice leads to nonnegative partial information terms, without changing those cumulative information values that are already present in the original lattice. If this approach succeeds, the answer to the question from the introduction will be negative: Simply classifying information according to “who knows what” (i.e. shared information) does not work, since it does not capture higher order redundancy. The analysis of extensions of the partial information lattice is scope for future work.

Acknowledgments

I thank Fero Matúš for teaching me about secret sharing schemes. I am grateful to Guido Montúfar and Pradeep Kr. Banerjee for their remarks about the manuscript, and to Nils Bertschinger, Jürgen Jost and Eckehard Olbrich for many inspiring discussions on the topic. I thank the reviewers for many comments, in particular concerning the discussion. I thank the organizers and participants of the PID workshop in December 2016 in Frankfurt, where the material was first presented.

Appendix A Combined secret sharing properties for small $k$

This section discusses the defining equation (2) of the combined secret sharing property for $k=1$ and $k=2$. The case $k=1$ is incorporated in the definition of a combination of perfect secret sharing schemes: The following lemma implies that any measure of shared information that satisfies self-redundancy satisfies (2) for $k=1$. Recall that Williams and Beer’s self-redundancy axiom implies that $I_{\cap}(S;X_{A})=I(S;X_{A})$.

Lemma 6**.**

Let $(S_{1},\dots,S_{l},X_{1},\dots,X_{n})$ be a combination of perfect secret sharing schemes with access structures $\mathcal{A}_{1},\dots,\mathcal{A}_{l}$. Then

[TABLE]

Proof.

Suppose that the secret for which $A$ is authorized are $S_{1},\dots,S_{m}$. Then

[TABLE]

On the other hand,

[TABLE]

By independence (remark after Definition 4), $\sum_{i=m+1}^{l}H(S_{i})=H(S_{m+1},\dots,S_{l})$ and $\sum_{i=1}^{m}H(S_{i})=H(S_{1},\dots,S_{m})$. Thus,

[TABLE]

The next result shows that the bivariate measure of shared information $\widetilde{SI}(S;X,Y)$ proposed by Bertschinger et al. (2014) satisfies Eq. (2) for $k\leq 2$. The reader is referred to loc. cit. for definitions and elementary properties of $\widetilde{SI}$.

Proposition 2**.**

Let $(S_{1},\dots,S_{l},X_{1},\dots,X_{n})$ be a combination of perfect secret sharing schemes with access structures $\mathcal{A}_{1},\dots,\mathcal{A}_{l}$, Then

[TABLE]

Proof.

For given $A_{1},A_{2}$, suppose that $S_{1},\dots,S_{m}$ are the secrets for which at least one of $A_{1}$ or $A_{2}$ is authorized and that $S_{m+1},\dots,S_{l}$ are the secrets for which neither $A_{1}$ nor $A_{2}$ is authorized alone.

Let $P$ be the joint distribution of $S_{1},\dots,S_{l},X_{A_{1}},X_{A_{2}}$. Let $\Delta_{P}$ be the set of alternative joint distributions for $S_{1},\dots,S_{l},X_{A_{1}},X_{A_{2}}$ that have the same marginal distributions as $P$ on the subsets $(S_{1},\dots,S_{l},X_{A_{1}})$ and $(S_{1},\dots,S_{l},X_{A_{2}})$. According to the definition of $\widetilde{SI}$, we need to compare $P$ with the elements of $\Delta_{P}$ and find the maximum of H_{Q}\big{(}(S_{1},\dots,S_{l})\big{|}X_{A_{1}},X_{A_{2}}\big{)} over $Q\in\Delta_{P}$, where the subscript to $H$ indicates with respect to which of these joint distributions the conditional entropy is evaluated.

Define a distribution $Q^{*}$ for $S_{1},\dots,S_{l},X_{A_{1}},X_{A_{2}}$ by

[TABLE]

Then $Q^{*}\in\Delta_{P}$. Under $P$, the secrets $S_{m+1},\dots,S_{l}$ are independent of $X_{A_{1}}$ (marginally) and independent of $X_{A_{2}}$, and so $S_{m+1},\dots,S_{l}$ are independent of the pair $(X_{A_{1}},X_{A_{2}})$ under $Q^{*}$. On the other hand, $S_{1},\dots,S_{m}$ are a function of either $X_{A_{1}}$ or $X_{A_{2}}$ under $P$, and so $S_{1},\dots,S_{m}$ is a function of $(X_{A_{1}},X_{A_{2}})$ under $Q^{*}$. Thus,

[TABLE]

On the other hand, under any joint distribution $Q\in\Delta_{P}$, the secrets $S_{1},\dots,S_{m}$ are functions of $X_{A_{1}},X_{A_{2}}$, whence

[TABLE]

It follows that $Q^{*}$ solves the optimization problem in the definition of $\widetilde{SI}$.

Suppose that the secrets for which $X_{A_{1}}$ is authorized are $S_{1},\dots,S_{r}$ and that the secrets for which $X_{A_{2}}$ is authorized are $S_{s},\dots,S_{m}$ (with $1\leq r,s\leq m$). One computes

[TABLE]

whence

[TABLE]