Compressed Secret Key Agreement: Maximizing Multivariate Mutual Information Per Bit

TL;DR

This paper introduces a source compression approach to maximize secret key rates in multiterminal settings, linking it to multivariate mutual information and providing bounds and dualities for different models.

Contribution

It formulates a compressed secret key agreement problem, derives bounds, and establishes duality results, offering new insights and methods for secret key generation with source compression.

Findings

Derived single-letter bounds for general sources

Explicit polynomial-time formula for pairwise independent networks

Established duality for two-user one-way discussion case

Abstract

The multiterminal secret key agreement problem by public discussion is formulated with an additional source compression step where, prior to the public discussion phase, users independently compress their private sources to filter out strongly correlated components for generating a common secret key. The objective is to maximize the achievable key rate as a function of the joint entropy of the compressed sources. Since the maximum achievable key rate captures the total amount of information mutual to the compressed sources, an optimal compression scheme essentially maximizes the multivariate mutual information per bit of randomness of the private sources, and can therefore be viewed more generally as a dimension reduction technique. Single-letter lower and upper bounds on the maximum achievable key rate are derived for the general source model, and an explicit polynomial-time computable formula is obtained for the pairwise independent network model. In particular, the converse results and the upper bounds are obtained from those of the related secret key agreement problem with rate-limited discussion. A precise duality is shown for the two-user case with one-way discussion, and such duality is extended to obtain the desired converse results in the multi-user case. In addition to posing new challenges in information processing and dimension reduction, the compressed secret key agreement problem helps shed new light on resolving the difficult problem of secret key agreement with rate-limited discussion, by offering a more structured achieving scheme and some simpler conjectures to prove.

Equations

$${\mathsf{Z}}_V$$

$$Z_V$$

$${\mathsf{Z}}_i^n:=({\mathsf{Z}}_{it}|t\in [n])=({\mathsf{Z}}_{i1},{\mathsf{Z}}_{i2},\dots ,{\mathsf{Z}}_{in})$$

$$H({\mathsf{U}}_V|{\mathsf{Z}}_V)=\sum _{i\in V}H({\mathsf{U}}_i).$$

$${\tilde{\mathsf{Z}}}_i$$

$${\tilde{\mathsf{F}}}_t$$

$${\mathsf{F}}_i$$

$$\mathsf{F}$$

$$\underset{n\to \infty }{\lim }\text{Pr}(\exists i\in V,\mathsf{K}\ne {\theta }_i({\tilde{\mathsf{Z}}}_i,\mathsf{F}))=0,$$

$$\underset{n\to \infty }{\lim }\frac{1}{n}‘1[\log |K|-H(\mathsf{K}|\mathsf{F})‘2]=0,$$

$${\tilde{C}}_{\text{operator@font}S}(‘a)$$

$$\underset{n\to ‘8}{\mathrm{limsup}}\frac{1}{n}H({\tilde{\mathsf{Z}}}_V)-‘a\le 0.$$

$$\underset{n\to \infty }{\mathrm{limsup}}\frac{1}{n}H({\tilde{\mathsf{Z}}}_B)-‘a\le 0.$$

$$C_{\text{operator@font}S}(R)$$

$$R\ge \underset{n\to \infty }{\mathrm{limsup}}\frac{1}{n}\log |F|=\underset{n\to \infty }{\mathrm{limsup}}\frac{1}{n}\sum _{i\in V}\log |F_i|.$$

$$C_{\text{operator@font}S}(‘8)$$

$$R_{\text{operator@font}S}:=\inf \{R\ge 0\mid C_{\text{operator@font}S}(R)=C_{\text{operator@font}S}(‘8)\}.$$

$$C_{\text{operator@font}S}(‘8)=\underset{‘a\to \infty }{\mathrm{liminf}}{\tilde{C}}_{\text{operator@font}S}(‘a),$$

$$‘a_{\text{operator@font}S}:=\inf \{‘a\mid {\tilde{C}}_{\text{operator@font}S}(‘a)=C_{\text{operator@font}S}(‘8)\}$$

$${\mathsf{Z}}_1:=({\mathsf{X}}_a,{\mathsf{X}}_b),{\mathsf{Z}}_2:=({\mathsf{X}}_a,{\mathsf{X}}_b,{\mathsf{X}}_c),\text{and}{\mathsf{Z}}_3:=({\mathsf{X}}_a,{\mathsf{X}}_c),$$

$${\tilde{C}}_{\text{operator@font}S}(‘a)\ge ‘a\text{for ‘a∈[0,1]}.$$

$${\tilde{C}}_{\text{operator@font}S}(‘a)$$

$$C_{\text{operator@font}S,1}(R)$$

$$I({\mathsf{Z}}_1^{\prime }\wedge {\mathsf{Z}}_1)-I({\mathsf{Z}}_1^{\prime }\wedge {\mathsf{Z}}_2)\le R$$

$$I({\mathsf{Z}}_1^{\prime }\wedge {\mathsf{Z}}_2|{\mathsf{Z}}_1)=0.$$

$$J_{\text{operator@font}GK}({\mathsf{Z}}_1\wedge {\mathsf{Z}}_2):=\sup \{H(\mathsf{U})\mid H(\mathsf{U}|{\mathsf{Z}}_1)=H(\mathsf{U}|{\mathsf{Z}}_2)=0\}$$

$$R_{\text{operator@font}S,1}$$

$$J_{\text{operator@font}W,1}({\mathsf{Z}}_1\wedge {\mathsf{Z}}_2):=\inf \{H(\mathsf{W})\mid H(\mathsf{W}|{\mathsf{Z}}_1)=0,I({\mathsf{Z}}_1\wedge {\mathsf{Z}}_2|\mathsf{W})=0\}$$

$${\tilde{C}}_{\text{operator@font}S,1}(‘a)$$

$$I({\mathsf{Z}}_1^{\prime }\wedge {\mathsf{Z}}_1)\le ‘a$$

$$I({\mathsf{Z}}_1^{\prime }\wedge {\mathsf{Z}}_2|{\mathsf{Z}}_1)=0.$$

$$C_{\text{operator@font}S}(‘8)$$

$$R_{\text{operator@font}CO}$$

$$r(B)$$

$$R_{\text{operator@font}S}$$

Full text

1 Introduction

In Information-theoretic security, the secret key agreement problem by public discussion is the problem where a group of users discuss in public to generate a common secret key that is independent of their discussion. The problem was first formulated by Maurer [34], Ahlswede and Csiszár [1] under a private source model involving two users who observe some correlated private sources. Rather surprisingly, public discussion was shown to be useful in generating the secret key, i.e., it strictly increases the maximum achievable key rate called the secrecy capacity. Such phenomenon was also discovered in [4] in a different formulation. Furthermore, the secrecy capacity was given an information-theoretically appealing characterization— it is equal to Shannon’s mutual information [41] between the two private sources, assuming the wiretapper can listen to the entire public discussion but not observe any other side information of the private sources. It was also shown that the capacity can be achieved by one-way public discussion, i.e., with only one of the users discusses in public.

As a simple illustration, let $\mathsf{X}_{0}$, $\mathsf{X}_{1}$ and $\mathsf{J}$ be three uniformly random independent bits, and suppose user $1$ observes $\mathsf{Z}_{1}:=(\mathsf{X}_{0},\mathsf{X}_{1})$ privately while user $2$ observes $\mathsf{Z}_{2}:=(\mathsf{X}_{\mathsf{J}},\mathsf{J})$, where $\mathsf{X}_{\mathsf{J}}=\mathsf{X}_{0}$ when $\mathsf{J}=0$ but $\mathsf{X}_{\mathsf{J}}=\mathsf{X}_{1}$ when $\mathsf{J}=1$. If user $2$ reveals $\mathsf{J}$ in public, then user $1$ can recover $\mathsf{X}_{\mathsf{J}}$ and therefore $\mathsf{Z}_{2}$. Furthermore, since $\mathsf{X}_{\mathsf{J}}$ is independent of $\mathsf{J}$, it can serve as a secret key bit that is recoverable by both users but remains perfectly secret to a wiretapper who observes only the public message $\mathsf{J}$. This scheme achieves the secrecy capacity equal to the mutual information $I(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})=1$ roughly because user $2$ reveals $H(\mathsf{Z}_{2}|\mathsf{Z}_{1})=1$ bit in public so there is $H(\mathsf{Z}_{2})-H(\mathsf{Z}_{2}|\mathsf{Z}_{1})=I(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$ bits of randomness left for the secret key. However, if no public discussion is allowed, it follows from the work of Gác and Körner [27] that no common secret key bit can be extracted from the sources. In particular, $\mathsf{X}_{\mathsf{J}}$ cannot be used as a secret key because user $1$ does not know whether $\mathsf{X}_{\mathsf{J}}$ is $\mathsf{X}_{0}$ or $\mathsf{X}_{1}$. $\mathsf{X}_{0}$ and also $\mathsf{X}_{1}$ cannot be used as a secret key either because they may not be observed by user $2$ when $\mathsf{J}=1$ and $\mathsf{J}=0$ respectively. It can be seen that, while the private sources are clearly statistical dependent, public discussion is needed to consolidate the mutual information of the sources into a common secret key.

The secret key agreement formulation was subsequently extended to the multi-user case by Csiszár and Narayan [22]. Some users are also allowed to act as helpers who can participate in the public discussion but need not share the secret key. The designated set of users who need to share the secret key are referred to as the active users. Different from the two-user case, one-way discussion may not achieve the secrecy capacity when there are more than two users. Instead, an omniscience strategy was considered in [22] where the users first communicate minimally in public until omniscience, i.e., the users discuss in public at the smallest total rate until every active user can recover all the private sources. The scheme was shown to achieve the secrecy capacity in the case when the wiretapper only listens to the public discussion. This assumes, however, that the public discussion is lossless and unlimited in rate, and the sources take values from finite alphabet sets. If the sources were continuous or if the public discussion were limited to a certain rate, it may be impossible to attain omniscience.

This work is motivated by the search of a better alternative to the omniscience strategy for multiterminal secret key agreement. A prior work of Csiszár and Narayan [21] considered secret key agreement under rate-limited public discussion. The model involves two users and a helper observing correlated discrete memoryless sources. The public discussion by the users is conducted in a particular order and direction. While the region of achievable secret key rate and discussion rates remains unknown, single-letter characterizations involving two auxiliary random variables were given for many special cases, including the two-user case with two rounds of interactive public discussion, where each user speaks once in sequence, with the last public message possibly depending on the first. By further restricting to one-way public discussion, the characterization involves only one auxiliary random variable and was extended to continuous sources by Watanabe and Oohama in [48], where they also gave an explicit characterization without any auxiliary random variable for scalar Gaussian sources in [48]. For vector Gaussian sources, the characterization by the same authors in [49] involving some matrix optimization was further improved in [31] to a more explicit formula. However, if the discussion is allowed to be two-way and interactive, Tyagi [45] showed with a concrete two-user example that the minimum total discussion rate required, called the communication complexity, can be strictly reduced. Using the technique of Kaspi [30], multi-letter characterizations were given in [45] for the communication complexity and, similarly, by Liu et al. in [32] for the region of achievable secret key rate. [32] further simplified the characterization using the idea of convex envelope using the technique by Ma et al [33]. While these characterizations provide many new insights and properties, they are not considered computable, compared to the usual single-letter and explicit characterizations. Further extension to the multi-user case also appears difficult, as the converse can be seen to rely on the Csiszár sum identity [1, Lemma 4.1], which does not appear to extend beyond the two-user case.

Nevertheless, partial solutions under more restrictive public discussion constraints were possible. By simplifying the problem to the right extent, new results were discovered in the multi-user case, which has led to the formulation in this work. For instance, Gohari and Anantharam [28] characterized the secrecy capacity in the multi-user case under the simpler vocality constraint where some users have to remain silent throughout the public discussion. Using this result, simple necessary and sufficient conditions can be derived as to whether a user can remain silent without diminishing the maximum achievable key rate [36, 50, 7]. This is a simpler result than characterizing the achievable rate region because it does not say how much discussion is required if a user must discuss. Another line of work [19, 35, 37, 9] follows [45] to characterize the communication complexity but in the multi-user case. Courtade and Halford [19] characterized the communication complexity under a special non-asymptotic hypergraphical source model with linear discussion. [37] obtained a multi-letter lower bound on the communication complexity for the asymptotic general source model. It also gave a precise and simple condition under which the omniscience strategy for secret key agreement is optimal for a special source model called the pairwise independent network (PIN) [40], which is a special hypergraphical source model [18]. [9, 17] further derived some single-letter and more easily computable explicit lower bounds, from which one can also obtain conditions for the omniscience strategy to be optimal under the hypergraphical source model, which covers the PIN model as a special case. [10] considered the more general problem of characterizing the multiterminal secrecy capacity under rate-limited public discussion. In particular, an objective of [10] is to characterize the constrained secrecy capacity defined as the maximum achievable key rate as a function of the total discussion rate. This covers the communication complexity as a special case when further increase in the public discussion rate does not increase the secrecy capacity. While only single-letter bounds were derived for the general source model, a surprisingly simple explicit formula was derived for the PIN model [10]. The optimal scheme in [10] follows the tree-packing protocol in [39]. It turns out to belong to the more general approach of decremental secret key agreement in [6, 5] inspired by the achieving scheme in [19] and the notion of excess edge in [18]. More precisely, the omniscience strategy is applied after some excess or less useful edge random variables are removed (decremented) from the source. Since the entropy of the decremented source is smaller, the discussion required to attain omniscience of the decremented source is also smaller. Such decremental secret key agreement approach applies to hypergraphical sources more generally, and it results in one of the best upper bounds in [35] for communication complexity. However, for more general source models that are not necessarily hypergraphical, the approach does not directly apply.

The objective of this work is to formalize and extend the idea of decremental secret key agreement beyond the hypergraphical source model. More precisely, the secret key agreement problem is considered with an additional source compression step before public discussion where each user independently compresses their private source component to filter away less correlated randomness that does not contribute much to the achievable secret key rate. The compression is such that the entropy rate of the compressed sources is reduced to under certain specified level. In particular, the edge removal process in decremental secret key agreement can be viewed as a special case of source compression, and the more general problem will be referred to as compressed secrecy key agreement. The objective is to characterize the achievable secret key rate maximized over all valid compression schemes. For simplicity, this work will focus on the case without helpers, i.e., when all users are active and want to share a common secret key. A closely related formulation is by Nitinawarat and Narayan [38], which characterized the maximum achievable key rate for the two-user case under the scalar gaussian source model where one of the user is required to quantize the source to within a given rate. [46] also extended the formulation and techniques in [38] to the multi-user case where every user can quantize their sources individually to a certain rate. The compression considered in this work is more general than quantizations for gaussian sources, and the new results are meaningful beyond continuous sources.

The compressed secret key agreement problem is also motivated by the study of multivariate mutual information (MMI) [15], i.e., an extension of Shannon’s mutual information to the multivariate case involving possibly more than two random variables. The unconstrained secrecy capacity in the no-helper case has been viewed as a measure of mutual information in [11, 15], not only because of its mathematically appealing interpretations such as the residual independence relation and data processing inequalities in [15], but also because of its operational significance in undirected network coding [13, 14], data clustering [8] and feature selection [16] (cf. [20]). The optimal source compression scheme that achieves the compressed secrecy capacity can be viewed more generally as an optimal dimension reduction procedure that maximizes the MMI per bit of randomness, which is an extension of the information bottleneck problem [44] to the multivariate case. However, different from the multivariate extension in [25], the MMI is used instead of Watanabe’s total correlation [47], and so it captures only the information mutual to all the random variables rather than the information mutual to any subsets of the random variables. Furthermore, the compression is on each random variable rather than subsets of random variables.

The paper is organized as follows. The problem of compressed secret key agreement is formulated in Section 2. Preliminary results of secret key agreement are given in Section 3. The main results are motivated in Section 4 and presented in Section 5, followed by the conclusion and some discussions on potential extensions in Section 6.

2 Problem Formulation

Similar to the multiterminal secret key agreement problem [22] without helpers or wiretapper’s side information, the setting of the problem involves a finite set $V$ of $\lvert V\rvert>1$ users, and a discrete memoryless multiple source

[TABLE]

N.b., letters in sans serif font are used for random variables and the corresponding capital letters in the usual math italic font denote the alphabet sets. $P_{\mathsf{Z}_{V}}$ denotes the joint distribution of $\mathsf{Z}_{i}$’s.

A secret key agreement protocol with source compression can be broken into the following phases:

Private observation:

Each user $i\in V$ observes an $n$-sequence

[TABLE]

i.i.d. generated from the source $\mathsf{Z}_{i}$ for some block length $n$. N.b., for convenience, $[n]$ denotes the set of positive intergers up to $n$, i.e, $\{1,\dots,n\}$.

Private randomization:

Each user $i\in V$ generates a random variable $\mathsf{U}_{i}$ independent of the private source, i.e.,

[TABLE]

Source compression:

Each user $i\in V$ computes

[TABLE]

for some function $`z_{i}$ that maps to a finite set. $\tilde{\mathsf{Z}}_{V}$ is referred to as the compressed source.

Public discussion:

Using a public authenticated noiseless channel, a user $i_{t}\in V$ is chosen in round $t\in[\ell]$ to broadcast a message

[TABLE]

Key generation:

A random variable $\mathsf{K}$, called the secret key, is required to satisfy the recoverability constraint that

[TABLE]

for some function $\theta_{i}$, and the secrecy constraint that

[TABLE]

where $K$ denotes the finite alphabet set of possible key values.

N.b., unlike [45], non-interactive discussion is considered different from one-way discussion in the two-user case since both users are allowed to discuss even though their messages cannot depend on each other. Different from [23], there is an additional source compression phase, after which the protocol can only depend on the origninal sources through the compressed sources.

The objective is to characterize the maximum achievable secret key rate for a continuum of different levels of source compression:

Definition 1

The compressed secrecy capacity with a joint entropy limit $`a\geq 0$ is defined as

[TABLE]

where the supremum is over all possible compressed secret key agreement schemes satisfying

[TABLE]

This constraint limits the joint entropy rate of the compressed source.

N.b., instead of the joint entropy limit, one may also consider entropy limits on some subset $B\subseteq V$ that

[TABLE]

If multiple entropy limits are imposed, $\tilde{C}_{\mathop{\operator@font S}}$ will be a higher-dimensional surface instead of a one-dimensional curve. For example, in the two-user case under the scalar gaussian source model, [38] considered the entropy limit only on one of the users. In the multi-user case under the gaussian markov tree model, [46] considered the symmetric case where the entropy limit is imposed on every user.

For simplicity, however, the joint entropy constraint (7) will be the primary focus in this work. It will be shown that $\tilde{C}_{\mathop{\operator@font S}}(`a)$ is closely related to the constrained secrecy capacity $C_{\mathop{\operator@font S}}(R)$ defined as [10]

[TABLE]

with $\tilde{\mathsf{Z}}_{i}:=(\mathsf{U}_{i},\mathsf{Z}_{i}^{n})$ instead of (2), i.e., without compression, and the entropy limit (7) replaced by the constraint on the total discussion rate

[TABLE]

N.b., it follows directly from the result of [22] that $\tilde{C}_{\mathop{\operator@font S}}(`a)$ remains unchanged whether the discussion is interactive or not. Indeed, the relation between $\tilde{C}_{\mathop{\operator@font S}}(`a)$ and $C_{\mathop{\operator@font S}}(R)$ to be shown in this work will not be affected either. Therefore, for notational simplicity, $C_{\mathop{\operator@font S}}(R)$ may refer to the case with or without interaction, even though $C_{\mathop{\operator@font S}}(R)$ may be smaller with non-interactive discussion.

It is easy to show that $C_{\mathop{\operator@font S}}(R)$ is continuous, non-decreasing and concave in $R$ [10, Proposition 3.1]. As $R$ goes to $`8$, the secrecy capacity

[TABLE]

is the usual unconstrained secrecy capacity defined in [22] without the discussion rate constraint (10). The smallest discussion rate that achieves the unconstrained secrecy capacity is the communication complexity denoted by

[TABLE]

Similar to $C_{\mathop{\operator@font S}}(R)$, the following basic properties can be shown for $\tilde{C}_{\mathop{\operator@font S}}(`a)$:

Proposition 1

$\tilde{C}_{\mathop{\operator@font S}}(`a)$* is continuous, non-decreasing and concave in $`a\geq 0$. Furthermore,*

[TABLE]

achieving the unconstrained secrecy capacity in the limit.

Proof

Continuity, monotonicity and (13) follow directly from the definition of $\tilde{C}_{\mathop{\operator@font S}}(`a)$. Concavity follows from the usual time-sharing argument, i.e., for any $`l\in[0,1]$, $`a^{\prime},`a^{\prime\prime}>0$, a secret key rate of $`l\tilde{C}_{\mathop{\operator@font S}}(`a^{\prime})+(1-`l)\tilde{C}_{\mathop{\operator@font S}}(1-`a^{\prime})$ is achievable with the entropy limit $`a:=`l`a^{\prime}+(1-`l)`a^{\prime\prime}$ by applying the optimal scheme that achieves $\tilde{C}_{\mathop{\operator@font S}}(`a^{\prime})$ for the first $n^{\prime}:=\lfloor`ln\rfloor$ samples of $\mathsf{Z}_{V}^{n}$ and applying the optimal scheme that achieves $\tilde{C}_{\mathop{\operator@font S}}(`a^{\prime\prime})$ for the remaining $n^{\prime\prime}:=n-n^{\prime}$ samples.

Because of (13), a quantity playing the same role of $R_{\mathop{\operator@font S}}$ for $C_{\mathop{\operator@font S}}$ can be defined for $\tilde{C}_{\mathop{\operator@font S}}(`a)$ as follows.

Definition 2

The smallest entropy limit that achieves the unconstrained secrecy capacity is defined as

[TABLE]

and referred to as the minimum admissible joint entropy.

One may also consider both the entropy limit (7) and discussion rate constraint (10) simultaneously, and define the secrecy capacity as a function of $`a$ and $R$. For simplicity, however, we will not consider this case but, instead, focus on the relationship between $\tilde{C}_{\mathop{\operator@font S}}(`a)$ and $C_{\mathop{\operator@font S}}(R)$.

The following example illustrates the problem formulation. It will be revisited at the end of Section 5 (Example 3) to illustrate the main results.

Example 1

Consider $V:=\{1,2,3\}$ and

[TABLE]

where $\mathsf{X}_{a},\mathsf{X}_{b}$ and $\mathsf{X}_{c}$ are uniformly random and independent bits. It is easy to argue that

[TABLE]

For $`a=H(\mathsf{Z}_{V})=H(\mathsf{X}_{a},\mathsf{X}_{b},\mathsf{X}_{c})=3$, every user can simply retain their source without compression, i.e., with $\tilde{\mathsf{Z}}_{i}=\mathsf{Z}_{i}$ for $i\in V$ while satisfying the entropy limit (7). Now, with $\mathsf{K}=(\mathsf{X}_{a}^{n},\mathsf{X}_{b}^{n})$ and $\mathsf{F}=\mathsf{F}_{2}=\mathsf{X}_{b}^{n}\oplus\mathsf{X}_{c}^{n}$ where $\oplus$ is the elementwise XOR, it can be shown that both the recoverability (4) and secrecy (5) constraints hold. This is because user $3$ can recover $\mathsf{X}_{b}$ from the XOR $\mathsf{X}_{b}\oplus\mathsf{X}_{c}$ with the side information $\mathsf{X}_{c}$. Furthermore, the XOR bit is independent of $(\mathsf{X}_{a},\mathsf{X}_{b})$ and therefore does not leak any information about the key bits. With this scheme, $\tilde{C}_{\mathop{\operator@font S}}(3)\geq 2$. By the usual time-sharing argument,

[TABLE]

Indeed, the reverse inequality can be argued using one of the main results (Theorem 5.1) and so the minimum admissible joint entropy will turn out to be $`a_{\mathop{\operator@font S}}=3$.

3 Preliminaries

In this section, a brief summary of related results for the secrecy capacity and communication complexity will be given. The results for the two-user case will be introduced first, followed by the more general results for the multi-user case, and the stronger results for the special hypergraphical source model. An example will also be given at the end to illustrate some of the results.

3.1 Two-user case

As mentioned in the introduction, no single-letter characterization is known for $C_{\mathop{\operator@font S}}(R)$ and $\tilde{C}_{\mathop{\operator@font S}}(`a)$ even in the two-user case where $V:=\{1,2\}$. Furthermore, while multi-letter characterizations for $R_{\mathop{\operator@font S}}$ and $C_{\mathop{\operator@font S}}(R)$ were given in [45] and [32] respectively in the two-user case under interactive discussion, no such multi-letter characterization is known for the case with non-interactive discussion. Nevertheless, if one-way discussion from user $1$ is considered, then the result of [21, Theorem 2.4] and its extension [48] to continuous sources gave the following characterization of $C_{\mathop{\operator@font S}}(R)$:

[TABLE]

The last constraint (17c) corresponds to the Markov chain $\mathsf{Z}^{\prime}_{1}"-"\mathsf{Z}_{1}"-"\mathsf{Z}_{2}$ and so the supremum is taken over the choices of the conditional distribution $P_{\mathsf{Z}^{\prime}_{1}|\mathsf{Z}_{1}}=P_{\mathsf{Z}^{\prime}_{1}|\mathsf{Z}_{1},\mathsf{Z}_{2}}$. Using the double Markov property as in [45], it follows that $C_{\mathop{\operator@font S}}(0)$ can be characterized more explicitly by the Gács–Körner common information

[TABLE]

where $\mathsf{U}$ is a discrete random variable. If (18) is finite, a unique optimal solution $\mathsf{U}$ exists and is called the maximum common function of $\mathsf{Z}_{1}$ and $\mathsf{Z}_{2}$ because any common function of $\mathsf{Z}_{1}$ and $\mathsf{Z}_{2}$ must be a function of $\mathsf{U}$. The communication complexity also has a more explicit characterization [45, (44)]

[TABLE]

and $\mathsf{W}$ is a discrete random variable. If $J_{\mathop{\operator@font W},1}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$ is finite, a unique optimal solution $\mathsf{W}$ exists and is called the minimum sufficient statistics of $\mathsf{Z}_{1}$ for $\mathsf{Z}_{2}$ since $\mathsf{Z}_{2}$ can only depend on $\mathsf{Z}_{1}$ through $\mathsf{W}$.

In Section 4, the expression $C_{\mathop{\operator@font S,1}}(R)$ will be related to the compressed secret key agreement restricted to the two-user case when the entropy limit is imposed only on user $1$. This duality relationship in the two-user case will serve as the motivation of the main results for the multi-user case. Indeed, the desired characterization of $\tilde{C}_{\mathop{\operator@font S}}(`a)$ for the two-user case has appeared in [38, Lemma 4.1] for the scalar gaussian source model:

[TABLE]

For the general source model, the expression (21) has also appeared before with other information-theoretic interpretations as mentioned in [24]. The lagrangian dual of (21), in particular, reduces to the dimension reduction technique called the information bottleneck method in [44], where $\mathsf{Z}_{1}$ is an observable used to predict the target $\mathsf{Z}_{2}$, and $\mathsf{Z}^{\prime}_{1}$ is a feature of $\mathsf{Z}_{1}$ that captures as much mutual information with the target variable as possible per bit of mutual information with the observable. Interestingly, the principal of the information bottleneck method was also proposed in  [43, 42] as a way to understand deep learning, since the best prediction of $\mathsf{Z}_{2}$ from $\mathsf{Z}_{1}$ is nothing but a particular feature of $\mathsf{Z}_{1}$ sharing a lot of mutual information with $\mathsf{Z}_{2}$.

3.2 General source with finite alphabet set

Consider the multi-user case where $\lvert V\rvert\geq 2$. If $\mathsf{Z}_{V}$ takes values from a finite set, then the unconstrained secrecy capacity was shown in [22] to be achievable via communication for omniscience (CO) and equal to

[TABLE]

where $R_{\mathop{\operator@font CO}}$ is the smallest rate of CO [22] characterized by the linear program

[TABLE]

where $r(B)$ denotes the sum $\sum_{i\in B}r_{i}$. Further, $R_{\mathop{\operator@font CO}}$ can be achieved by non-interactive discussion. It follows that

[TABLE]

It was also pointed out in [22] that private randomization does not increase $C_{\mathop{\operator@font S}}(`8)$. Hence, if $Z_{V}$ is finite, we have

[TABLE]

because $C_{\mathop{\operator@font S}}(`8)$ can be achieved with $\tilde{\mathsf{Z}}_{i}=\mathsf{Z}_{i}$. While it seems plausible that randomization does not decrease $R_{\mathop{\operator@font S}}$ nor increase $C_{\mathop{\operator@font S}}(R)$ for any $R\geq 0$, a rigorous proof remains elusive. Similarly, it appears plausible that neither $`a_{\mathop{\operator@font S}}$ nor $\tilde{C}_{\mathop{\operator@font S}}(`a)$ are affected by randomization but, again, no proof is known yet.

An alternative characterization of $C_{\mathop{\operator@font S}}(`8)$ was established in [11, 18] by showing that the divergence bound in [22] is tight in the case without helpers. More precisely, with $\Pi^{\prime}(V)$ defined as the set of partitions of $V$ into at least two non-empty disjoint sets, then

[TABLE]

In the bivariate case when $V=\{1,2\}$, $I(\mathsf{Z}_{V})$ reduces to Shannon’s mutual information $I(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$. It was further pointed out in [15] that $I(\mathsf{Z}_{V})$ is the minimum solution $`g$ to the residual independence relation

[TABLE]

for some $\mathcal{P}\in\Pi^{\prime}(V)$. To get an intuition of the above relation, notice that $`g=0$ is a solution when the joint entropy $H(\mathsf{Z}_{V})$ on the left is equal to the sum of entropies $H(\mathsf{Z}_{C})$’s on the right for some partition $\mathcal{P}$. In other words, the MMI is the smallest value of $`g$ removal of which leads to an independence relation, i.e., the total residual randomness on the left is equal to the sum of individual residual randomness on the right according to some partitioning of the random variables. It was further shown in [15] that there is a unique finest optimal partition to (26a) with a clustering interpretation in [8]. The MMI is also computable in polynomial time, following the result of Fujishige [26].

In the opposite extreme with $R\to 0$, it is easy to argue that

[TABLE]

where $J_{\mathop{\operator@font GK}}(\mathsf{Z}_{V})$ is the multivariate extension of the Gács–Körner common information in (18)

[TABLE]

with $\mathsf{U}$ again chosen as a discrete random variable. Note that, even without any public discussion, every user can compress their source independently to $\mathsf{U}^{n}$ where $\mathsf{U}$ is the maximum common function if $J_{\mathop{\operator@font GK}}(\mathsf{Z}_{V})$ is finite. Hence, it is easy to achieve a secret key rate of $H(\mathsf{U})=J_{\mathop{\operator@font GK}}(\mathsf{Z}_{V})$ without any discussion. The reverse inequality of (28) seems plausible but has not been proven yet except in the two-user case. The technique in [21] which relies on the Csiszár sum identity does not appear to extend to the multi-user case to give a matching converse.

3.3 Hypergraphical sources

Stronger results have been derived for the following special source model:

Definition 3 (Definition 2.4 of [18])

$\mathsf{Z}_{V}$ is a hypergraphical source w.r.t.  a hypergraph $(V,E,`x)$ with edge functions $`x:E\to 2^{V}`/\{\emptyset\}$ iff, for some independent edge variables $\mathsf{X}_{e}$ for $e\in E$ with $H(\mathsf{X}_{e})>0$,

[TABLE]

In the special case when the hypergraph is a graph, i.e., $\lvert`x(e)\rvert=2$, the model reduces to the pairwise independent network (PIN) model in [40]. The hypergrahical source can also be viewed as a special case of the finite linear source considered in [12] if the edge random variables take values from a finite field.

For hypergraphical sources, various bounds on $R_{\mathop{\operator@font S}}$ and $C_{\mathop{\operator@font S}}(R)$ have been derived in [35, 37, 9, 10]. The achieving scheme makes use of the idea of decremental secret key agreement [6, 5], where the redundant or less useful edge variables are removed or reduced before public discussion. This is a special case of the compressed secret key agreement, where the compression step simply selects the more useful edge variables up to the joint entropy limit.

For the PIN model, it turns out that decremental secret key agreement is optimal, leading to a single-letter characterization of $R_{\mathop{\operator@font S}}$ and $C_{\mathop{\operator@font S}}(R)$ in [10]:

[TABLE]

It can be verified that (31a) is the smallest value of $R$ such that $C_{\mathop{\operator@font S}}(R)=C_{\mathop{\operator@font S}}(`8)$ using (31b). While the proof of converse, i.e., $\leq$ for (31b), is rather involved, the achievability is by a simple tree packing protocol, which belongs to the decremental secret key agreement approach that removes excess edges unused for the maximum tree packing. In other words, the achieving scheme is a compressed secret key agreement scheme. This connection will lead to a single-letter characterization of $\tilde{C}_{\mathop{\operator@font S}}(`a)$ for the PIN model (in Theorem 5.2).

To illustrate the above results, a single-letter characterization for $C_{\mathop{\operator@font S}}(R)$ will be derived in the following for the source in Example 1. It will also demonstrate how an exact characterization for $C_{\mathop{\operator@font S}}(R)$ can be extended from a PIN model to a hypergraphical model via some contrived arguments. The characterization will also be useful later in Example 3 to give an exact characterization of $\tilde{C}_{\mathop{\operator@font S}}(`a)$.

Example 2

The source defined in (15) in Example 1, for instance, is a hypergraphical source with $E=\{a,b,c\}$, $`x(a)=\{1,2,3\}$, $`x(b)=\{1,2\}$ and $`x(c)=\{2,3\}$. By (23), we have $R_{\mathop{\operator@font CO}}=1$ with the optimal solution $r_{1}=r_{3}=0$ and $r_{2}=1$. This means that user $2$ needs to discuss $1$ bit to attain omniscience. In particular, user $2$ can reveal the XOR $\mathsf{X}_{b}\oplus\mathsf{X}_{c}$ so that user $1$ and $3$ can recover $\mathsf{X}_{c}$ and $\mathsf{X}_{b}$ respectively from their observations. By (24b), then, we have

[TABLE]

It can also be checked that the alternative characterization of $C_{\mathop{\operator@font S}}(`8)$ in (26) gives

[TABLE]

Next, we argue that

[TABLE]

The achievability, i.e., the inequality $C_{\mathop{\operator@font S}}(R)\geq 1+R$, is by the usual time-sharing argument. In particular, the bound $C_{\mathop{\operator@font S}}(0.5)\geq 1.5$, for example, can be achieved by the compressed secret key agreement scheme in Example 1 with $`a=2$, i.e., by time-sharing the compressed secret key agreement schemes for $`a=1$ and for $`a=3$ equally. More precisely, we set $\tilde{\mathsf{Z}}_{1}=(\mathsf{X}_{a}^{n},\mathsf{X}_{b}^{\lfloor n/2\rfloor})$, $\tilde{\mathsf{Z}}_{2}=(\mathsf{X}_{a}^{n},\mathsf{X}_{b}^{\lfloor n/2\rfloor},\mathsf{X}_{c}^{\lfloor n/2\rfloor})$, $\tilde{\mathsf{Z}}_{3}=(\mathsf{X}_{a}^{n},\mathsf{X}_{c}^{\lfloor n/2\rfloor})$, $\mathsf{K}=(\mathsf{X}_{a}^{n},\mathsf{X}_{b}^{\lfloor n/2\rfloor})$ and $\mathsf{F}=\mathsf{F}_{2}=\mathsf{X}_{b}^{\lfloor n/2\rfloor}\oplus\mathsf{X}_{c}^{\lfloor n/2\rfloor}$. It follows that the public discussion rate is $\limsup_{n\to\infty}\frac{1}{n}\log\lvert F\rvert=0.5$.

Now, to prove the reverse inequality $\leq$ for (33), we modifies the source $\mathsf{Z}_{V}$ to another source $\mathsf{Z}^{\prime}_{V}$ defined as follows with an additional uniformly random and independent bit $\mathsf{X}_{d}$:

[TABLE]

N.b., $\mathsf{Z}^{\prime}_{V}$ is different from $\mathsf{Z}_{V}$, namely, $\mathsf{Z}^{\prime}_{2}$ is obtained from $\mathsf{Z}_{2}$ by adding $\mathsf{X}_{d}$, and $\mathsf{Z}^{\prime}_{3}$ is obtained from $\mathsf{Z}_{3}$ by adding $\mathsf{X}_{d}$ and removing $\mathsf{X}_{a}$. It follows that $\mathsf{Z}^{\prime}_{V}$ is a PIN. By (26) and (31b), the constrained secrecy capacity for the modified source $\mathsf{Z}^{\prime}_{V}$ is

[TABLE]

The desired inequality is proved if we can show that

[TABLE]

To argue this, note that, if user $2$ reveals $\mathsf{F}_{2}^{\prime}=\mathsf{X}_{a}\oplus\mathsf{X}_{d}$ in public, then user $3$ can recover $\mathsf{X}_{a}$. Furthermore, $\mathsf{F}_{2}^{\prime}$ does not leak any information about $\mathsf{X}_{a}$, and so the source $\mathsf{Z}^{\prime}_{V}$ effectively emulates the source $\mathsf{Z}_{V}$. Consequently, any optimal discussion scheme $\mathsf{F}_{V}$ that achieves $C_{\mathop{\operator@font S}}(R)$ for $\mathsf{Z}_{V}$ can be used to achieve the same secret key rate but after an additional bit of discussion $\mathsf{F}_{2}^{\prime}$. This gives the desired inequality that establishes (33).

4 Multi-letter characterization

We start with a simple multi-letter characterization of the compressed secrecy capacity in terms of the MMI (26).

Proposition 2

For any $`a\geq 0$, we have

[TABLE]

where the supremum is over all valid compressed source $\tilde{\mathsf{Z}}_{V}$ satisfying the joint entropy limit (7).

Proof

This is because the compressed secrecy capacity is simply the secret key agreement on a compressed source. Hence, by (26), the MMI on the compressed source gives the compressed secrecy capacity.∎

The characterization in (34) is simpler than the formulation in (6) because it does not involve the random variables $\mathsf{F}$ and $\mathsf{K}$, nor the recoverability (4) and secrecy (5) constraints. Although such a multi-letter expression is not computable and therefore not accepted as a solution to the problem, it serves as an intermediate step that helps derive further results. More precisely, consider the bivariate case where $V=\{1,2\}$. Then, (34) becomes

[TABLE]

then $\tilde{C}_{\mathop{\operator@font S}}(`a)$ can be single-letterized by standard techniques as in [21] to $\tilde{C}_{\mathop{\operator@font S},1}(`a)$ defined in (21). The following gives a simple upper bound that is tight for sufficiently small $`a$.

Proposition 3

$\tilde{C}_{\mathop{\operator@font S},1}(`a)$* defined in (21) is continuous, non-decreasing and concave in $`a\geq 0$ with*

[TABLE]

Furthermore, equality holds iff $`a\leq J_{\mathop{\operator@font GK}}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$.

Proof

Monotonicity is obvious. Continuity and concavity can be shown by the usual time-sharing argument as in Proposition 1. (36) follows directly from the data processing inequality that $I(\mathsf{Z}^{\prime}_{1}\wedge\mathsf{Z}_{2})\leq I(\mathsf{Z}^{\prime}_{1}\wedge\mathsf{Z}_{1})$ under the Markov chain $\mathsf{Z}^{\prime}_{1}"-"\mathsf{Z}_{1}"-"\mathsf{Z}_{2}$ required in (21c). If $`a\leq J_{\mathop{\operator@font GK}}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$, then there exist a feasible solution $\mathsf{U}$ to (18) (a common function of $\mathsf{Z}_{1}$ and $\mathsf{Z}_{2}$) with $H(\mathsf{U})\geq`a$, and so the compressed sources $\tilde{\mathsf{Z}}_{1}$ and $\tilde{\mathsf{Z}}_{2}$ can be chosen as a function of $\mathsf{U}^{n}$ to achieve the equality for (36). Conversely, suppose $J_{\mathop{\operator@font GK}}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$ is finite and (36) is satisfied with equality. Then, in addition to $\mathsf{Z}^{\prime}_{1}"-"\mathsf{Z}_{1}"-"\mathsf{Z}_{2}$, we also have $\mathsf{Z}^{\prime}_{1}"-"\mathsf{Z}_{2}"-"\mathsf{Z}_{1}$, which implies by the double Markov property that, for the maximum common function $\mathsf{U}$ achieving $J_{\mathop{\operator@font GK}}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$ defined in (18),

[TABLE]

In other words, the optimal $\mathsf{Z}^{\prime}_{1}$ is a stochastic function of the maximum common function of $\mathsf{Z}_{1}$ and $\mathsf{Z}_{2}$, and so $`a=I(\mathsf{Z}^{\prime}_{1}\wedge\mathsf{Z}_{2})\leq J_{\mathop{\operator@font GK}}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$ as desired.∎

We will show that the above upper bound in (36) extends to the multi-user case (in Proposition 5). However, for $`a\geq J_{\mathop{\operator@font GK}}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$, the above upper bound is not tight even in the two-user case. To improve the upper bound, the following duality between $\tilde{C}_{\mathop{\operator@font S},1}$ and $C_{\mathop{\operator@font S},1}$ will be used and extended to the multi-user case (in Theorem 5.1).

Proposition 4

For $`a\geq J_{\mathop{\operator@font GK}}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$,

[TABLE]

Furthermore, the set of optimal solutions to the left (achieving $\tilde{C}_{\mathop{\operator@font S},1}(`a)$ defined in (21)) is the same as the set of optimal solutions to the right (achieving $C_{\mathop{\operator@font S},1}(R)$ in (17) with $R=`a-\tilde{C}_{\mathop{\operator@font S},1}(`a)$). It follows that the minimum admissible entropy (12) but with the entropy constraint on user $1$ instead is

[TABLE]

where $R_{\mathop{\operator@font S},1}$ and $J_{\mathop{\operator@font W},1}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$ are defined in (19) and (20) respectively.

Proof

Set $R=`a-\tilde{C}_{\mathop{\operator@font S},1}(`a)$. Consider first an optimal solution $\mathsf{Z}^{\prime}_{1}$ to $\tilde{C}_{\mathop{\operator@font S},1}(`a)$ and show that it is also an optimal solution to $C_{\mathop{\operator@font S},1}(R)$. By optimality,

[TABLE]

By the constraint (21b), $I(\mathsf{Z}^{\prime}_{1}\wedge\mathsf{Z}_{1})\leq`a$. It follows that the constraint (17b) holds, and so $\mathsf{Z}^{\prime}_{1}$ is a feasible solution to $C_{\mathop{\operator@font S},1}(R)$, i.e., we have $\geq$ for (37) that

[TABLE]

To show that $\mathsf{Z}^{\prime}_{1}$ is also optimal to $C_{\mathop{\operator@font S},1}(R)$, suppose to the contrary that there exists a strictly better solution $\mathsf{Z}^{\prime\prime}_{1}$ to $C_{\mathop{\operator@font S},1}(R)$, i.e., with

[TABLE]

It follows that

[TABLE]

The last equality means that the constraint (21b) is satisfied with equality. If to the contrary that the equality does not hold, setting $\mathsf{Z}^{\prime}_{1}$ to be $\mathsf{Z}^{\prime\prime}_{1}$ for some fraction $`l>0$ of time gives a better solution to $C_{\mathop{\operator@font S},1}(R)$, contradicting the optimality of $\mathsf{Z}^{\prime}_{1}$. The first inequality can also be argued similarly by the optimality of $\mathsf{Z}^{\prime}_{1}$. Now, we have

[TABLE]

where (a) is by the concavity of $\tilde{C}_{\mathop{\operator@font S},1}(`a)$; and (b) is by the upper bound $\tilde{C}_{\mathop{\operator@font S},1}(`a)\leq`a$ in (36). N.b., equality cannot hold simultaneously for (a) and (b) because, otherwise, we have $\frac{I(\mathsf{Z}^{\prime\prime}_{1}\wedge\mathsf{Z}_{2})}{I(\mathsf{Z}^{\prime\prime}_{1}\wedge\mathsf{Z}_{1})}=1$, which, together with (41) and (42), contradicts the result in Proposition 3 that $\tilde{C}_{\mathop{\operator@font S},1}(`a)<`a$ (with strict inequality) for $`a>J_{\mathop{\operator@font GK}}(\mathsf{Z}_{1}\wedge\mathsf{Z}_{2})$. Hence,

[TABLE]

which, together with (41) and (42), implies

[TABLE]

contradicting even the feasibility of $\mathsf{Z}^{\prime\prime}_{1}$ to $C_{\mathop{\operator@font S},1}(R)$, namely, the constraint (17b) with $\mathsf{Z}^{\prime}_{1}$ replaced with $\mathsf{Z}^{\prime\prime}_{1}$. This completes the proof of the optimality of $\mathsf{Z}^{\prime}_{1}$ to $C_{\mathop{\operator@font S},1}(R)$.

Next, consider showing that an optimal solution $\mathsf{Z}^{\prime}_{1}$ to $C_{\mathop{\operator@font S},1}(R)$ is also optimal to $\tilde{C}_{\mathop{\operator@font S},1}(`a)$. Then,

[TABLE]

where the first inequality is by (17b); the second equality is by the optimality of $\mathsf{Z}^{\prime}_{1}$; and the last inequality follows from (40). Hence, the constraint (21b) holds and so $\mathsf{Z}^{\prime}_{1}$ is a feasible solution for $\tilde{C}_{\mathop{\operator@font S},1}(`a)$. If to the contrary that we have a better solution $\mathsf{Z}^{\prime\prime}_{1}$ for $\tilde{C}_{\mathop{\operator@font S},1}(`a)$, then $\mathsf{Z}^{\prime\prime}_{1}$ can be shown to be a feasible solution for $C_{\mathop{\operator@font S},1}(R)$, contradicting the optimality of $\mathsf{Z}^{\prime}_{1}$.∎

5 Main Results

The following extends the single-letter upper bound (36) in Proposition 3 to the muli-user case.

Proposition 5

$\tilde{C}_{\mathop{\operator@font S}}(`a)\leq`a$* with equality if $`a\leq J_{\mathop{\operator@font GK}}(\mathsf{Z}_{V})$.*

Proof

The upper bound $\tilde{C}_{\mathop{\operator@font S}}(`a)\leq`a$ is because $n\tilde{C}_{\mathop{\operator@font S}}(`a)$ cannot exceed the unconstrained secrecy capacity for the compressed source $\tilde{\mathsf{Z}}_{V}$, which, by (22) and (7), is upper bounded by $H(\tilde{\mathsf{Z}}_{V})\leq n`1[`a+`d_{n}`2]$ for some $`d_{n}\to 0$ as $n\to`8$.

Next, to prove the equality condition is sufficient, suppose $`a\leq J_{\mathop{\operator@font GK}}(\mathsf{Z}_{V})$. Then, each user can compress their source directly to a common secret key at rate $`a$ without any public discussion. Hence, $\tilde{C}_{\mathop{\operator@font S}}(`a)=`a$ as desired.∎

N.b., unlike the two-user case in Proposition 3, the equality condition above in terms of the multivariate Gács–Körner common information is sufficient but not shown to be necessary. Nevertheless, necessity seems very plausible, as there seems to be no counter-example that suggests otherwise.

As in Proposition 4, a duality can be proved in the multi-user case, relating the compressed secret key agreement problem to the constrained secrecy key agreement problem.

Theorem 5.1

With $C_{\mathop{\operator@font S}}(R)$ and $R_{\mathop{\operator@font S}}$ defined in (9) and (12) respectively, we have

[TABLE]

for all $`a\geq 0$.

Proof

(43a) can be obtained from (43b) by setting $`a=`a_{\mathop{\operator@font S}}$ as follows:

[TABLE]

where (b) is given by (43b) with $`a=`a_{\mathop{\operator@font S}}$; while (a) and (c) follows directly from (11), (13) and monotonicity. It follows that the inequalities (a) and (b) hold with equality. In particular, equality for (a) means that $C_{\mathop{\operator@font S}}(R)=C_{\mathop{\operator@font S}}(`8)$ for $R\geq`a_{\mathop{\operator@font S}}-\tilde{C}_{\mathop{\operator@font S}}(`a_{\mathop{\operator@font S}})=`a_{\mathop{\operator@font S}}-C_{\mathop{\operator@font S}}(`8)$, implying (43a) as desired.

To show (43b), consider an optimal compressed secret key agreement scheme achieving $\tilde{C}_{\mathop{\operator@font S}}(`a)$ with an arbitrary entropy limit $`a$. It suffices to show that the discussion rate need not be larger than $`a-\tilde{C}_{\mathop{\operator@font S}}(`a)$. Let $\tilde{\mathsf{Z}}_{V}$ be the optimal compressed source and $\tilde{R}_{\mathop{\operator@font CO}}$ be the smallest rate of communication for omniscience of $\tilde{\mathsf{Z}}_{V}$, which is given by (23) with $\mathsf{Z}_{V}$ replaced by $\tilde{\mathsf{Z}}_{V}$. The discussion rate for the omniscience strategy is

[TABLE]

by (22). This simplifies to $`a-\tilde{C}_{\mathop{\operator@font S}}(`a)$ as desired in the limit $n\to`8$. N.b., since the omniscience strategy is non-interactive, the desired hold even if $C_{\mathop{\operator@font S}}$ and $R_{\mathop{\operator@font S}}$ are defined with non-interactive discussion.∎

While it is obvious from the above proof that a compressed secret key agreement scheme can be used as a constrained secret key agreement scheme, yielding one of the best lower bounds for $C_{\mathop{\operator@font S}}(R)$ in [10], the above result also means that a converse result on constrained secret key agreement can be applied to compressed secret key agreement. Upper bounds on $\tilde{C}_{\mathop{\operator@font S}}(`a)$ may be obtained from the upper bounds for $C_{\mathop{\operator@font S}}(R)$ such as those in [10]. It turns out that this approach can give better upper bounds which, surprisingly, is tight for the PIN model as mentioned in Section 3.3. This leads to the following exact single-letter characterization of $\tilde{C}_{\mathop{\operator@font S}}(`a)$.

Theorem 5.2

For the PIN model in Definition 3,

[TABLE]

Proof

(44a) follows easily from (44b) by setting the two terms in the minimization to be equal and solving for $`a$. To show (44b), note that, by (31b), we have

[TABLE]

because $C_{\mathop{\operator@font S}}(R)$ is non-decreasing and concave, and so it must be strictly non-decreasing before it reaches $C_{\mathop{\operator@font S}}(`8)=C_{\mathop{\operator@font S}}(`8)$. Now, by (43b),

[TABLE]

for any $`a\geq 0$ such that $\tilde{C}_{\mathop{\operator@font S}}(`a)<C_{\mathop{\operator@font S}}(`8)$, i.e., for $`a\leq`a_{S}$, and so $\tilde{C}_{\mathop{\operator@font S}}(`a)\leq\frac{`a}{\lvert V\rvert-1}$. This implies $\leq$ for (44b). The bound is achievable by the same achieving scheme in [10, Theorem 4.4] along the idea of decremental secrecy key agreement and the tree packing protocol in [39]. More precisely, every $(\lvert V\rvert-1)$ bits of edge variable forming a spanning tree are turned into a secret key bit by the tree packing protocol. This results in the factor of $(\lvert V\rvert-1)$ in (44), which corresponds to the number of edges in a spanning tree.∎

For the more general source model, the idea of decremental secret key agreement needs to be refined because there need not be any edge variables to remove. The following is a simple extension that leads to a single-letter lower bound on $\tilde{C}_{\mathop{\operator@font S}}(`a)$.

Theorem 5.3

A single-letter lower bound on $\tilde{C}_{\mathop{\operator@font S}}(`a)$ is

[TABLE]

for any random vector $(\mathsf{Q},\mathsf{Z}_{V}^{\prime})$ taking values from a finite set and satisfying

[TABLE]

Furthermore, it is admissible to have $\lvert Q\rvert\leq 3$.

Proof

By (46b), we have $\mathsf{Z}_{i}^{\prime}=`x_{i}(\mathsf{Z}_{i},\mathsf{Q})$ for some function $`x_{i}$. W.l.o.g., let $Q:=\{1,\dots,k\}$ for some integer $k>0$. Choose $\tilde{\mathsf{Z}}_{i}$ to be the following function of $\mathsf{Z}_{i}^{n}$:

[TABLE]

Basically, $\mathsf{Q}$ acts as a time-sharing random variable where $P_{\mathsf{Q}}(q)$ is the fraction of time the source $\mathsf{Z}_{i}$ is processed to $\mathsf{Z}_{i}^{(q)}:=`x_{i}(\mathsf{Z}_{i},q)$, for $1\leq q\leq k$. More precisely, we have $\frac{n_{q}-n_{q-1}}{n}$ converge to $P_{\mathsf{Q}}(q)$, and so

[TABLE]

Similarly,

[TABLE]

by (46c), satisfying the entropy limit (7). Hence, $\tilde{\mathsf{Z}}_{V}$ is a valid compressed source, the unconstrained capacity of which is $I(\mathsf{Z}_{V}^{\prime}|\mathsf{Q})$, leading to the desired lower bound (45).

The condition that $\lvert Q\rvert\leq 3$ is admissible follows from the usual argument by the well-known Eggleston–Carathéodory theorem. More precisely, let

[TABLE]

It can be seen that the conditions above are equivalent to (46a) and (46b) respectively, and so the set of feasible values to (46), namely

[TABLE]

is equal to the convex hull of $\mathscr{S}$. Since the dimension of $\mathscr{S}$ is at most $2$, the pair $(\tilde{C}_{\mathop{\operator@font S}}(\mathsf{Z}_{V}),`a)$ can be obtained as a convex combination of at most $3$ points in $\mathscr{S}$ as desired by the Eggleston–Carathéodory theorem.∎

The main results above can be illustrated as follows using the hypergraphical source in Example 1 given earlier. In particular, an exact single-letter characterization of $\tilde{C}_{\mathop{\operator@font S}}(`a)$ will be derived, even though such an exact characterization is not known for general hypergraphical sources.

Example 3

Consider the source defined in (15) in Example 1. It will be shown that (16a) and (16b) are satisfied with equality, which gives the desired single-letter characterization of $\tilde{C}_{\mathop{\operator@font S}}(`a)$.

It is easy to show that $J_{\mathop{\operator@font GK}}(\mathsf{Z}_{V})=1$ since $\mathsf{X}_{a}$ is the maximum common function of $\mathsf{Z}_{1}$, $\mathsf{Z}_{2}$ and $\mathsf{Z}_{3}$. Hence, the reverse inequality of (16a) follows from Proposition 5.

The reverse inequality for (16b) can be argued using the bound in Theorem 5.1 by $C_{\mathop{\operator@font S}}(R)$ and the characterization of $C_{\mathop{\operator@font S}}(R)$ in Example 2. More precisely, by (32), that the unconstrained secrecy capacity $C_{\mathop{\operator@font S}}(`8)=2$. Then, by (33), we have $C_{\mathop{\operator@font S}}^{-1}(`g)\leq`g-1$ for all $`g\leq C_{\mathop{\operator@font S}}(`8)=2$. Now, by (43b),

[TABLE]

and so $\tilde{C}_{\mathop{\operator@font S}}(`a)\leq\frac{1+`a}{2}$ for $\tilde{C}_{\mathop{\operator@font S}}(`a)\leq 2$. This completes the proof.

6 Conclusion and Extensions

Inspired by the idea of decremental secret key agreement and its application to the constrained secret key agreement problem, we have formulated a multiterminal secret key agreement problem with a more general source compression step that applies beyond the hypergraphical source model. This formulation allow us to separate and compare the issues of source compression and discussion rate constraint in secret key agreement. While a single-letter characterization of the compressed secrecy capacity and admissible entropy limit remains unknown, single-letter bounds have been derived and they are likely to be tight for the hypergraphical model, and possibly more general source models such as the finite linear source model [12]. For the PIN model, in particular, the bounds are tight, giving rise to a complete characterization of the capacity in Theorem 5.2. One way to improve the current converse results is to show whether the equality condition in Proposition 5 is necessary, that is, $\tilde{C}_{\mathop{\operator@font S}}(`a)<`a$ for $`a>J_{GK}(\mathsf{Z}_{V})$. By the duality in Theorem 5.1, the condition is necessary if one can show that $C_{\mathop{\operator@font S}}(0)=J_{\mathop{\operator@font GK}}(\mathsf{Z}_{V})$, i.e., (28) holds with equality. Such equality can be proved for hypergraphical as well as finite linear sources by extending the lamination techniques in [10]. It is hopeful that a complete solution can be given for the finite linear source model and the well-known jointly gaussian source model. The bounds (43) in the duality result may plausibly be tight for these special sources, in which case non-interactive discussion suffices to achieve the constrained secrecy capacity. The current achievability results may also be improved. In particular, for the two-user case with joint entropy constraint (35), the lower bound in (45) can be improved to $\tilde{C}_{\mathop{\operator@font S}}(`a)\geq\max I(\mathsf{Z}^{\prime}_{1}\wedge\mathsf{Z}^{\prime}_{2})$ where $I(\mathsf{Z}^{\prime}_{1}\wedge\mathsf{Z}_{1})+I(\mathsf{Z}^{\prime}_{2}\wedge\mathsf{Z}_{2})\leq`a$ and $\mathsf{Z}^{\prime}_{1}"-"\mathsf{Z}_{1}"-"\mathsf{Z}_{2}"-"\mathsf{Z}^{\prime}_{2}$. Whether this improvement is strict or is the best possible is not clear yet but an extension to the multi-user case seems possible. A related open problem is to characterize the $C_{\mathop{\operator@font S}}(R)$ in the two-user case with two-way non-interactive discussion. A simpler question is whether two-way non-interactive discussion can be strictly better than one-way discussion.

As pointed out before, by regarding the secrecy capacity as a measure of mutual information, an optimal source compression scheme translates to a dimension reduction technique potentially useful for machine learning. A closely related line of work is the study of the strong data processing inequality in [24, 2, 3], in particular, the ratio $s^{*}(\mathsf{Z}_{1};\mathsf{Z}_{2}):=\sup\frac{I(\mathsf{Z}^{\prime}_{1}\wedge\mathsf{Z}_{2})}{I(\mathsf{Z}_{1}^{\prime}\wedge\mathsf{Z}_{1})}$ where, as in (21), the supremum is taken over the choice of the conditional distribution $P_{\mathsf{Z}^{\prime}_{1}|\mathsf{Z}_{1},\mathsf{Z}_{2}}$ such that $\mathsf{Z}^{\prime}_{1}"-"\mathsf{Z}_{1}"-"\mathsf{Z}_{2}$ forms a Markov chain and $I(\mathsf{Z}^{\prime}_{1}\wedge\mathsf{Z})>0$. It is straightforward to show that $\sup_{`a\geq 0}\frac{\tilde{C}_{\mathop{\operator@font S}}(`a)}{`a}$ for the two-user case in (35) is upper bounded by $s^{*}(\mathsf{Z}_{1};\mathsf{Z}_{2})$ and $s^{*}(\mathsf{Z}_{2};\mathsf{Z}_{1})$. However, a sharper bound and a more precise mathematical connection may be possible, and the result may be extended to the multivariate case. Furthermore, the linearization considered in [29] may potentially be adopted to provide a single-letter lower bound on the compressed secrecy capacity. As in [2, 32], the problem may also be related to a notion of maximum correlation appropriately extended to the multivariate case.

Acknowledgment

The author would like to thank Dr. Ali Al-Bashabsheh for pointing out a mistake in an earlier proof and Qiaoqiao Zhou for the discussion of the two-user case. The author would also like to thank Prof. Shao-Lun Huang, Prof. Navin Kashyap, and Manuj Mukherjee for their valuable comments and pointers to related work.