Capacity Bounds on the Downlink of Symmetric, Multi-Relay, Single Receiver C-RAN Networks

TL;DR

This paper derives capacity bounds for symmetric multi-relay C-RAN downlink networks, using coding techniques and auxiliary variables, and characterizes capacity in key regimes.

Contribution

It introduces new capacity bounds for symmetric multi-relay C-RANs and demonstrates their tightness in certain operational regimes.

Findings

Bounds meet and characterize capacity in specific regimes

Marton's coding achieves the lower bound

Ozarow's technique provides the upper bound

Abstract

The downlink of symmetric Cloud Radio Access Networks (C-RANs) with multiple relays and a single receiver is studied. Lower and upper bounds are derived on the capacity. The lower bound is achieved by Marton's coding which facilitates dependence among the multiple-access channel inputs. The upper bound uses Ozarow's technique to augment the system with an auxiliary random variable. The bounds are studied over scalar Gaussian C-RANs and are shown to meet and characterize the capacity for interesting regimes of operation.

Equations

$$I(X_{\mathcal{S}})=\sum _{m\in \mathcal{S}}H(X_m)-H(X_{\mathcal{S}}).$$

$$H({\mathbb{V}}_m)\le nC,m=1,\dots ,M.$$

$$C_1=\dots =C_M=:C$$

$${\mathcal{X}}_1=\dots ={\mathcal{X}}_M=:\mathcal{X}$$

$$p_{Y|X_1\dots X_M}(y|x_1,\dots ,x_M)=p_{Y|X_1\dots X_M}(y|x_1^{\prime },\dots ,x_M^{\prime })$$

$$Y=\sum _{m=1}^M{X}_M+Z$$

$$\frac{1}{n}\sum _{i=1}^n\mathbb{E}(X_{m,i}^2)\le P_m,m=1,\dots ,M.$$

$$R_m,R_m^{\prime }\ge 0$$

$$R_m+R_m^{\prime }\le C_m.$$

$$R=\sum _{m=1}^M{R}_m.$$

$$(x_1^n(w_1,w_1^{\prime \star }),\dots ,x_M^n(w_M,w_M^{\prime \star }))\in {\mathcal{T}}_{ϵ}^n.$$

$$(x_1^n({\hat{w}}_1,{\hat{w}}_1^{\prime }),\dots ,x_M^n({\hat{w}}_M,{\hat{w}}_M^{\prime }),y^n)\in {\mathcal{T}}_{ϵ}^n.$$

$$\sum _{m\in \mathcal{S}}{R}_m^{\prime }\ge$$

$$\sum _{m\in \mathcal{S}}{R}_m+R_m^{\prime }\le$$

$${\mathcal{X}}_1=\dots ={\mathcal{X}}_M=\mathcal{X}$$

$$p_{X_{\mathcal{S}}}(x_1,\dots ,x_{|\mathcal{S}|})=p_{X_{{\mathcal{S}}^{\prime }}}$$

$$\forall (x_1,\dots ,x_{|\mathcal{S}|})\in {\mathcal{X}}^{|\mathcal{S}|}.$$

$$R\le MC-I(X_{\mathcal{M}})$$

$$R\le I(X_{\mathcal{M}};Y)$$

$$\displaystyle C^{(M)}\leq\max_{p(\mathbf{x})}\min_{\begin{subarray}{c}p(u|\mathbf{x}y)\\ =p(u|y)\end{subarray}}\max_{\begin{subarray}{c}p(q|\mathbf{x},u,y)\\ =p(q|\mathbf{x})\end{subarray}}\left\{\begin{array}[]{l}MC-(M\!-\!1)H(U|Q)+\sum_{m=1}^{M}H(U|X_{m}Q)-H(U|X_{\mathcal{M}})\\ \min_{\mathcal{S}\subseteq\mathcal{M}}|\mathcal{S}|C+I(X_{\bar{\mathcal{S}}};Y|X_{\mathcal{S}}Q)\end{array}\right\}.$$

$$nR$$

$$I(X_{\mathcal{M}}^n)\ge$$

$$=$$

$$nR\le$$

$$+\left[\sum _{m=1}^{M}H(U^n|X_m^n)\right]-H(U^n|X_{\mathcal{M}}^n).$$

$$R\le$$

$$\displaystyle\mathbf{K}_{M}(\rho)=\left[\begin{array}[]{cccc}P&\rho P&\ldots&\rho P\\ \rho P&P&\ldots&\rho P\\ \vdots&&\ddots&\vdots\\ \rho P&\ldots&\rho P&P\end{array}\right].$$

$$R\le MC-\frac{1}{2}\log \left(\frac{P^M}{\det ({\mathbf{K}}_M(\rho ))}\right)$$

$$R\le \frac{1}{2}\log (1+PM(1+(M-1)\rho )).$$

$$\det ({\mathbf{K}}_M(\rho ))=$$

$$=$$

$${\rho }^{(\ell )}=0\text{and}{R}^{(\ell )}=MC.$$

$$\frac{1}{2}\log (1+PM(1+(M-1)\rho ))$$

Full text

I Introduction

Cloud Radio Access Networks (C-RANs) are expected to be a part of future mobile network architectures. In C-RANs, information processing is done in a cloud-based central unit that is connected to remote radio heads (or relays) by rate-limited fronthaul links. C-RANs improve energy and bandwidth efficiency and reduce complexity of relays by facilitating centralized information processing and cooperative communication. We refer to[1, 2, 3] and the references therein for an overview of the challenges and coding techniques for C-RANs. Several coding schemes have been proposed in recent years for the downlink of C-RANs including message sharing [4], backhaul compression [5], hybrid schemes [6], and generalized data sharing using Marton’s coding [7, 8]. While none of these schemes are known to be optimal, [9] has proved an upper bound on the sum-rate of $2$-relay C-RANs with two users and numerically compared the performance of the aforementioned schemes with the upper bound.

We consider the downlink of a C-RAN with multiple relays and a single user. This network may be modeled by an $M$-relay diamond network where the broadcast component is modeled by rate-limited links and the multiaccess component is modeled by a memoryless multiple access channel (MAC), see Fig. 1. The capacity of this class of networks is not known in general, but lower and upper bounds were derived in [10, 11, 12] for $2$-relay networks. Moreover, the capacity was found for binary adder MACs [12], and for certain regimes of operation in Gaussian MACs [11, 12]. In this work, we derive lower and upper bounds for symmetric C-RANs with multiple relays and find the capacity in interesting regimes of operation for symmetric Gaussian C-RANs.

The rest of the paper is organized as follows. Section II introduces notation and the problem setup. In Section III, we propose a coding strategy based on Marton’s coding and discuss simplifications for symmetric networks. In Section IV, we generalize the bounding technique in [11, 12]. The case of Gaussian C-RANs is studied in Section V, where we compute lower and upper bounds and show that they meet in certain regimes of operation characterized in terms of power, number of users, and broadcast link capacities.

II Preliminaries and Problem Setup

II-A Notation

Random variables are denoted by uppercase letters, e.g. $X$, their realizations are denoted by lowercase letters, e.g. $x$, and their corresponding probabilities are denoted by $p_{X}(x)$ or $p(x)$. The probability mass function (pmf) describing $X$ is denoted by $p_{X}$. $\mathcal{T}^{n}_{\epsilon}(X)$ denotes the set of sequences that are $\epsilon$-typical with respect to $P_{X}$ [13, Page 25]. When $P_{X}$ is clear from the context we write $\mathcal{T}^{n}_{\epsilon}$. The entropy of $X$ is denoted by $H(X)$, the conditional entropy of $X$ given $Y$ is denoted by $H(X|Y)$ and the mutual information between $X$ and $Y$ is denoted by $I(X;Y)$. Similarly, differential entropies and conditional differential entropies are denoted by $h(X)$ and $h(X|Y)$.

Matrices are denoted by bold letters, e.g. $\mathbf{K}$. We denote the entry of matrix $\mathbf{K}$ in row $i$ and column $j$ by $\mathbf{K}_{ij}$. Sets are denoted by script letters, e.g., $\mathcal{S}$. The cartesian product of $\mathcal{S}_{1}$ and $\mathcal{S}_{2}$ is denoted by $\mathcal{S}_{1}\times\mathcal{S}_{2}$, and the $n$-fold Cartesian product of $\mathcal{S}$ is denoted by $\mathcal{S}^{n}$. The cardinality of $\mathcal{S}$ is denoted by $|\mathcal{S}|$.

Given the set $\mathcal{S}=\{s_{1},\ldots,s_{|\mathcal{S}|}\}$, $X_{\mathcal{S}}$ denotes the tuple $(X_{s_{1}},\ldots,X_{s_{|\mathcal{S}|}})$. The random string $X_{1},\ldots,X_{n}$ is denoted by $X^{n}$. $I(X_{\mathcal{S}})$ is defined as follows (see [14, Eqn (74)]):

[TABLE]

For example, when $\mathcal{S}=\{s_{1},s_{2}\}$, (1) becomes the mutual information $I(X_{s_{1}};X_{s_{2}})$. The conditional version of (1), $I(X_{\mathcal{S}}|U)$, is defined similarly by conditioning all terms in (1) on $U$. Note that $I(X_{\mathcal{S}}|U)$ is non-negative.

II-B Model

Consider the C-RAN in Fig. 1, where a source communicates a message $W$ with $nR$ bits to a sink with the help of $M$ relays. Let $\mathcal{M}=\{1,\ldots,M\}$ be the set of relays.

The source encodes $W$ into descriptions $\mathbb{V}_{1},\ldots,\mathbb{V}_{M}$ that are provided to relays $1,\ldots,M$, respectively. We focus mainly on symmetric networks where $\mathbb{V}_{m}$ satisfies

[TABLE]

Each relay $m$, $m=1,\ldots,M$, maps its description $\mathbb{V}_{m}$ into a sequence $X_{m}^{n}$ which is sent over a multiple access channel. The MAC is characterized by the input alphabets $\mathcal{X}_{1},\ldots,\mathcal{X}_{M}$, the output alphabet $\mathcal{Y}$, and the transitional probabilities $p(y|x_{1},\ldots,x_{M})$ for all $(x_{1},\ldots,x_{M},y)\in\mathcal{X}_{1}\times\ldots\times\mathcal{X}_{M}\times\mathcal{Y}$. From the received sequence $Y^{n}$, the sink decodes an estimate $\hat{W}$ of $W$.

A coding scheme consists of an encoder, $M$ relay mappings, and a decoder, and is said to achieve the rate $R$ if, by choosing $n$ sufficiently large, we can make the error probability $\Pr(\hat{W}\neq W)$ as small as desired. We are interested in characterizing the largest achievable rate $R$. We refer to the maximum rate as the capacity $C^{(M)}$ of the network.

In this work, we focus on symmetric networks:

Definition 1**.**

The network in Fig. 1 is symmetric if we have

[TABLE]

and

[TABLE]

for all $y\in\mathcal{Y}$, $(x_{1},\ldots,x_{M})\in\mathcal{X}^{M}$ and any of its permutations $(x^{\prime}_{1},\ldots,x^{\prime}_{M})$.

When the MAC is Gaussian, the input and output alphabets are the set of real numbers and the output is given by

[TABLE]

where $Z$ is Gaussian noise with zero mean and unit variance. We consider average block power constraints $P_{1},\ldots,P_{M}$:

[TABLE]

The Gaussian C-RAN is symmetric if $C_{m}=C$ and $P_{m}=P$ for all $m=1,\ldots,M$.

III A Lower Bound

We outline an achievable scheme based on Marton’s coding. We remark that this scheme can be improved for certain regimes of $C$ by using superposition coding (e.g., see [15, Theorem 2] and [11, Theorem 2]).

Fix the pmf $p(x_{1},\ldots,x_{M})$, $\epsilon>0$, and the auxiliary rates $R_{m},R^{\prime}_{m}$, $m=1,\ldots,M$, such that

[TABLE]

III-1 Codebook construction

Set

[TABLE]

For every $m=1,\ldots,M$, generate $2^{n(R_{m}+R^{\prime}_{m})}$ sequences $x_{m}^{n}(w_{m},w^{\prime}_{m})$, $w_{M}=1,\ldots,2^{nR_{m}}$, $w^{\prime}_{M}=1,\ldots,2^{nR^{\prime}_{m}}$, in an i.i.d manner according to $\prod_{\ell}P_{X_{\ell}}(x_{m,\ell})$, independently across $m=1,\dots,M$. For each bin index $(w_{1},\ldots,w_{M})$, pick a jointly typical sequence tuple

[TABLE]

III-2 Encoding

Represent message $w$ as a tuple $(w_{1},\ldots,w_{M})$, and send $(w_{m},w^{\prime\star}_{m})$ to relay $m$, $m=1,\ldots,M$.

III-3 Relay mapping at relay $m$, $m=1,\ldots,M$

Relay $m$ sends $X_{m}^{n}(w_{m},w^{\prime\star}_{m})$ over the MAC.

III-4 Decoding

Upon receiving $y^{n}$, the receiver looks for indices $\hat{w}_{1},\ldots,\hat{w}_{M}$ for which the following joint typicality test holds for some $\hat{w}^{\prime}_{1},\ldots,\hat{w}^{\prime}_{M}$:

[TABLE]

We show in Appendix A that the above scheme has a vanishing error probability as $n\to\infty$ if in addition to (8)–(10) we have

[TABLE]

One can use Fourier-Motzkin elimination to eliminate $R_{m},R^{\prime}_{m}$, $m=1,\ldots,M$, from (8)–(10), (13), (14), and characterize the set of achievable rates $R$. For symmetric networks (see Definition 1), we bypass the above step and proceed by choosing $p_{X_{\mathcal{M}}}$ to be “symmetric”. We say $p_{X_{\mathcal{M}}}$ is symmetric if

[TABLE]

and for all subsets $\mathcal{S},\mathcal{S}^{\prime}\subseteq\mathcal{M}$ with $|\mathcal{S}|=|\mathcal{S}^{\prime}|$ we have

[TABLE]

We simplify the problem defined by (8)–(10), (13), (14) for symmetric distributions and prove the following result in Appendix B.

Theorem 1**.**

For symmetric C-RAN downlinks, the rate $R$ is achievable if

[TABLE]

for some symmetric distribution $p_{X_{\mathcal{M}}}$.

IV An upper bound

Our upper bound is motivated by [12, 14, 16, 11].

Theorem 2**.**

The capacity $C^{(M)}$ is upper bounded as shown in (21) on the top of this page, where $Q-X_{\mathcal{M}}-Y-U$ forms a Markov chain. Moreover, the alphabet size of $Q$ may be chosen to satisfy $|\mathcal{Q}|\leq\prod_{i=1}^{M}|\mathcal{X}_{i}|+2^{M}-1$.

Remark 1**.**

For $M\!=\!2$, Theorem 2 reduces to [12, Theorem 3].

Proof Outline.

We start with the following $n$-letter upper bound (see Appendix C):

[TABLE]

For any sequence $U^{n}$, we have

[TABLE]

By substituting (24) into (22), we obtain

[TABLE]

We now choose $U_{i}$, $i=1,\ldots,n$, to be the output of a memoryless channel $p_{U|Y}(u_{i}|y_{i})$ with input $Y_{i}$. The auxiliary channel $p_{U|Y}(.|.)$ will be optimized later. With this choice, we single letterize (22) and (23) and obtain

[TABLE]

Details of the proof are presented in Appendix C.∎

V The Symmetric Gaussian C-RAN

First, we specialize Theorem 1 to the symmetric Gaussian C-RAN defined in (6)–(7) where $P_{m}=P$ for all $m=1,\ldots,M$. Choose $(X_{1},\ldots,X_{M})$ to be jointly Gaussian with the covariance matrix

[TABLE]

Remark 2**.**

Choosing $(X_{1},\ldots,X_{M})$ to be jointly Gaussian (and/or symmetric) is not necessarily optimal for (13)–(14), but it gives a lower bound on the capacity.

Theorem 3**.**

The rate $R$ is achievable if it satisfies the following constraints for some non-negative parameter $\rho$, $0\leq\rho\leq 1$:

[TABLE]

Remark 3**.**

One can recursively calculate $\det(\mathbf{K}_{M}(\rho))$:

[TABLE]

Let $R^{(\ell)}$ be the maximum achievable rate given by (33)–(34). The RHS of (33) is non-increasing in $\rho$ and the RHS of (34) is increasing in $\rho$. Therefore, we have the following two cases for the optimizing solution $\rho^{(\ell)}$:

• •

If $MC\leq\frac{1}{2}\log\left(1+PM\right)$ then

[TABLE]

• •

Otherwise, $\rho^{(\ell)}$ is the unique solution of $\rho$ in

[TABLE]

and we have

[TABLE]

We next specialize Theorem 2 to symmetric Gaussian C-RANs.

Theorem 4**.**

The rate $R$ is achievable only if there exists $\rho$, $0\leq\rho\leq 1$, such that the following inequalities hold for all $N\geq 0$:

[TABLE]

Proof of Theorem 4.

Set

[TABLE]

where $\{Z_{N,i}\}_{i=1}^{n}$ are identically distributed according to the normal distribution $\mathcal{N}(0,N)$ and are independent from each other and $X_{1}^{n},\ldots,X_{M}^{n}$. The variance $N$ is to be optimized.

In order to find a computable upper bound in (21), we need to lower bound $h(U|Q)$. Recall that $U$ is a noisy version of $Y$. We thus use the conditional entropy power inequality [17, Theorem 17.7.3]:

[TABLE]

Substituting (42) into the first constraint of (21), we obtain:

[TABLE]

Now consider the second term in (21) with $\mathcal{S}=\varnothing$:

[TABLE]

Note that the RHSs of (43), (44) are both concave in $p(x_{\mathcal{M}})$ and symmetric with respect to $X_{1},\ldots,X_{M}$. Therefore, a symmetric $p(x_{\mathcal{M}})$ maximizes them. Let $\mathbf{K}$ denote the covariance matrix of an optimal symmetric solution. We have

[TABLE]

Using the conditional version of the maximum entropy lemma [18], we can upper bound the differential entropies that appear with positive sign in (43) and (44) by their Gaussian counterparts, and $h(U|X_{\mathcal{M}})$ and $h(Y|X_{\mathcal{M}})$ can be written explicitly because the channels from $X_{\mathcal{M}}$ to $U$ and $Y$ are Gaussian. We obtain (39) and (40). Note that the RHSs of both bounds are increasing in $P$ and therefore there is no loss of generality in choosing $\mathbf{K}_{ii}=P$ (among $\mathbf{K}_{ii}\leq P$). ∎

The upper bound of Theorem 4 and the lower bound of Theorem 3 are plotted in Fig. 2 for $M=3$ and $P=1$, and they are compared with the lower bounds of message sharing [4], and compression [5]. One sees that our lower and upper bounds are close and they match over a wide range of $C$. Moreover, establishing partial cooperation among the relays through Marton’s coding offers significant gains. Fig. 3 plots the capacity bounds for $P=1$ and different values of $M$.

We next compare the lower and upper bounds. Let

[TABLE]

where

[TABLE]

Theorem 5**.**

The lower bound of Theorem 3 matches the upper bound of Theorem 4 if

[TABLE]

or if

[TABLE]

where $C_{C},C_{L},C_{U}$ are defined in (46)–(48).

Remark 4**.**

Theorem 5 recovers [12, Theorem 5] for $M=2$.

Remark 5**.**

For $C\leq C_{C}$, no cooperation is needed among the transmitters and the capacity is equal to $MC$.

Remark 6**.**

For $C$ large enough, full cooperation is possible through superposition coding and the capacity is

[TABLE]

where

[TABLE]

The rate (52) is not achievable by Theorem 3 except when $C\to\infty$. This rate is achievable by message sharing.

Proof of Theorem 5.

To find regimes of $P$ and $C$ for which the lower and upper bounds match, we mimic the analysis in [12, Appendix F]. Consider the lower bound in Theorem 3, and in particular its maximum achievable rate $R^{(\ell)}$ which is attained by $\rho^{(\ell)}$, see (36)–(38). If (50) holds, we have $\rho^{(\ell)}=0$, $R^{(\ell)}=MC$, and thus the cut-set bound is achieved. Otherwise, we proceed as follows.

Consider (39). Since $R\geq R^{(\ell)}$, and using the definition of $R^{(\ell)}$ in (38), we can further upper bound (39) and obtain

[TABLE]

Call the RHS of (54) $g_{1}(\rho)$ and the RHS of (40) $g_{2}(\rho)$. Fix $N$ as a function of $\rho^{(\ell)}$ such that

[TABLE]

where $I^{\left(G,\rho^{(\ell)}\right)}(X_{\mathcal{M}}|U)$ is $I(X_{\mathcal{M}}|U)$ evaluated for a fully symmetric Gaussian distribution with correlation factor $\rho^{(\ell)}$. One can verify that the following choice of $N$ satisfies (55):

[TABLE]

The right inequality in (51) ensures $N\geq 0$.

With this choice of $N$, $g_{1}(\rho)$ is exactly equal to

[TABLE]

at $\rho=\rho^{(\ell)}$. Note that $\rho^{(\ell)}$ is defined in (37), and thus $g_{1}(\rho)$ crosses $g_{2}(\rho)$ at $\rho^{(\ell)}$. Since $g_{2}(\rho)$ is increasing in $\rho$, the maximum admissible rate by (40) and (54) matches $R^{(\ell)}$ if $g_{1}(\rho)$ is non-increasing for $\rho\geq\rho^{(\ell)}$. This is ensured by the left inequality in (51). ∎

VI Concluding Remarks

We studied the downlink of symmetric C-RANs with multiple relays and a single receiver, and established lower and upper bounds on the capacity. The lower bound uses Marton’s coding to establish partial cooperation among the relays and improves on schemes that are based on message sharing and compression for scalar Gaussian C-RANs (see Fig. 2). The upper bound generalizes the bounding techniques of [11, 12]. When specialized to symmetric Gaussian C-RANs, the lower and upper bounds meet over a wide range of $C$ and this range gets large as $M$ and/or $P$ get large.

Appendix A Analysis of the Achievable Scheme

The scheme fails only if one of the following events occur:

• •

(11) does not hold for any index tuple $(w_{1}^{\prime\star},\ldots,w_{M}^{\prime\star})$. This event has a vanishing error probability as $n\to\infty$ [13, Lemma 8.2] if we have (13).

• •

(12) does not hold for the original indices $(w_{1},w^{\prime\star}_{1},\ldots,w_{M},w^{\prime\star}_{M})$. This event has a vanishing error probability as $n\to\infty$ by (11) and the Law of Large Numbers.

• •

(12) holds for indices $(\tilde{w}_{1},\tilde{w}^{\prime}_{1},\ldots,\tilde{w}_{M},\tilde{w}^{\prime}_{M})$ where $(\tilde{w}_{1},\ldots,\tilde{w}_{M})\neq(w_{1},\ldots,w_{M})$. We show that this event has a vanishing error probability as $n\to\infty$ if we have (14). Since the codebook is symmetric with respect to all messages, we assume without loss of generality that $w_{1}=\ldots=w_{M}=1$ and $w_{1}^{\prime}=\ldots=w_{M}^{\prime}=1$. Fix the sets $\mathcal{S}_{1},\mathcal{S}_{2},\mathcal{S}_{3}\subset\mathcal{M}$ such that $\mathcal{S}_{1}\cup\mathcal{S}_{2}\cup\mathcal{S}_{3}=\mathcal{M}$. Consider the case where

[TABLE]

We denote (60) by $W(\mathcal{S}_{1}^{3})$. We have

[TABLE]

where $\tilde{X}_{\mathcal{S}_{2}}^{n}$ denotes $\{X_{m}^{n}(\tilde{w}_{m},\tilde{w}_{m}^{\prime})\}_{m\in\mathcal{S}_{2}}$ and $\hat{X}_{\mathcal{S}_{3}}^{n}$ denotes $\{X_{m}^{n}(\tilde{w}_{m},\tilde{w}_{m}^{\prime})\}_{m\in\mathcal{S}_{3}}$. In step $(a)$, we use that (i) $(X_{\mathcal{S}_{1}}^{n}Y^{n})$ is “almost independent” of $(\tilde{X}_{\mathcal{S}_{2}}^{n},\hat{X}_{\mathcal{S}_{3}}^{n})$ and (ii) the random sequences $X_{m}^{n}(\tilde{w}_{m},\tilde{w}_{m}^{\prime}),\ {m\in\mathcal{S}_{2}}$, and $X_{m}^{n}(\tilde{w}_{m},\tilde{w}_{m}^{\prime}),\ m\in\mathcal{S}_{3}$, are mutually “almost independent”. Note that we use the term almost independent, rather than independent, because we have assumed $w_{1}=\ldots=w_{M}=1$ and $w_{1}^{\prime}=\ldots=w_{M}^{\prime}=1$; i.e., we implicitly have a conditional probability and conditioned on $w_{1}^{\prime}=\ldots=w_{M}^{\prime}=1$, (i)-(iii) may not hold if we insist on exact independence. This has been dealt with in [19, 20, 21]. Following similar arguments, one can show that (i) and (iii) hold with “almost independence”. The probability of the considered error event is thus arbitrarily small for large enough $n$ if

[TABLE]

Inequality (62) is satisfied by (14) when we choose $\bar{S}=\mathcal{S}_{1}$. Note that the inequalities with $\mathcal{S}_{2}\neq\varnothing$ are redundant.

Appendix B Simplification for Symmetric Networks with Symmetric Distributions

Choose $R_{m}=\tilde{R}$ and $R_{m}^{\prime}=\tilde{R}^{\prime}$ for all $1,\ldots,M$. The problem defined by (9), (10), (13), (14) simplifies using the definition in (16):

[TABLE]

We prove that the tightest inequality in (66) and (67) is given by $\mathcal{S}=\mathcal{M}$. Eliminating $\tilde{R}^{\prime}$ from the remaining inequalities concludes the proof.

Let $\mathcal{S}$ be any subset of $\mathcal{M}$ and $s_{0}$ be an element of $\mathcal{M}$ such that $s_{0}\notin\mathcal{S}$. This is possible if $\mathcal{S}\neq\mathcal{M}$. Define $\mathcal{T}=\mathcal{M}\backslash\{\mathcal{S}\cup s_{0}\}$. We show

[TABLE]

and

[TABLE]

The following equalities come in handy in the proof:

[TABLE]

Suppose $\mathcal{S}=\{s_{1},\ldots,s_{|\mathcal{S}|}\}$ and $\mathcal{S}^{\prime}=\{s_{0},s_{1},\ldots,s_{|\mathcal{S}|-1}\}$. We have

[TABLE]

where $(a)$ and $(c)$ hold by (16) and $(b)$ is by (71), written for $\mathcal{S}^{\prime}$.

Similarly, we have

[TABLE]

where $(a)$ and $(b)$ are by (70), $(c)$ follows from (16) and $(d)$ follows from (16) and the symmetry of the channel in (5).

Appendix C Proof of Theorem 2

We first prove the multi-letter bound in (22) using Fano’s inequality and the data processing inequality. For any $\epsilon>0$ we have

[TABLE]

where $(a)$ is by Fano’s inequality and $(b)$ is by the data processing inequality as follows:

[TABLE]

Similarly, for any subset $\mathcal{S}\subset\mathcal{M}$ and $\epsilon>0$, we have

[TABLE]

where $(a)$ is because $W-\mathbb{V}_{\mathcal{S}}-X^{n}_{\mathcal{S}}$ forms a Markov chain and $(b)$ is because conditioning does not increase entropy and because $W\mathbb{V}_{\mathcal{M}}-X^{n}_{\mathcal{M}}-Y$ forms a Markov chain.

Next, we upper bound (17) using (25) and single-letterize (25) as follows:

[TABLE]

where $T$ is a uniform random variable on the set $\{1,\ldots,n\}$ independent from everything in the system, $Q$ is defined by $(U^{T-1}T)$ and $X_{1},\ldots,X_{M}$, $Y$, and $U$ are defined by $X_{1,T},\ldots,X_{M,T}$, $Y_{T}$, and $U_{T}$, respectively. Note that $U-Y-X_{1}X_{2}-Q$ forms a Markov chain.

Finally, we expand (23) as follows.

[TABLE]

where $(a)$ is because $Y_{i}-X_{\mathcal{S}}^{n}Y^{i-1}-U^{i-1}$ forms a Markov chain, $(b)$ is because $Y_{i}-X_{\mathcal{M},i}-U^{i-1}$ forms a Markov chain, and $(c)$ is by a standard time sharing argument. The cardinality of $\mathcal{Q}$ is bounded using the Fenchel-Eggleston-Carathéodory theorem [13, Appendix A] (see also [22, Appendix B]).