Strong Coordination over Noisy Channels: Is Separation Sufficient?

TL;DR

This paper investigates how two agents can achieve strong coordination over noisy channels, proposing two novel coding schemes and analyzing their effectiveness in terms of coordination capacity.

Contribution

It introduces two new schemes for noisy strong coordination and derives inner bounds for the coordination capacity region, comparing their performance.

Findings

Joint scheme reduces local randomness needed at agent Y.

Separate scheme extracts randomness after decoding.

Joint scheme can outperform separate scheme in certain scenarios.

Abstract

We study the problem of strong coordination of actions of two agents $X$ and $Y$ that communicate over a noisy communication channel such that the actions follow a given joint probability distribution. We propose two novel schemes for this noisy strong coordination problem, and derive inner bounds for the underlying strong coordination capacity region. The first scheme is a joint coordination-channel coding scheme that utilizes the randomness provided by the communication channel to reduce the local randomness required in generating the action sequence at agent $Y$. The second scheme exploits separate coordination and channel coding where local randomness is extracted from the channel after decoding. Finally, we present an example in which the joint scheme is able to outperform the separate scheme in terms of coordination rate.

Figures

Equations

$$\Vert {\hat{P}}_{X^n{Y}^n}-P_{XY}^{\otimes n}{\Vert }_{TV}<ϵ.$$

$$\mathsf{C}\triangleq (\mathcal{A},\mathcal{C})=\left\{a_{ijk}^n,c_{ij}^n:\begin{array}{c}i\in \{1,\dots ,2^{n{R}_c}\}\\ j\in \{1,\dots ,2^{n{R}_o}\}\\ k\in \{1,\dots ,2^{n{R}_a}\}\end{array}\right\},$$

$$\mathring{P}_{X^{n}Y^{n}IJK}(x^{n},y^{n},i,j,k)\triangleq\frac{P_{X|AC}^{\otimes n}(x^{n}|a_{ijk}^{n},c^{n}_{ij})}{2^{n(R_{c}+R_{o}+R_{a})}}\\ \times\Big{(}\sum_{b^{n},\hat{i}}P_{B|A}^{\otimes n}(b^{n}|a_{ijk}^{n})\mathsf{P}_{\hat{I}|B^{n}J}(\hat{i}|b^{n},j)P_{Y|BC}^{\otimes n}(y^{n}|b^{n},c^{n}_{\hat{i}j})\Big{)},$$

$$\check{P}_{X^{n}Y^{n}IJK}(x^{n},y^{n},i,j,k)=\frac{P_{X|AC}^{\otimes n}(x^{n}|a_{ijk}^{n},c^{n}_{ij})}{2^{n(R_{c}+R_{o}+R_{a})}}\\ \times\Big{(}\sum_{b^{n}}P_{B|A}^{\otimes n}(b^{n}|a_{ijk}^{n})P_{Y|BC}^{\otimes n}(y^{n}|b^{n},c^{n}_{ij})\Big{)}.$$

$$R_a+R_o+R_c$$

$$R_o+R_c$$

$$\begin{split}&\mathbb{E}_{\mathsf{C}}\big{[}\mathbb{D}(\check{P}_{X^{n}Y^{n}}||P^{\otimes n}_{XY})\big{]}\\ &=\mathbb{E}_{\mathsf{C}}\Bigg{[}\sum_{x^{n},y^{n}}\Big{(}\sum_{i,j,k}\dfrac{P(x^{n}|A^{n}_{ijk},C^{n}_{ij})P(y^{n}|A^{n}_{ijk},C^{n}_{ij})}{2^{nR}}\Big{)}\log\Bigg{(}\sum_{i^{\prime},j^{\prime},k^{\prime}}\dfrac{P(x^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},C^{n}_{i^{\prime}j^{\prime}})P(y^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},C^{n}_{i^{\prime}j^{\prime}})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}\Bigg{)}\Bigg{]}\\ &\stackrel{{\scriptstyle(a)}}{{=}}\!\sum_{x^{n},y^{n}}\!\mathbb{E}_{A^{n}_{ijk}C^{n}_{ij}}\Bigg{[}\!\Big{(}\sum_{i,j,k}\dfrac{P(x^{n}|A^{n}_{ijk},C^{n}_{ij})P(y^{n}|A^{n}_{ijk},C^{n}_{ij})}{2^{nR}}\Big{)}\mathbb{E}_{\mathrm{rest}}\Big{[}\!\log\Big{(}\!\!\sum_{i^{\prime},j^{\prime},k^{\prime}}\!\!\dfrac{P(x^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},\!C^{n}_{i^{\prime}j^{\prime}})P(y^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},\!C^{n}_{i^{\prime}j^{\prime}})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}\Big{)}\!\Big{|}A^{n}_{ijk}C^{n}_{ij}\Big{]}\!\Bigg{]}\\ &\stackrel{{\scriptstyle(b)}}{{\leq}}\!\sum_{x^{n},y^{n}}\!\mathbb{E}_{A^{n}_{ijk}C^{n}_{ij}}\Bigg{[}\!\Big{(}\sum_{i,j,k}\dfrac{P(x^{n}|A^{n}_{ijk},C^{n}_{ij})P(y^{n}|A^{n}_{ijk},C^{n}_{ij})}{2^{nR}}\Big{)}\log\Big{(}\mathbb{E}_{\mathrm{rest}}\Big{[}\!\!\sum_{i^{\prime},j^{\prime},k^{\prime}}\!\!\dfrac{P(x^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},\!C^{n}_{i^{\prime}j^{\prime}})P(y^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},\!C^{n}_{i^{\prime}j^{\prime}})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}\!\Big{|}A^{n}_{ijk}C^{n}_{ij}\Big{]}\!\Big{)}\!\Bigg{]}\\ &\stackrel{{\scriptstyle(c)}}{{=}}\!\sum_{x^{n},y^{n}}\sum_{a^{n}_{ijk},c^{n}_{ij}}\sum_{i,j,k}\dfrac{P(x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij})}{2^{nR}}\log\Bigg{(}\sum_{\begin{subarray}{c}i^{\prime},j^{\prime},k^{\prime}:\\ (i^{\prime},j^{\prime},k^{\prime})=(i,j,k)\end{subarray}}\mathbb{E}_{A^{n}_{ijk}C^{n}_{ij}}\Big{[}\dfrac{P(x^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},C^{n}_{i^{\prime}j^{\prime}})P(y^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},C^{n}_{i^{\prime}j^{\prime}})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}\Big{|}A^{n}_{ijk}C^{n}_{ij}\Big{]}\\ &\hskip 180.67499pt+\sum_{\begin{subarray}{c}i^{\prime},j^{\prime},k^{\prime}:\\ (i^{\prime},j^{\prime})=(i,j),(k^{\prime}\neq k)\end{subarray}}\mathbb{E}_{A^{n}_{ijk}C^{n}_{ij}}\Big{[}\dfrac{P(x^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},C^{n}_{i^{\prime}j^{\prime}})P(y^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},C^{n}_{i^{\prime}j^{\prime}})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}\Big{|}A^{n}_{ijk}C^{n}_{ij}\Big{]}\\ &\hskip 180.67499pt+\sum_{\begin{subarray}{c}i^{\prime},j^{\prime},k^{\prime\prime}:\\ (i^{\prime},j^{\prime})\neq(i,j)\end{subarray}}\mathbb{E}_{A^{n}_{ijk}C^{n}_{ij}}\Big{[}\dfrac{P(x^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},C^{n}_{i^{\prime}j^{\prime}})P(y^{n}|A^{n}_{i^{\prime}j^{\prime}k^{\prime}},C^{n}_{i^{\prime}j^{\prime}})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}\Big{|}A^{n}_{ijk}C^{n}_{ij}\Big{]}\Bigg{)}\\ &\stackrel{{\scriptstyle(d)}}{{=}}\sum_{x^{n},y^{n}}\sum_{a^{n}_{ijk},c^{n}_{ij}}\sum_{i,j,k}\dfrac{P(x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij})}{2^{nR}}\log\Bigg{(}\dfrac{P(x^{n},y^{n}|a^{n}_{ijk},c^{n}_{ij})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}+\sum_{\begin{subarray}{c}i^{\prime},j^{\prime},k^{\prime}:\\ (i^{\prime},j^{\prime})=(i,j),(k^{\prime}\neq k)\end{subarray}}\dfrac{P(x^{n},y^{n}|c^{n}_{ij})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}\\ &\hskip 187.90244pt+\sum_{\begin{subarray}{c}i^{\prime},j^{\prime},k^{\prime}:\\ (i^{\prime},j^{\prime})\neq(i,j)\end{subarray}}\dfrac{P^{\otimes n}_{XY}(x^{n},y^{n})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}\Bigg{)}\\ &\stackrel{{\scriptstyle(e)}}{{\leq}}\sum_{x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij}}P(x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij})\log\Bigg{(}\dfrac{P(x^{n},y^{n}|a^{n}_{ijk},c^{n}_{ij})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}+(2^{R_{a}})\dfrac{P(x^{n},y^{n}|c^{n}_{ij})}{2^{nR}P^{\otimes n}_{XY}(x^{n},y^{n})}+1\Bigg{)}\\ &\stackrel{{\scriptstyle(f)}}{{\leq}}\Bigg{[}\sum_{\begin{subarray}{c}x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij}:\\ (x^{n},y^{n},a^{n},c^{n})\in{\cal T}_{\epsilon}^{n}(p_{XYAC})\end{subarray}}P(x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij})\log\Bigg{(}\dfrac{2^{-nH(XY|AC)(1-\epsilon)}}{2^{nR}2^{-nH(XY)(1+\epsilon)}}+\dfrac{2^{-nH(XY|C)(1-\epsilon)}}{2^{n(R_{o}+R_{c})}2^{-nH(XY)(1+\epsilon)}}+1\Bigg{)}\Bigg{]}\\ &\hskip 173.44756pt+\mathbb{P}\big{(}(x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij})\notin{\cal T}_{\epsilon}^{n}(p_{XYAC})\big{)}\log(2\mu_{XY}^{-n}+1)\\ &\stackrel{{\scriptstyle(g)}}{{\leq}}\Bigg{[}\sum_{\begin{subarray}{c}x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij}:\\ (x^{n},y^{n},a^{n},c^{n})\in{\cal T}_{\epsilon}^{n}(p_{XYAC})\end{subarray}}P(x^{n},y^{n},a^{n}_{ijk},c^{n}_{ij})\log\Bigg{(}\dfrac{2^{n(I(XY;AC)+\delta(\epsilon))}}{2^{nR}}+\dfrac{2^{n(I(XY;C)+\delta(\epsilon))}}{2^{n(R_{o}+R_{c})}}+1\Bigg{)}\Bigg{]}\\ &\hskip 173.44756pt+\big{(}2{\cal|X||Y||A||C|}e^{-n\epsilon^{2}\mu_{XYAC}}\big{)}\log(2\mu_{XY}^{-n}+1)\xrightarrow{n\rightarrow\infty}0.\end{split}$$

$$R_a+R_o+R_c>I(XY;AC),R_o+R_c>I(XY;C),$$

$$\displaystyle\mathbb{E}_{\mathsf{C}}\big{[}||\check{P}_{X^{n}Y^{n}}$$

$$\displaystyle\leq\sqrt{2\mathbb{E}_{\mathsf{C}}\big{[}\mathbb{D}(\check{P}_{X^{n}Y^{n}}||P^{\otimes n}_{XY})\big{]}}\mathop{\longrightarrow}^{n\rightarrow\infty}0.$$

$$||{\check{P}}_{X^n{Y}^n}-P_{XY}^{\otimes n}|{|}_{TV}<ϵ.$$

$$\displaystyle\mathbb{E}_{\mathsf{C}}\big{[}\mathbb{P}[\hat{I}\neq I]\big{]}$$

$$\displaystyle=\sum_{\mathsf{C}}P_{\mathsf{C}}(\mathsf{c})\sum_{i,j,k}\frac{1}{2^{nR}}\mathbb{P}\Big{[}\hat{I}\neq I\Big{|}\begin{subarray}{c}I=i\\ J=j\\ K=k\end{subarray}\Big{]}$$

$$\displaystyle=\sum_{i,j,k}\frac{1}{2^{nR}}\sum_{\mathsf{C}}P_{\mathsf{C}}(\mathsf{c})\mathbb{P}\Big{[}\hat{I}\neq I\Big{|}\begin{subarray}{c}I=i\\ J=j\\ K=k\end{subarray}\Big{]}$$

$$\displaystyle\stackrel{{\scriptstyle(a)}}{{=}}\mathbb{P}\Big{[}\hat{I}\neq I\Big{|}\begin{subarray}{c}I=1\\ J=1\\ K=1\end{subarray}\Big{]},$$

$$\displaystyle\mathbb{E}_{\mathsf{C}}[\mathds{1}\big{(}(A_{111}^{n},B^{n},C^{n}_{11})\in{\cal T}_{\epsilon}^{n}(P_{ABC})\big{)}]\xrightarrow{n\rightarrow\infty}1.$$

$${\hat{S}}_{j,b^n,\mathsf{c}}\triangleq \{i:(b^n,c_{ij}^n)\in {\mathcal{T}}_{ϵ}^n(P_{BC})\}.$$

$$\displaystyle\mathbb{E}_{\mathsf{C}}\Big{[}\mathbb{P}$$

$$\displaystyle=\sum_{a^{n},b^{n},c^{n}}\Big{(}P_{C}^{\otimes n}(c^{n})P_{A|C}^{\otimes n}(a^{n}|c^{n})P_{B|A}^{\otimes n}(b^{n}|a^{n})$$

$$\displaystyle\hskip 72.26999pt\times\mathds{1}\big{(}(c^{n},b^{n})\in{\cal T}_{\epsilon}^{n}(P_{BC})\big{)}\Big{)}$$

$$\displaystyle\stackrel{{\scriptstyle}}{{=}}\sum_{b^{n},c^{n}}P_{BC}^{\otimes n}(b^{n},c^{n})\mathds{1}\big{(}(b^{n}\!,c^{n})\!\in\!{\cal T}_{\epsilon}^{n}(P_{BC})\big{)}$$

$$\stackrel{(a)}{\ge }1-\delta (ϵ)\stackrel{n\to \infty }{\to }1,$$

$$\displaystyle\mathbb{E}_{\mathsf{C}}\mathbb{P}\Big{[}\hat{S}_{J,B^{n},\mathsf{C}}\cap\{2,\dots,2^{nR_{c}}\}\!=\!\emptyset\Big{|}\begin{subarray}{c}I=1\\ J=1\\ K=1\end{subarray}\Big{]}$$

$$\displaystyle\qquad=1-\sum_{i^{\prime}\neq 1}\mathbb{E}_{\mathsf{C}}\mathbb{P}\Big{[}i^{\prime}\in\hat{S}_{J,B^{n},\mathsf{C}}\Big{|}\begin{subarray}{c}I=1\\ J=1\\ K=1\end{subarray}\Big{]}$$

$$=1-\sum _{i^{\prime }\ne 1}\mathbb{P}[(C_{i^{\prime }1}^n,B^n)\in {\mathcal{T}}_{ϵ}^n(P_{BC})]$$

$$\stackrel{(a)}{\ge }1-\sum _{i^{\prime }\ne 1}{2}^{-n(I(B;C)-\delta (ϵ))}$$

$$=1-(2^{n{R}_c}-1)2^{-n(I(B;C)-\delta (ϵ))}$$

$$=1-2^{-n(I(B;C)-R_c-\delta (ϵ))}+2^{-nI(B;C)}$$

$$\stackrel{(b)}{\ge }1-\delta (ϵ)\stackrel{n\to \infty }{\to }1,$$

$$\displaystyle\mathbb{E}_{\mathsf{C}}\big{[}\mathbb{P}[\hat{I}\neq I]\big{]}=\mathbb{E}_{\mathsf{C}}\mathbb{P}\Big{[}\hat{I}\neq I\Big{|}\begin{subarray}{c}I=1\\ J=1\\ K=1\end{subarray}\Big{]}$$

$$\displaystyle\qquad\leq\left(\mathbb{E}_{\mathsf{C}}\mathbb{P}\Big{[}I\notin\hat{S}_{J,B^{n},\mathsf{C}}\Big{|}\begin{subarray}{c}I=1\\ J=1\\ {K=1}\end{subarray}\Big{]}\right.$$

$$\displaystyle\qquad\qquad\qquad+\left.\mathbb{E}_{\mathsf{C}}\mathbb{P}\Big{[}\hat{S}_{J,B^{n},\mathsf{C}}\cap\{2,\dots,2^{nR_{c}}\}\!\neq\!\emptyset\Big{|}\begin{subarray}{c}I=1\\ J=1\\ K=1\end{subarray}\Big{]}\right)$$

$$\stackrel{n\to \infty }{\to }0$$

$$\displaystyle\lim\limits_{n\rightarrow\infty}\mathbb{E}_{\mathsf{C}}\big{[}\lVert\check{P}_{X^{n}Y^{n}IJK}-\mathring{P}_{X^{n}Y^{n}IJK}\rVert_{{\scriptscriptstyle TV}}\big{]}=0.$$

$$R_a+R_c$$

$$R_c$$

$$\displaystyle\mathbb{E}_{\mathsf{C}}\big{[}||\check{P}_{X^{n}J}$$

$$\displaystyle\leq\sqrt{2\mathbb{E}_{\mathsf{C}}\big{[}\mathbb{D}(\check{P}_{X^{n}J}||P^{\otimes n}_{X}P_{J})\big{]}}\mathop{\longrightarrow}^{n\rightarrow\infty}0.$$

$$||{\check{P}}_{X^{n}J}-P_X^{\otimes n}{P}_J|{|}_{TV}<ϵ.$$

$${\hat{P}}_{X^n{Y}^{n}IJK}\triangleq P_X^{\otimes n}{P}_J{\mathring{P}}_{IK|X^{n}J}{\mathring{P}}_{Y^n|IJK}.$$

Full text

I Introduction

The problem of communication-based coordination of multi-agent systems arises in numerous applications including mobile robotic networks, smart traffic control, and distributed computing such as distributed games and grid computing [1]. Several theoretical and applied studies on multi-agent coordination have targeted questions on how agents exchange information and how their actions can be correlated to achieve a desired overall behavior. Two types of coordination have been addressed in the literature – empirical coordination where the histogram of induced actions is required to be close to a prescribed target distribution, and strong coordination, where the induced sequence of joint actions of all the agents is required to be statistically close (i.e., nearly indistinguishable) from a chosen target probability mass function (pmf).

Recently, the capacity regions of several empirical and strong coordination network problems have been established [2, 1, 3, 4, 5, 6]. Bounds for the capacity region for the point-to-point case were obtained in [7] under the assumption that the nodes communicate in a bidirectional fashion in order to achieve coordination. A similar framework was adopted and improved in [8]. In [4, 9, 6], the authors addressed inner and outer bounds for the capacity region of a three-terminal network in the presence of a relay. The work of [4] was later extended in [10, 5] to derive a precise characterization of the strong coordination region for multi-hop networks. Starkly, the majority of the recent works on coordination have considered noise-free communication channels with the exception of two works: joint empirical coordination of the channel inputs/outputs of a noisy communication channel with source and reproduction sequences is considered in [11], and in [12], the notion of strong coordination is used to simulate a discrete memoryless channel via another channel.

In this work, we consider the point-to-point coordination setup illustrated in Fig. 1, where in contrast to [11] only source and reproduction sequences at two different nodes ($X$ and $Y$) are coordinated by means of a suitable communication scheme over a discrete memoryless channel (DMC).

Specifically, we propose two different novel achievable coding schemes for this noisy coordination scenario, and derive inner bounds to the underlying strong capacity region. The first scheme is a joint coordination channel coding scheme that utilizes randomness provided by the DMC to reduce the local randomness required in generating the action sequence at Node $Y$ (see Fig. 1). The second scheme exploits separate coordination and channel coding where local randomness is extracted from the channel after decoding. Even though the proposed joint scheme is related to the scheme in [12], the presented scheme exhibits a significantly different codebook construction adapted to our coordination framework. Our scheme requires the quantification of the amount of common randomness shared by the two nodes as well as the local randomness at each of the two nodes. This is a feature that is absent from the analysis in [12]. Lastly, when the noisy channel and the correlation between $X$ to $Y$ are both given by binary symmetric channels (BSCs), we study the effect of the capacity of the noisy channel on the sum rate of common and local randomness. We conclude this work by showing that the joint scheme outperforms the separate scheme in terms of the coordination rate in the high-capacity regime.

The remainder of the paper is organized as follows: Section II sets the notation. The problem of strong coordination over a noisy communication link is presented in Section III. We then derive achievability results for the noisy point-to-point coordination in Section IV for the joint scheme and in Section V for the separate scheme, respectively. In Section VI, we present numerical results for both schemes when the target joint distribution is described as a doubly binary symmetric source and the noisy channel is given by a BSC.

II Notation

Throughout the paper, we denote a discrete random variable with upper-case letters (e.g., $X$) and its realization with lower case letters (e.g., $x$), respectively. The alphabet size of the random variable $X$ is denoted as $|\mathcal{X}|$. We use $X^{n}$ to denote the finite sequence $[X_{1},X_{2},\dots,X_{n}]$. The binary entropy function is denoted as $h_{2}(\cdot)$, the indicator function by $\mathds{1}(w)$, and the counting function as $N(\omega|w^{n})=\sum_{i=1}^{n}\mathds{1}(w_{i}=\omega)$. $\mathbb{P}[A]$ is the probability that the event $A$ occurs. The pmf of the discrete random variable $X$ is denoted as $P_{X}(x)$. However, we sometime use the lower case notation (e.g., $p_{X}(x)$) to distinguish target pmfs or alternative definitions. We let $\mathbb{D}(P_{X}(x)||Q_{X}(x))$ denote the Kullback-Leibler divergence between two distributions $P_{X}(x)$ and $Q_{X}(x)$ defined over an alphabet $\cal{X}$. ${\cal T}_{\epsilon}^{n}(P_{X})$ denotes the set of $\epsilon$-strongly letter-typical sequences of length $n$. Finally, $P^{\otimes n}_{X_{1}X_{2}\dots X_{k}}$ denotes the joint pmf of $n$ i.i.d. random variables $X_{1},X_{2},\dots,X_{k}$.

III Problem Definition

The point-to-point coordination setup we consider in this work is depicted in Fig. 1. Node $X$ receives a sequence of actions $X^{n}\in\mathcal{X}^{n}$ specified by nature where $X^{n}$ is i.i.d. according to a pmf $p_{X}$. Both nodes have access to shared randomness $J$ at rate $R_{o}$ bits/action from a common source, and each node possesses local randomness $M_{k}$ at rate $\rho_{k}$, $k=1,2$. Thus, in designing a block scheme to coordinate $n$ actions of the nodes, we assume $J\in\{1,\ldots,2^{nR_{o}}\}$, and $M_{k}\in\{1,\ldots,2^{n\rho_{k}}\}$, $k=1,2$, and we wish to communicate a codeword $A^{n}(I)$ over the rate-limited DMC $P_{B|A}(b|a)$ to Node $Y$, where $I$ denotes the (appropriately selected) coordination message. The codeword $A^{n}(I)$ is constructed based on the input action sequence $X^{n}$, the local randomness $M_{1}$ at Node $X$, and the common randomness $J$. Node $Y$ generates a sequence of actions $Y^{n}\in\mathcal{Y}^{n}$ based on the received codeword $B^{n}$, common randomness $J$, and local randomness $M_{2}$. We assume that the common randomness is independent of the action specified at Node $X$. A tuple $(R_{o},\rho_{1},\rho_{2})$ is deemed achievable if for each $\epsilon>0$, there exist $n\in\mathbb{N}$ and a (strong coordination) coding scheme such that the joint pmf of actions $\hat{P}_{X^{n},Y^{n}}$ induced by this scheme and the $n$-fold product111This is the joint pmf of $n$ i.i.d. copies of $(X,Y)\sim p_{XY}$. of the desired joint pmf $P^{\otimes n}_{XY}$ are close in total variation, i.e.,

[TABLE]

We now present the two achievable coordination schemes.

IV Joint Coordination Channel Coding

This scheme follows an approach similar to those in [1, 10, 4, 5] where coordination codes are designed based on allied channel resolvability problems [13]. The structure of the allied problem pertinent to the coordination problem at hand is given in Fig. 2. The aim of the allied problem is to generate $n$ symbols for two correlated sources $X^{n}$ and $Y^{n}$ whose joint statistics is close to $P^{\otimes n}_{XY}$ as defined by (1). To do so, we employ three independent and uniformly distributed messages $I$, $K$, and $J$ and two codebooks $\mathscr{A}$ and $\mathscr{C}$ as shown in Fig. 2. To define the two codebooks, consider auxiliary random variables $A\in\mathcal{A}$ and $C\in\mathcal{C}$ jointly correlated with $(X,Y)$ as $P_{XYABC}=P_{AC}P_{X|AC}P_{B|A}P_{Y|BC}$.

From this factorization it can be seen that the scheme consists of two reverse test channels $P_{X|AC}$ and $P_{Y|AC}$ used to generate the sources from the codebooks. In particular, $P_{Y|AC}=P_{B|A}P_{Y|BC}$, i.e., the randomness of the DMC contributes to the randomized generation of $Y^{n}$.

Generating $X^{n}$ and $Y^{n}$ from $I$, $K$, $J$ represents a complex channel resolvability problem with the following ingredients:

• •

Nested codebooks: Codebook $\mathscr{C}$ of size $2^{n(R_{o}+R_{c})}$ is generated i.i.d. according to pmf $P_{C}$, i.e., $C^{n}_{ij}\sim P_{C}^{\otimes n}$ for all $(i,j)\in\cal{I}\times\cal{J}$. Codebook $\mathscr{A}$ is generated by randomly selecting $A^{n}_{ijk}\sim P_{A|C}^{\otimes n}(\cdot|{C_{ij}^{n}})$ for all $(i,j,k)\in\cal{I}\times\cal{J}\times\cal{K}$.

• •

Encoding functions:

$C^{n}\!:\{1,2,\dots,2^{nR_{c}}\}\!\times\!\{1,2,\dots,2^{nR_{o}}\}\!\rightarrow\mathcal{C}^{n}$,

$A^{n}\!:\{1,\dots,2^{nR_{c}}\}\!\times\!\{1,\dots,2^{nR_{o}}\}\!\times\!\{1,\dots,2^{nR_{a}}\}\!\rightarrow\mathcal{A}^{n}$.

• •

Indices: $I,J,K$ are independent and uniformly distributed over $\{1,\dots,2^{nR_{c}}\}$, $\{1,\dots,2^{nR_{o}}\}$, and $\{1,\dots,2^{nR_{a}}\}$, respectively. These indices select the pair of codewords $C^{n}_{IJ}$ and $A^{n}_{IJK}$ from codebooks $\mathscr{C}$ and $\mathscr{A}$.

• •

The selected codewords $C^{n}_{IJ}$ and $A^{n}_{IJK}$ are then passed through DMC $P_{X|AC}$ at Node $X$, while at Node $Y$, codeword $A^{n}_{IJK}$ is sent through DMC $P_{B|A}$ whose output $B^{n}$ is used to decode codeword $C^{n}_{\hat{I}J}$ and both are then passed through DMC $P_{Y|BC}$ to obtain $Y^{n}$.

Since the codewords are randomly chosen, the induced joint pmf of the generated actions and codeword indices in the allied problem is itself a random variable and depends on the random codebook. Given a realization of the codebooks

[TABLE]

the code-induced joint pmf of the actions and codeword indices in the allied problem is given by

[TABLE]

where $\mathsf{P}_{\hat{I}|B^{n}J}$ denotes the pmf induced by the operation of decoding the index $I$ using the common randomness and the channel output at Node $Y$. Note that by denoting the decoding operation as a pmf, we can even incorporate randomized decoders. Note also that the indices for the $C$-codeword that generate $X$ and $Y$ sequences in (3) can be different since the decoding of the index $I$ at Node $Y$ may fail. We are done if we accomplish the following tasks: (1) identify conditions on $R_{o},R_{c},R_{a}$ under which the code-induced pmf $\mathring{P}_{X^{n}Y^{n}}$ is close to the design pmf $P_{XY}^{\otimes n}$ in the total variation sense; and (2) devise a strong coordination scheme by inverting the operation at Node $X$. This will be done in following sections by subdividing the analysis of the allied problem.

IV-A Resolvability constraints

Assuming that the decoding of $I$ and the codeword $C^{n}_{IJ}$ occurs perfectly at Node $Y$, we see that the code-induced joint pmf induced by the scheme for the allied problem for a given realization of the codebook $\mathsf{C}$ in (2) is

[TABLE]

The following result quantifies when the induced distribution in (4) is close to the $n$-fold product of the design pmf $P_{XY}$.

Lemma 1** (Resolvability constraints).**

The total variation between the code-induced pmf $\check{P}_{X^{n}Y^{n}}$ in (4) and the desired pmf $P^{\otimes n}_{XY}$ asymptotically vanishes, i.e., {\mathbb{E}_{\mathsf{C}}}\big{[}\left\lVert\check{P}_{X^{n}Y^{n}}-P^{\otimes n}_{XY}\right\rVert_{{\scriptscriptstyle TV}}\big{]}\rightarrow 0 as $n\rightarrow\infty$, if

[TABLE]

Note that here $\mathbb{E}_{\mathsf{C}}$ denotes the expectation over the random realization of the codebooks.

Proof.

In the following, we drop the subscripts from the pmfs for simplicity. Let $R\triangleq R_{a}+R_{c}+R_{o}$, and choose $\epsilon>0$. Consider the argument for \mathbb{E}_{\mathsf{C}}\big{[}\mathbb{D}(\check{P}_{X^{n}Y^{n}}||P^{\otimes n}_{XY})\big{]} shown at the top of the following page.

In this argument:

• (a)

follows from the law of iterated expectation. Note that we have used $(a^{n}_{ijk},c^{n}_{ij})$ to denote the codewords corresponding to the indices $(i,j,k)$, and $(a^{n}_{i^{\prime}j^{\prime}k^{\prime}},c^{n}_{i^{\prime}j^{\prime}})$ to denote the codewords corresponding to the indices $(i^{\prime},j^{\prime},k^{\prime})$, respectively.

• (b)

follows from Jensen’s inequality.

• (c)

follows from dividing the inner summation over the indices $(i^{\prime},j^{\prime},k^{\prime})$ into three subsets based on the indices $(i,j,k)$ from the outer summation.

• (d)

follows from taking the expectation within the subsets in (c) such that when

• –

$(i^{\prime},j^{\prime})=(i,j),(k^{\prime}\neq k)$: $a^{n}_{i^{\prime}j^{\prime}k^{\prime}}$ is conditionally independent of $a^{n}_{ijk}$ following the nature of the codebook construction (i.e., i.i.d. at random);

• –

$(i^{\prime},j^{\prime})\neq(i,j)$: both codewords ($a^{n}_{ijk},c^{n}_{ij}$) are independent of $(a^{n}_{i^{\prime}j^{\prime}k^{\prime}},c^{n}_{i^{\prime}j^{\prime}})$ regardless of the value of $k$. As a result, the expected value of the induced distribution with respect to the input codebooks is the desired distribution $P^{\otimes n}_{XY}$ [1].

• (e)

follows from

• –

$(i^{\prime},j^{\prime},k^{\prime})=(i,j,k)$: there is only one pair of codewords $(a^{n}_{ijk},c^{n}_{ij})$;

• –

when $(k^{\prime}\neq k)$ while $(i^{\prime},j^{\prime})=(i,j)$ there are $(2^{nR_{a}}-1)$ indices in the sum;

• –

$(i^{\prime},j^{\prime})\neq(i,j)$: the number of the indices is at most $2^{nR}.$

• (f)

results from splitting the outer summation: The first summation contains typical sequences and is bounded by using the probabilities of the typical set. The second summation contains the tuple of sequences when the pair of actions sequences $x^{n},y^{n}$ and codewords $c^{n},a^{n}$ are not $\epsilon$-jointly typical (i.e., $(x^{n},y^{n},a^{n},c^{n})\notin{\cal T}_{\epsilon}^{n}(P_{XYAC})$). This sum is upper bounded following [4] with \mu_{XY}=\min_{x,y}\big{(}P_{XY}(x,y)\big{)}.

• (g)

following the Chernoff bound of the probability that a sequence is not strongly typical [14] where \mu_{XYAC}=\min_{x,y,a,c}\big{(}P_{XYAC}(x,y,a,c)\big{)}.

Consequently, the contribution of typical sequences can be made asymptotically small if

[TABLE]

while the second term converges to zero exponentially fast with $n$ [14]. Finally, by applying Pinsker’s inequality we have

[TABLE]

∎

Remark*.*

Given $\epsilon>0$, $R_{a}$, $R_{o}$, $R_{c}$ satisfying (5) and (6), it follows from (IV-A) that there exist an $n\in\mathbb{N}$ and a random codebook realization for which the code-induced pmf between the indices and the pair of actions satisfies

[TABLE]

IV-B Decodability constraint

Since the operation at Node $Y$ in Fig. 2 involves the decoding of $I$ and thus the codeword $C^{n}(I,J)$ using $B^{n}$ and $J$, the induced distribution of the scheme for the allied problem will not match that of (4) unless and until we ensure that the decoding succeeds with high probability as $n\rightarrow\infty$. The following lemma quantifies the necessary rate for this decoding to succeed asymptotically almost always.

Lemma 2** (Decodability constraint).**

Let $\hat{I},C^{n}_{\hat{I}J}$ be the output of a typicality-based decoder that uses common randomness $J$ to decode the index $I$ and the sequence $C^{n}_{{I}J}$ from $B^{n}$. If the rate for the index $I$ satisfies $R_{c}<I(B;C)$ then,

1. i.

\mathbb{E}_{\mathsf{C}}\big{[}\mathbb{P}[\hat{I}\neq I]\big{]}\rightarrow 0* as $n\rightarrow\infty$, where $\mathbb{P}[\hat{I}\neq I]$ is the probability that the decoding fails for a realization of the random codebook, and* 2. ii.

\lim\limits_{n\rightarrow\infty}\mathbb{E}_{\mathsf{C}}\big{[}\lVert\check{P}_{X^{n}Y^{n}IJK}-\mathring{P}_{X^{n}Y^{n}IJK}\rVert_{{\scriptscriptstyle TV}}\big{]}=0.**

Proof.

We start the proof of i) by calculating the average probability of error, averaged over all codewords in the codebook and averaged over all random codebook realizations.

[TABLE]

where in (a) we have used the fact that the conditional probability of error is independent of the triple of indices due to the i.i.d. nature of the codebook construction. Also, due to the random construction and the properties of jointly typical set, we have

[TABLE]

We now continue the proof by constructing the sets for each $j$ and $b^{n}\in\mathcal{B}^{n}$ that Node $Y$ will use to identify the transmitted index:

[TABLE]

The set $\hat{S}_{j,b^{n},\mathsf{c}}$ consists of indices $i\in I$ such that for a given common randomness index $J=j$ and channel realization $B^{n}=b^{n}$, the sequences $(b^{n},c_{ij}^{n})$ are jointly-typical. Assuming $(i,j,k)=(1,1,1)$ was realized, and if $\hat{S}_{1,b^{n},\mathsf{c}}=\{1\}$, then the decoding will be successful. The probability of this event is divided into two steps as follows:

•  First, assuming $(i,j,k)=(1,1,1)$ was realized, for successful decoding, $1$ must be an element of $\hat{S}_{J,B^{n},\mathsf{c}}$. The probability of this event can be bounded as follows.

[TABLE]

where (a) follows from the properties of jointly typical sets.

•  Next, assuming again that $(i,j,k)=(1,1,1)$ was realized, for successful decoding no index greater than or equal to $2$ must be an element of $\hat{S}_{J,B^{n},\mathsf{c}}$. The probability of this event can be bounded as follows:

[TABLE]

where (a) follows from the packing lemma [15], and (b) results if $R_{c}<I(B;C)-\delta(\epsilon)$.

Then from (9), the claim in i) follows as given by

[TABLE]

Finally, the proof of ii) follows in a straightforward manner. If the previous two conditions are met, then ${\mathbb{E}_{\mathsf{C}}[\mathbb{P}[\hat{I}\neq I]]\rightarrow 0}$ and $\mathbb{E}_{\mathsf{C}}[{P_{\hat{I}|B^{n}J}(\hat{i}|b^{n},j)]\rightarrow\delta_{I\hat{I}}}$, where $\delta_{I\hat{I}}$ denotes the Kronecker delta. Consequently, from (3) and (4)

[TABLE]

∎

IV-C Independence constraint

We complete modifying the allied structure to mimic the original problem with a final step. By assumption, we have a natural independence between the action sequence $X^{n}$ and the common randomness $J$. As a result, the joint distribution over $X^{n}$ and $J$ in the original problem is a product of the marginal distributions $P^{\otimes n}_{X}$ and $P_{J}$. To mimic this behavior in the scheme for the allied problem, in Lemma 3 we artificially enforce independence by ensuring that the mutual information between $X^{n}$ and $J$ vanishes.

Lemma 3** (Independence constraint).**

Consider the scheme for the allied problem given in Fig. 2. Both $I(J;X^{n})\rightarrow 0$ and \mathbb{E}_{\mathsf{C}}\big{[}||\check{P}_{X^{n}J}-P^{\otimes n}_{X}P_{J}||_{{\scriptscriptstyle TV}}\big{]}\rightarrow 0 as $n\rightarrow\infty$ if the code rates satisfy

[TABLE]

The proof of Lemma 3 builds on the results of Section IV-B and the proof of Lemma 1 of Section IV-A, resulting in

[TABLE]

Remark*.*

Given $\epsilon>0$, $R_{a}$, $R_{c}$ meeting (11) and (12), it follows from (IV-C) that there exist an $n\in\mathbb{N}$ and a random codebook realization for which the code-induced pmf between the common randomness $J$ and the actions of Node $X$ satisfies

[TABLE]

In the original problem of Fig. 1, the input action sequence $X^{n}$ and the index $J$ from the common randomness source are available and the $A$- and $C$-codewords are to be selected. Now, to devise a scheme for the strong coordination problem, we proceed as follows. We let Node $X$ choose indices $I$ and $K$ (and, consequently, the $A$- and $C$-codewords) from the realized $X^{n}$ and $J$ using the conditional distribution $\mathring{P}_{IK|X^{n}J}$. The joint pmf of the actions and the indices is then given by

[TABLE]

Finally, we can argue that

[TABLE]

since the total variation between the marginal pmf $\hat{P}_{X^{n}Y^{n}}$ and the design pmf $P_{XY}^{\otimes n}$ can be bounded as

[TABLE]

where (a) follows from the triangle inequality; (b) follows from (3), (4), (15) and [3, Lemma V.1]; (c) follows from [3, Lemma V.2]. The terms in the RHS of (c) can be made vanishingly small provided the resolvability, decodability, and independence conditions are met. Thus, we are guaranteed that by meeting the five conditions of Lemmas 1-3, the scheme defined by (15) achieves strong coordination between Nodes $X$ and $Y$ by communicating over the DMC $P_{B|A}$. Note that since the operation at Nodes $X$ and $Y$ amount to an index selection according to $\mathring{P}_{IK|X^{n}J}$, and a generation of $Y^{n}$ using the DMC $P_{Y|BC}$, both operations are randomized. The last step is to derandomize the operations at Nodes $X$ and $Y$ by viewing the corresponding local randomness as the source of randomness in these operations. This is detailed next.

IV-D Local randomness rates

At Node $X$, local randomness is employed to randomize the selection of indices $(I,K)$ by synthesizing the channel $\mathring{P}_{IK|X^{n}J}$ whereas Node $Y$ utilizes its local randomness to generate the action sequence $Y^{n}$ by simulating the channel $P_{Y|BC}$. Using the arguments in [5], we can argue that for any given realization of $J$, the minimum rate of local randomness required for the probabilistic selection of indices $(I,K)$ can be derived by quantifying the number of $A$ and $C$ codewords (equivalently the pair of indices $I,K$) jointly typical with $X^{n}$. Quantifying the list size as in [5] yields $\rho_{1}\geq R_{a}+R_{c}-I(X;AC)$. At Node $Y$, the necessary local randomness for the generation of the action sequence is bounded by the channel simulation rate of DMC $P_{Y|BC}$ [16]. Thus, $\rho_{2}\geq H(Y|BC)$.

Moreover, one can always view a part of the common randomness as local randomness, which then allows us to incorporate the rate-transfer arguments given in [5, Lemma 2]. Combining the rate-transfer argument with the constraints in Lemmas 1-3, we obtain following inner bound to the strong coordination capacity region.

Theorem 1**.**

A tuple $(R_{o},\rho_{1},\rho_{2})$ is achievable for the strong noisy communication setup in Fig. 1 if for some $R_{a},R_{c},\delta_{1},\delta_{2}\geq 0$,

[TABLE]

V Separate Coordination-Channel Coding Scheme with Randomness Extraction

As a basis for comparison, we will now introduce a separation-based scheme that involves randomness extraction. We first use a $(2^{nR_{c}},2^{nR_{o}},n)$ noiseless coordination code with the codebook $\mathscr{U}$ to generate a message $I$ of rate $R_{c}$. Such a code exists if and only if the rates $R_{o},R_{c}$ satisfy [1]

[TABLE]

This coordination message $I$ is then communicated over the noisy channel using a rate-$R_{a}$ channel code over $m$ channel uses with codebook $\mathscr{A}$. Hence, $R_{c}=\lambda R_{a}$, where $\lambda=m/n$. The probability of decoding error can be made vanishingly small if $R_{a}<I(A;B)$. Then, from the decoder output $\hat{I}$ and the common randomness message $J$ we reconstruct the coordination sequence $U^{n}$ and pass it though a test channel $P_{Y|U}$ to generate the action sequence at Node $Y$. Note that this separation scheme is constructed as a special case of the joint coordination-channel scheme of Fig. 2 by choosing $C=U$ and $P_{AC}=P_{A}P_{U}$.

In the following, we restrict ourselves to additive-noise DMCs, i.e.,

[TABLE]

where $Z$ is the noise random variable drawn from some finite field $\mathcal{Z}$, and “$+$” is the native addition operation in the field. To extract randomness, we exploit the additive nature of the channel to recover the realization of the channel noise from the decoded codeword. Thus, at the channel decoder output we obtain

[TABLE]

where $B^{m}$ is the channel output and $A^{m}(\hat{I})$ the corresponding decoded channel codeword. We can then utilize a randomness extractor on $\hat{Z}^{m}$ to supplement the local randomness available at Node $Y$. The following lemma provides some guarantees with respect to the randomness extraction stage.

Lemma 4**.**

Consider the separation based scheme over a finite-field additive DMC. If $R_{a}<I(A;B)$ and ${m,n\rightarrow\infty}$ with $\frac{m}{n}=\lambda$, the following hold:

1. i.

${\mathbb{P}[Z^{m}\neq\hat{Z}^{m}]\rightarrow 0},$** 2. ii.

${\frac{1}{m}H(\hat{Z}^{m})\rightarrow H(Z)},$* and* 3. iii.

${I(\hat{Z}^{m};I\hat{I})\rightarrow 0}$.

Proof.

Let $P_{e}$ be the probability of decoding error (i.e.,  $P_{I_{e}}=\mathbb{P}[I\neq\hat{I}]$ and $P_{Z_{e}}=\mathbb{P}[Z^{m}\neq\hat{Z}^{m}]$). We first show the claim in i). From the channel coding theorem we obtain that $P_{I_{e}}\leq 2^{-n\varepsilon}$. Consequently, from (18) and (19) $\mathbb{P}[Z^{m}\neq\hat{Z}^{m}]$ will follow directly as $P_{Z_{e}}\leq 2^{-m\varepsilon}$.

Then, the claim in ii) is shown as follows

[TABLE]

where (a) follows from the chain rule of entropy; (b) follows from Fano’s inequality and the fact that $Z^{m}\sim P_{Z}^{\otimes n}$;

Finally, the claim in iii) is shown by the following chain of inequalities:

[TABLE]

where (a) follows from Fano’s inequality; (b) follows from $P_{I_{e}}\leq 2^{-n\varepsilon}$, $P_{Z_{e}}\leq 2^{-m\varepsilon}$ and $\epsilon,\varepsilon\rightarrow 0$ as $n,m\rightarrow\infty$ respectively. ∎

Now, similar to the joint scheme, we can quantify the local randomness at both nodes, apply the rate transfer lemma [5, Lemma 2], and set $\lambda=1$ to facilitate comparison with the joint scheme from Section IV. The following theorem then describes an inner bound to the strong coordination region using the separate-based scheme with randomness extraction.

Theorem 2**.**

There exists an achievable separation based coordination-channel coding scheme for the strong setup in Fig 1 such that (1) is satisfied for $\delta_{1}\geq 0,\delta_{2}\geq 0$ if

[TABLE]

The proof follows in a straightforward way from the proofs of both Theorem 1 and Lemma 4 and is therefore omitted.

VI Example

In the following, we compare the performance of the joint scheme in Section IV and the separation-based scheme in Section V using a simple example. Specifically, we let $X$ be a Bernoulli-$\frac{1}{2}$ source, the communication channel $P_{B|A}$ be a binary symmetric channel with crossover probability $p_{o}$ (BSC($p_{o}$)), and the conditional distribution $P_{Y|X}$ be a BSC($p$).

VI-A Basic separation scheme with randomness extraction

To derive the rate constraints for the basic separation scheme, we consider $X-U-Y$ with $U\!\sim\!\mathrm{Bernoulli}-\frac{1}{2}$ (which is known to be optimal [3]), $P_{U|X}=\mathrm{BSC}(p_{1})$, and $P_{Y|U}=\mathrm{BSC}(p_{2})$, $p_{2}\in[0,p]$, $p_{1}=\dfrac{p-p_{2}}{1-2p_{2}}$. Using this to obtain the mutual information terms in Theorem 2, we get

[TABLE]

After a round of Fourier-Motzkin elimination by using (21a)-(21c) in Theorem 2, we obtain the following constraints for the achievable region using the separation-based scheme with randomness extraction:

[TABLE]

Note that (22a) presents the achievable sum rate constraint for the total required randomness in the system.

VI-B Joint scheme

The rate constraints for the joint scheme are constructed in two stages. First, we derive the scheme for the codebook cardinalities ${|{\cal A}|=2}$ and ${|{\cal C}|=2}$, an extension to larger $|{\cal C}|$ is straightforward but more tedious (see Figs. 3 and 4)222Note that these cardinalities are not optimal. They are, however, analytically feasible and provide a good intuition about the performance of the scheme.. The joint scheme correlates the codebooks while ensuring that the decodability constraint (17e) is satisfied. To get the best tradeoff, we find the joint distribution $P_{AC}$ that maximizes $I(B;C)$. For ${|\mathcal{C}|=2}$ this is simply given by ${P_{A|C}(a|c)=\delta_{ac}}$. Then, the distribution $P_{X}(x)P_{CA|X}(c,a|x)P_{B|A}(b|a)P_{Y|BC}(y|b,c)$ that produces the boundary of the strong coordination region for the joint scheme is formed by cascading two BSCs and another symmetric channel, yielding the Markov chain ${X-(C,A)-(C,B)-Y}$, with the channel transition matrices

[TABLE]

for some $\alpha,\beta\in[0,1].$

Then, the mutual information terms in Theorem 1 can be expressed with $p_{2}\triangleq(1-p_{o})\alpha+p_{o}\beta$ as

[TABLE]

To find the minimum achievable sum rate we first perform Fourier-Motzkin elimination on the rate constraints in Theorem 1 and then minimize the information terms with respect to the parameters $p_{2}$, $\alpha$, and $\beta$ as follows:

[TABLE]

VI-C Numerical results

Fig. 3 presents a comparison between the minimum randomness sum rate $R_{o}+\rho_{1}+\rho_{2}$ required to achieve coordination using the joint and the separate scheme with randomness extraction when the communication channel is given by BSC($p_{o}$). The target distribution is set as $p_{Y|X}\!=\!\mathrm{BSC}(0.4)$. The rates for the joint scheme are obtained by solving the optimization problem in (29). Similar results are obtained for the joint scheme with $|{\cal C}|>2$. For the separate scheme we choose $p_{2}$ such that $h_{2}(p_{1})=h_{2}(p_{0})$ to maximize the amount of extracted randomness. We also include the performance of the separate scheme without randomness extraction. As can be seen from Fig. 3, both the joint scheme and the separate scheme with randomness extraction provide the same sum rate $R_{o}\!+\!\rho_{1}\!+\!\rho_{2}$ for $p_{o}\!\leq\!p^{\prime}_{o}$ where $p^{\prime}_{o}\!\triangleq\!\frac{1-\sqrt{1-2p}}{2}$. We also observe that for noisy channels the joint scheme approaches the performance of the separate scheme when the cardinality of $C$ is increased. In this regime, we let $p_{2}=p_{0}$ such that $h_{2}(p_{2})=h_{2}(p_{0})$ in order to maximize the amount of extracted randomness. This is done by selecting $\alpha=0$ and $\beta=1$ associated with $P_{Y|BC}$. However, it can be easily shown that for $p_{0}>p_{0}^{\prime}$ this does not ensure a target distribution of $P_{XY}^{\otimes n}$ anymore. Therefore, the optimization over the parameters $\alpha$ and $\beta$ now results in a larger sum rate $R_{o}\!+\!\rho_{1}\!+\!\rho_{2}$ as can be seen from Fig. 3. As $p_{o}$ increases further, the required total randomness of the joint scheme approaches the one for the basic separate scheme again.

Fig. 4 provides a comparison of the communication rate for both schemes. Note that the joint scheme provides significantly smaller rates than the separation scheme with randomness extraction for $p_{o}\leq p^{\prime}_{o}$, independent of the cardinality of $|\mathcal{C}|$. Thus, in this regime joint coordination-channel coding provides an advantage in terms of communication cost and outperforms the separation-based scheme for the same amount of randomness injected into the system.