On the Common Randomness Capacity of a Special Class of Two-way Channels

TL;DR

This paper investigates the capacity for two nodes to generate shared randomness over special intertwined two-way channels, providing bounds and constructive schemes for achieving this in complex communication settings.

Contribution

It introduces new bounds and constructive schemes for common randomness capacity in intertwined two-way channels, extending understanding of their fundamental limits.

Findings

Outer bound on CR capacity for intertwined receiver-decomposable channels

Bound on auxiliary variable cardinality

Outer bound matches Venkatesan-Anantharam CR capacity for certain settings

Abstract

In this paper, we would like to study the common randomness (CR) capacity of intertwined two-way channels, namely those whose marginal channel transition probabilities depends also on the signal they transmit. We bring a few special settings and provide constructive schemes with which the two nodes can agree upon a common randomness. We then provide an outer bound on the CR capacity of intertwined receiver-decomposable (RD) two-way channel and will provide a bound on the cardinality of the available auxiliary variables. We will also show this outer bound is bounded above by Venkatesan-Anantharam CR capacity which makes it tight for decomposing two-way setting.

Equations

$$Y_1|X_1=0,X_2\sim BSC(p_1)Y_1|X_1=1,X_2\sim BSC(p_2)$$

$$Y_2|X_1,X_2=0\sim BSC(q_1)Y_2|X_1,X_2=1\sim BSC(q_2)$$

$$\frac{{\sum }_{i=1}^{n}i(\frac{1}{2}{)}^i}{n}$$

$$\frac{Z_1+Z_2+...+Z_n}{n}$$

$$\mathbb{E}(Z_i)$$

$$p(y_1,y_2|x_1,x_2)=p(y_1|x_1,x_2)p(y_2|x_1,x_2)$$

$$f=(f_1,f_2,...,f_n)$$

$$g=(g_1,g_2,...,g_n)$$

$$\frac{1-\lambda }{K}\le \text{Pr}\{\Phi (Y_1^n)=\Psi (Y_2^n)=l\}\le \frac{1+\lambda }{K}\text{for each }l=1,2,...,K.$$

$$\underset{n\to \infty }{\mathrm{liminf}}\frac{\log K_n}{n}\ge R\text{and}\underset{n\to \infty }{\lim }{\lambda }_n=0.$$

$$\underset{\lambda \to 0}{\lim }\underset{n\to \infty }{\mathrm{limsup}}\frac{\log K_n}{n}\le R.$$

$$\underset{n\to \infty }{\mathrm{limsup}}\frac{\log K_n}{n}\le R.\text{for every}\lambda \ge 0$$

$$\Phi :{\mathcal{X}}_1^n\times {\mathcal{Y}}_1^n\mapsto [1:K]\cup \{e\}\text{and}\Psi :{\mathcal{X}}_2^n\times {\mathcal{Y}}_2^n\mapsto [1:K]\cup \{e\}$$

$$H(Y_1^n,Y_2^n)$$

$$H(Y_1^n)$$

$$H(Y_2^n)$$

$$I(Y_1^n;Y_2^n)$$

$$g_{j}(p(v|u))=\left\{\begin{array}[]{lr}p(x_{1},x_{2}|u)=\sum_{v}p(v|u)p(x_{1}|u)p(x_{2}|v)&j=1,2,\ldots,|\mathcal{X}_{1}|.|\mathcal{X}_{2}|-1\\ H(Y_{1}|X_{1},U=u)&j=|\mathcal{X}_{1}|.|\mathcal{X}_{2}|\\ p(v,x_{2},y_{2}|u)=\sum_{x_{1}}p(v|u)p(x_{1}|u)p(x_{2}|v)p(y_{2}|x_{1},x_{2})&j=|\mathcal{X}_{1}|.|\mathcal{X}_{2}|+1\end{array}\right.$$

$$h_{j}(p(x_{1},x_{2}|u^{{}^{\prime}},v^{{}^{\prime}}))=\left\{\begin{array}[]{lr}p(x_{1},x_{2}|u^{{}^{\prime}},v^{{}^{\prime}})&j=1,2,\ldots,|\mathcal{X}_{1}|.|\mathcal{X}_{2}|-1\\ H(Y_{2}|X_{2},V^{{}^{\prime}}=v^{{}^{\prime}})&j=|\mathcal{X}_{1}|.|\mathcal{X}_{2}|\end{array}\right.$$

$$V^{{}^{\prime \prime \prime }}\triangleq (V^{{}^{\prime \prime }},X_2)U^{{}^{\prime \prime }}\triangleq (U^{{}^{\prime }},X_1)$$

$$I(X_2;Y_1|X_1,U^{{}^{\prime \prime }})$$

$$I(X_1;Y_2|X_2,V^{{}^{\prime \prime \prime }})$$

$$|\mathcal{U}|\le |{\mathcal{X}}_1|.(|{\mathcal{X}}_1|.|{\mathcal{X}}_2|+1)\text{and}|\mathcal{V}|\le |{\mathcal{X}}_1|.|{\mathcal{X}}_2{|}^2$$

$$p(S=T)$$

$$P(\Phi \ne \Psi )\le \lambda$$

$$\max \{H(\Psi |\Phi ),H(\Phi |\Psi )\}\le 1+\lambda \log (K)$$

$$H(\Phi ,\Psi )$$

$$\min \{H(\Psi ),H(\Phi )\}\ge (1-2\lambda )\log (K)-2$$

$$-3+(1-3\lambda )\log (K)$$

$$-2+(1-2\lambda )\log (K)$$

$$-1+(1-\lambda )\log (K)$$

$$-3+(1-3\lambda )\log (K)$$

Full text

On the Common Randomness Capacity of a Special Class of Two-way Channels

Saeed Hajizadeh

I Abstratc

In this paper, we would like to study the common randomness (CR) capacity of intertwined two-way channels, namely those whose marginal channel transition probabilities depends also on the signal they transmit. We bring a few special settings and provide constructive schemes with which the two nodes can agree upon a common randomness. We then provide an outer bound on the CR capacity of intertwined receiver-decomposable (RD) two-way channel and will provide a bound on the cardinality of the available auxiliary variables. We will also show this outer bound is bounded above by Venkatesan-Anantharam’s CR capacity which makes it tight for decomposing two-way setting.

II Introduction

The concept of CR capacity of different settings has been introduced and studied by Ahlswede and Csiszar [1]. The CR capacity of decoupling two-way settings, namely those with general transition function factored as $p(y_{1},y_{2}|x_{1},x_{2})=p(y_{1}|x_{2})p(y_{2}|x_{1})$, has been solved by Venkatesan and Anantharam [2]. Recently, two-way channel transmission capacities have been studied [3], [4], [5]. We would like to study the CR capacity of intertwined two-way settings, namely those with $p(y_{1},y_{2}|x_{1},x_{2})=p(y_{1}|x_{1},x_{2})p(y_{2}|x_{1},x_{2})$. To get a better understanding of what one means by the maximum rate of generating common randomness in this setting, and also observe the difference of the CR capacity of a setting with its transmission capacity, we look at an example which at this point it is unknown how complicated finding its CR capacity can be. Consider two terminals, e.g. Alice and Bob, communicate over a special intertwined two-way setting in the absence of any external random sources. In other words, each symbol transmitted through the channel is only a function of the past symbols received. The setting is as follows: Assume Alice’s transmitted and received symbols are $X_{1}$ and $Y_{1}$ and Bob’s transmitted and received symbols are $X_{2}$ and $Y_{2}$, respectively. Assuming $0\leq p_{1},q_{1},p_{2},q_{2}\leq\frac{1}{2}$, the transition functions are defined as follows,

[TABLE]

We now explain different case,

• i.

$p_{1}=p_{2}=q_{1}=0$ and $q_{2}=1/2$: In this case, at the end of each block, Bob can receive some randomness, depending on whether he has sent $X_{2}=1$ or not at the beginning of that block, while Alice receives no randomness. The channel from Bob to Alice is always a $BSC(0)$ so set $X_{1}=1$ at all blocks. Also to start, set $X_{2}=1$ in the first block and in blocks $b=2,...,n$ Bob sends whatever he receives back to Alice using the-always-$BSC(0)$ backward channel. With this coding scheme it is clear that the first $Y_{2}=0$ random bit received by Bob stops the random communication from thereon. It can be seen that the average number of random bits per step generated this way is

[TABLE]

which vanishes as $n\rightarrow\infty$. However, we can use an even more adaptive coding scheme. In this scheme, Alice always sends $X_{1}=1$ and Bob starts the first block by sending $X_{2}=1$. If Bob receives $Y_{2}=1$, he sends back $X_{2}=1$ in the next block to agree on that bit with Alice and can still receive a random bit at the end of second block from Alice. This scheme continues until Bob receives $Y_{2}=0$. When he receives [math] in block $N_{1}$, he sends it back to Alice in block $N_{1}+1$ and starts a new block of communication by sending $X_{2}=1$. Notice that $N_{1}$, i.e. the first time Bob receives [math], is distributed geometrically and $\frac{N_{1}}{N_{1}+1}$ bits per step is transmitted in the first $N_{1}+1$ blocks. This scheme continues and in the next set of blocks $\frac{N_{2}}{N_{2}+1}$ random bits per step are generated and agreed upon. Call each $N_{i}+1$ blocks of communication a stage. Communication takes place in $n$ stages and the number of common random bits per step generated in $n$ stages is

[TABLE]

where $Z_{i}\triangleq\frac{N_{i}}{N_{i}+1}$. Since clearly $Z_{i}$’s are $i.i.d$, if the number of stages is large, the law of large numbers states that $\mathbb{E}(Z_{i})$ common random bits per step are generated and it is easy to see that,

[TABLE]

where $(a)$ follows from the fact that each $N_{i}$ is distributed geometrically. Notice that the sum-rate transmission capacity of this setting is at least $1.5$ bits per channel use.

• ii.

$p_{1}=p_{2}=q_{2}=1/2$ and $q_{1}=0$: In this case, set $X_{2}=0$ and as a result, $Y_{1}$ after each block of communication is a random bit distributed according to $Y_{1}|X_{2}=0\sim BSC(1/2)$ and can send it back noiselessly to terminal 2, Bob. Therefore, after $n$ symbol transmissions, Alice and Bob can agree upon $n-1$ uniformly common bits and thus $R=\frac{\log(2^{n-1})}{n}$ bits per step of communication is achievable. Clearly, $1$ bit per step of communication is the maximum amount of CR that Alice and Bob can agree upon and thus $C=1$ bits/step.

• iii.

$p_{1}=q_{1}=0$ and $p_{2}=q_{2}=1/2$: In this case, Alice and Bob communicate in $n$ blocks. On the first block $X_{2}$ sends [math] to set the $Y_{2}|X_{1}$ channel as a $BSC(0)$ and $X_{1}$ sends $1$ to set $Y_{1}|X_{2}$ as a $BSC(1/2)$. In the next block, Alice who has receives the bit from $X_{2}$ randomly retransmitts that bit back to Bob and Bob sends $X_{2}=0$ to open up a noiseless medium to receive the random bit generated in the first block. What $X_{2}$ sends in the first block is irrelevant. With this scheme, after $n$ transmissions, Alice and Bob can generate and agree upon $\frac{n}{2}$ bits of randomness which achieves a CR rate of $\frac{\log(2^{\frac{n}{2}})}{n}$ bits per step.

A clear defect in this schemes is Bob’s inactiveness in the first block of each transmission cycle. Another approach could be this: In the first block, Alice and Bob each send symbol $1$ and each receive a (possibly different) random bit. To make the generated random bit common, in the next two blocks, Alice and Bob take turn in sending [math] to open up the lane for its partner to deliver the random bit noiselessly. With this scheme, $\frac{2n}{3}$ bits are generated and agreed upon in $n$ steps of communication and thus the achievable CR rate is improved to be $2/3$ bits per step. Notice that an outer bound on the CR capacity of this setting is $1$ bit per step. I did not understand how you got 0.75 bit per step. Could you please write it with more detail?

• iv.

$p_{1}=p_{2}=0$: In this case, $Y_{1}|X_{2}\sim BSC(0)$ at all times thus set, say, $X_{1}=0$. In every block, set $X_{2,i}=Y_{2,i-2}$, $i=1,2,\ldots,n$. To begin the show, set $X_{2,1}=0$. As a result, $Y_{2,1}=0$ w.p. $q_{1}$ and $Y_{2,1}=1$ w.p. $1-q_{1}$ hence in the second block, the channel switches to $BSC(q_{2})$ w.p. $1-q_{1}$ and stays at $BSC(q_{1})$ w.p. $q_{1}$. Similarly, the channel transition function wavers when it is a $BSC(q_{2})$. The whole setting is depicted in Fig. 1. So $nC_{FS}$ bits of randomness are generated in this case and can be agreed upon in the other direction. But are they uniformly distributed? Constructively, it is not clear how we can achieve CR in this setting. It turns out that the entropy rate of the output might be the CR of this setting.

Now let us define the problem formally.

III Definition

We begin with the definition of a special class of intertwined two-way channels,

Definition 1

A special class of two-way channels, depicted as in Fig. 2, which we call the receiver-decomposable (RD) two-way channel is described as those in which the channel transition matrix is factored as follows,

[TABLE]

We desire to find the maximum amount of common randomness the two terminals can agree upon after a block of $n$ uses of the channel. In other words, let the terminals communicate through the channel in $n$ blocks and after gathering the corresponding outputs, each of them can compute a common random output. This random output will take its values from a set, say $\mathcal{K}$. The supremum of the cardinality of $\mathcal{K}$ devided by the number of communication steps, i.e. $\limsup_{n}\frac{1}{n}K$, is the common randomness capacity of RD two-way channel. To make it more precise, let $X_{1}$, $X_{2}$, $Y_{1}$, and $Y_{2}$ take values from finite sets $\mathcal{X}_{1}$, $\mathcal{X}_{2}$, $\mathcal{Y}_{1}$, and $\mathcal{Y}_{2}$, respectively. Also let $(f,g)$ be a strategy for transmission by the terminals defined as follows,

[TABLE]

and

[TABLE]

The terminals would then try to communicate over a certain number of steps, say $n$. In the first step, terminal 1 (T1) sends $f_{1}\in\mathcal{X}_{1}$ and T2 sends $g_{1}\in\mathcal{X}_{2}$. In step $k,2\leq k\leq n$, T1 constructs from the received symbols in the previous steps, i.e. $Y_{1}^{k-1}$, the symbol $f_{k}(Y_{1}^{k-1})\in\mathcal{X}_{1}$. Similarly, T2 sends $g_{k}(Y_{2}^{k-1})\in\mathcal{X}_{2}$. Come the end of the communication steps, the terminals have the sequence of their received signals, namely $Y_{1}^{n}$ and $Y_{2}^{n}$. T1 and T2 map $Y_{1}^{n}$ and $Y_{2}^{n}$ via $\Phi:\mathcal{Y}_{1}^{n}\mapsto[1:K]\cup\{e\}$ and $\Psi:\mathcal{Y}_{2}^{n}\mapsto[1:K]\cup\{e\}$ for some $\mathcal{K}=\{1,2,...,K\}$ and with $\{e\}$ being the error event in the common randomness generation. We would like to design the strategy such that for every $\lambda>0$

[TABLE]

This is called an $(n,K,\lambda)$ strategy to generate common randomness. We claim that a non-negative number $R$ is an achievable common randomness if there exists a sequence of $(n,K_{n},\lambda_{n})$ strategy such that,

[TABLE]

We also claim that a non-negative number $R$ is an outer bound on the common randomness capacity if for every strategy $(n,K_{n},\lambda)$ we have,

[TABLE]

The rate $R$ is said to be an strong outer bound on the common randomness capacity of a setting if for every strategy $(n,K_{n},\lambda)$ satisfying (6) we have

[TABLE]

The strong converse, from the definition, only assures strong convergence of the inner bound and the outer bound but by no means, can necessarily give us a better outer bound.

Notice that in an $(n,K,\lambda)$ protocol, the decoding functions $\Phi$ and $\Psi$ could be taken as the following mappings,

[TABLE]

However, it is clear that since each $X_{k,i}$, $k=1,2$, is a function of $Y_{k}^{i-1}$, mapping from the sequence of received output is sufficient.

Now to find an outer bound on the CR capacity of the RD two-way setting introduced in Definition 1, let us first bound some entropies. Starting with the joint entropy of the outputs, we have,

[TABLE]

where $(a)$ follows since $X_{k,i}$ is a function of $Y_{k}^{i-1}$, $k=1,2$, $(b)$ follows from the memorylessness of the setting, and $(c)$ follows from the RD (receiver decompose) property of the channel. We now bound the marginal entropy of each output,

[TABLE]

where $(a)$ follows since $X_{k,i}$ is a function of $Y_{k}^{i-1}$, $k=1,2$ and in the penultimate equality we have defined $U_{i}\triangleq Y_{1}^{i-1}$. Similarly,

[TABLE]

where $V_{i}\triangleq Y_{2}^{i-1}$. Notice that this selection of the auxiliary RVs induces the distribution of $p(u,v)p(x_{1}|u)p(x_{2}|v)$ and thus the Markov chains $U\rightarrow V\rightarrow X_{2}$ and $V\rightarrow U\rightarrow X_{1}$.

We also have,

[TABLE]

To bound the cardinality of $U$ and $V$, fix $p(x_{1}|u)$ and $p(x_{2}|v)$. Take the following $|\mathcal{X}_{1}|.|\mathcal{X}_{2}|+1$ continuous functions of $p(v|u)$,

[TABLE]

The first set of functions preserve $p(x_{1},x_{2})$ which for the fixed channel transition, preserves $H(Y_{1}|X_{1},X_{2})$. The second function preserves $H(Y_{1}|X_{1},U)$ and the last function preserves $p(v,x_{2},y_{2})$ and thus $H(Y_{2}|X_{2},V)$. Therefore $\mathcal{U}$ suffices to be taken $|\mathcal{U}^{{}^{\prime}}|\leq|\mathcal{X}_{1}|.|\mathcal{X}_{2}|+1$. Let the corresponding RV of $V$, after replacing $U$ by $U^{{}^{\prime}}$ be denoted by $V^{{}^{\prime}}$. Now for fixed $U^{{}^{\prime}}$ take the following $|\mathcal{X}_{1}|.|\mathcal{X}_{2}|$ functions of $p(x_{1},x_{2}|u^{{}^{\prime}},v^{{}^{\prime}})$,

[TABLE]

Similarly, $V^{{}^{\prime}}|U^{{}^{\prime}}=u^{{}^{\prime}}$ suffices to be taken to satisfy $|\mathcal{V}^{{}^{\prime\prime}}|\leq|\mathcal{X}_{1}|.|\mathcal{X}_{2}|$. However, the aforementioned Markov chains are still not necessarily satisfied by these RVs. An option to construct new RVs to satisfy the aforementioned Markov chains, which may not be necessary, can be to define,

[TABLE]

and the Markov chains are also satisfied and also,

[TABLE]

Thus we can take

[TABLE]

Now notice that from (6) we have,

[TABLE]

Therefore,

[TABLE]

Now from Fano’s inequality we have,

[TABLE]

We also have,

[TABLE]

where $(a)$ follows from conditioning on the event $A\triangleq\{\Phi=\Psi\}$ and $(b)$ follows from (6). From (18) and (19) we have

[TABLE]

Thus from (18) and (20) we have for every $0\leq\lambda\leq 1$

[TABLE]

and thus we have the following Theorem,

For instance for every $0\leq\lambda\leq 1$ we have,

[TABLE]

and thus,

[TABLE]

Theorem 2

The CR capacity of the intertwined two-way setting as described in Definition 1 is dominated by,

[TABLE]

where

[TABLE]

for some distribution $p(u,v)p(x_{1}|u)p(x_{2}|v)$ with $|\mathcal{U}|\leq|\mathcal{X}_{1}|.(|\mathcal{X}_{1}|.|\mathcal{X}_{2}|+1)$ and $|\mathcal{V}|\leq|\mathcal{X}_{1}|.|\mathcal{X}_{2}|^{2}$.

Notice that time-sharing RV $Q$ is used in the achievable rates to convexify (and possibly enlarge) the region. Here, however, we do not need to make the outer bound convex and enlarge it. Hence, there is no point in using a time-sharing RV in the outer bound. Now let us compare this outer bound with Venkatesan-Anantharam’s CR capacity,

[TABLE]

Therefore, for the case of $p(y_{1},y_{2}|x_{1},x_{2})=p(y_{1}|x_{2})p(y_{2}|x_{1})$, our outer bound is bounded above as follows,

From (26) and (27) we have

[TABLE]

and from (28) and (29) we have,

[TABLE]

Therefore, from (30) and (31) we have,

[TABLE]

which is the CR capacity of decomposing two-way setting of Anantharam. Therefore, our outer bound is tight in that case and the CR capacity of RD decomposing setting $p(y_{1},y_{2}|x_{1},x_{2})=p(y_{1}|x_{1},x_{2})p(y_{2}|x_{1},x_{2})$ is no more than CR capacity of decomposing setting $p(y_{1},y_{2}|x_{1},x_{2})=p(y_{1}|x_{2})p(y_{2}|x_{1})$.