Feedback-Capacity of Degraded Gaussian Vector BC using Directed Information and Concave Envelopes

TL;DR

This paper extends the understanding of feedback capacity in physically degraded Gaussian vector broadcast channels by employing directed information and concave envelope factorization, showing feedback does not enlarge capacity regions.

Contribution

It introduces a novel approach using directed information and concave envelope factorization to determine feedback capacity in Gaussian vector BCs, beyond EPI-based methods.

Findings

Feedback does not enlarge capacity in physically degraded Gaussian vector BCs.

Directed information and concave envelope factorization effectively characterize feedback capacity.

Reversely physically degraded BCs are unaffected by feedback in terms of capacity.

Abstract

It is known that the capacity region of a two user physically degraded discrete memoryless (DM) broadcast channel (BC) is not enlarged by feedback. An identical result holds true for a physically degraded Gaussian BC, established later using a variant of the Entropy Power Inequality (EPI). In this paper, we extend the latter result to a physically degraded Gaussian Vector BC (PD-GVBC). However, the extension is not EPI based, but employs a recent result on the factorization of concave envelopes. While the existing concave envelope factorization results do not hold in the presence of feedback, we show that factorizing the corresponding directed information quantities suffice to attain the feedback capacity region of a PD-GVBC. Our work demonstrates that factorizing concave envelopes of directed information can handle situations involving feedback. We further show that the capacity region of a discrete memoryless reversely physically degraded BC is not enlarged by feedback.

Equations

$$P_{e,1}^{(n)}=\frac{1}{2^{n{R}_1}}\sum _{m_1}\mathbb{P}\left[(d_1({\mathbf{Y}}^n)\ne m_1|m_1\text{sent})\right]$$

$$P_{e,2}^{(n)}=\frac{1}{2^{n{R}_2}}\sum _{m_2}\mathbb{P}\left[(d_2({\mathbf{Z}}^n)\ne m_2|m_2\text{sent})\right]$$

$${\lambda }_n=\max (P_{e,1},P_{e,2}).$$

$$R_1\le I(X;Y|U)$$

$$R_2\le I(U;Z),$$

$$\displaystyle R_{1}\leq\frac{1}{2}\log\Bigg{(}\frac{\left|{\textbf{B}_{\textbf{1}}+\textbf{K}}\right|}{\left|{\textbf{K}}\right|}\Bigg{)}$$

$$\displaystyle R_{2}\leq\frac{1}{2}\log\Bigg{(}\frac{\left|{\textbf{B}_{\textbf{2}}+\textbf{B}_{\textbf{1}}+\textbf{K}+\tilde{\textbf{K}}}\right|}{\left|{\textbf{B}_{\textbf{1}}+\textbf{K}+\tilde{\textbf{K}}}\right|}\Bigg{)},$$

$$g(x)=\underset{p(x):\mathbb{E}\left[X\right]=x}{\sup }\mathbb{E}\left[f(X)\right].$$

$$s_{\lambda }^q(\mathbf{X}):=I(\mathbf{X};\mathbf{Y})-\lambda I(\mathbf{X};\mathbf{Z}).$$

$$s_{\lambda }^q(\mathbf{X}|V):=I(\mathbf{X};\mathbf{Y}|V)-\lambda I(\mathbf{X};\mathbf{Z}|V).$$

$$S_{\lambda }^q(\mathbf{X}):=\mathfrak{C}(s_{\lambda }^q(\mathbf{X})).$$

$$\mathfrak{C}(s_{\lambda }^q(\mathbf{X}))$$

$$=\underset{\begin{array}{c}p(v|\mathbf{x}):\\ V\to \mathbf{X}\to (\mathbf{Y},\mathbf{Z})\end{array}}{\sup }I(\mathbf{X};\mathbf{Y}|V)-\lambda I(\mathbf{X};\mathbf{Z}|V).$$

$$S_{\lambda }^q(\mathbf{X}|W):=\sum _{w}p(w)S_{\lambda }^q(\mathbf{X}|W=w).$$

$$S_{\lambda }^q(\mathbf{X}|W)\le S_{\lambda }^q(\mathbf{X}).$$

$$s_{\lambda }^{q\times q}({\mathbf{X}}_1,{\mathbf{X}}_2):=I({\mathbf{X}}_1,{\mathbf{X}}_2;{\mathbf{Y}}_1,{\mathbf{Y}}_2)-\lambda I({\mathbf{X}}_1,{\mathbf{X}}_2;{\mathbf{Z}}_1,{\mathbf{Z}}_2).$$

$$S_{\lambda }^{q\times q}({\mathbf{X}}_1,{\mathbf{X}}_2)\le S_{\lambda }^q({\mathbf{X}}_1)+S_{\lambda }^q({\mathbf{X}}_2).$$

$$I(X^N\to Y^N)=\sum _{n=1}^{N}I(X^n;Y_n|Y^{n-1}).$$

$$I(X^N\to Y^N)\le I(X^N;Y^N)$$

$${\overrightarrow{s}}_{\lambda }^{q\times q}({\mathbf{X}}_1,{\mathbf{X}}_2)$$

$${\mathcal{O}}_{{\mathbf{K}}^{{}^{\prime }}}=\bigcup \left(I(V;\mathbf{Z}),I(\mathbf{X};\mathbf{Y}|V)\right).$$

$$\mathcal{S}={\cap }_{\lambda \ge 1}\{(R_1,R_2)\in {\mathbb{R}}_{+}^2:R_1+\lambda R_2\le V_{\lambda }\},$$

$$V_{\lambda }=\underset{(R_1,R_2)\in \mathcal{S}}{\max }{R}_1+\lambda R_2.$$

$$\underset{(R_1,R_2)\in {\mathcal{O}}_{{\text{K}}^{{}^{\prime }}}}{\max }{R}_1+\lambda R_2\le \underset{(R_1,R_2)\in {\mathcal{I}}_{{\text{K}}^{{}^{\prime }}}}{\max }{R}_1+\lambda R_2.$$

$$\underset{(R_1,R_2)\in {\mathcal{O}}_{{\text{K}}^{{}^{\prime }}}}{\max }{R}_1+\lambda R_2$$

$$\le \underset{\text{X}:\mathbb{E}\left[\text{X}{\text{X}}^{\text{T}}\right]\preccurlyeq {\text{K}}^{{}^{\prime }}}{\sup }\lambda I(V;\text{Z})+I(\text{X};{\text{Y}}_1|V)$$

$$=\underset{\text{X}:\mathbb{E}\left[\text{X}{\text{X}}^{\text{T}}\right]\preccurlyeq {\text{K}}^{{}^{\prime }}}{\sup }\lambda I(X;\text{Z})+I(\text{X};\text{Y}|V)-\lambda I(\text{X};\text{Z}|V)$$

$$\le \underset{\text{X}:\mathbb{E}\left[\text{X}{\text{X}}^{\text{T}}\right]\preccurlyeq {\text{K}}^{{}^{\prime }}}{\sup }\lambda I(\text{X};\text{Z})+\underset{\begin{array}{c}V\to \text{X}\to (\text{Y},\text{Z})\\ \mathbb{E}\left[\text{X}{\text{X}}^{\text{T}}\right]\preccurlyeq {\text{K}}^{{}^{\prime }}\end{array}}{\sup }{\overrightarrow{s}}_{\lambda }^q(\text{X}|V),$$

$$\underset{\text{X}:\mathbb{E}\left[\text{X}{\text{X}}^{\text{T}}\right]\preccurlyeq {\text{K}}^{{}^{\prime }}}{\sup }\lambda I(\text{X};\text{Z})+\underset{\begin{array}{c}V\to \text{X}\to (\text{Y},\text{Z})\\ \mathbb{E}\left[\text{X}{\text{X}}^{\text{T}}\right]\preccurlyeq {\text{K}}^{{}^{\prime }}\end{array}}{\sup }{\overrightarrow{s}}_{\lambda }^q(\text{X}|V)$$

$$\displaystyle\leq\frac{1}{2}\log\Bigg{(}\frac{\left|{\textbf{B}_{\textbf{1}}+\textbf{K}}\right|}{\left|{\textbf{K}}\right|}\Bigg{)}+\frac{{\lambda}}{2}\log\Bigg{(}\frac{\left|{\textbf{B}_{\textbf{2}}+\textbf{B}_{\textbf{1}}+\textbf{K}+\widetilde{\textbf{K}}}\right|}{\left|{\textbf{B}_{\textbf{1}}+\textbf{K}+\widetilde{\textbf{K}}}\right|}\Bigg{)}.$$

$$I({\text{X}}_{\text{1}},{\text{X}}_{\text{2}}\to {\text{Y}}_{\text{1}},{\text{Y}}_{\text{2}})\le I({\text{X}}_{\text{1}}\to {\text{Y}}_{\text{1}})+I({\text{X}}_{\text{2}}\to {\text{Y}}_{\text{2}})$$

$$I({\text{X}}_{\text{1}},{\text{X}}_{\text{2}}\to {\text{Y}}_{\text{1}},{\text{Y}}_{\text{2}})-I({\text{X}}_{\text{1}};{\text{Y}}_{\text{1}})-I({\text{X}}_{\text{2}};{\text{Y}}_{\text{2}})$$

$$=I({\text{X}}_{\text{1}};{\text{Y}}_{\text{1}})+I({\text{X}}_{\text{1}},{\text{X}}_{\text{2}};{\text{Y}}_{\text{2}}|{\text{Y}}_{\text{1}})-I({\text{X}}_{\text{1}};{\text{Y}}_{\text{1}})-I({\text{X}}_{\text{2}};{\text{Y}}_{\text{2}})$$

$$=I({\text{X}}_{\text{1}},{\text{X}}_{\text{2}};{\text{Y}}_{\text{2}}|{\text{Y}}_{\text{1}})-I({\text{X}}_{\text{2}};{\text{Y}}_{\text{2}})$$

$$=h({\text{Y}}_{\text{2}}|{\text{Y}}_{\text{1}})-h({\text{Y}}_{\text{2}}|{\text{Y}}_{\text{1}},{\text{X}}_{\text{1}},{\text{X}}_{\text{2}})-h({\text{Y}}_{\text{2}})+h({\text{Y}}_{\text{2}}|{\text{X}}_{\text{2}})$$

Full text

I Introduction

The capacity region of a physically degraded (PD) discrete memoryless (DM) broadcast channel (BC) is not enlarged by the presence of perfect feedback from both the receivers, a result which was established by El Gamal [1]. The same conclusion was arrived at for PD Gaussian BCs with feedback by El Gamal [2]. The latter proof involved a variant of the Entropy Power Inequality (EPI), in order to incorporate the presence of feedback.

Evaluating the capacity regions or bounds to it in the additive Gaussian settings many-a-times involve the computation of extremal auxiliaries, for example, the broadcast channel [3]. The standard technique for proving the optimality of Gaussian auxiliaries in additive Gaussian noise settings is the EPI, or its variants. The application of EPI may not always be directly viable, and can be cumbersome, see [4]. Recently, Geng and Nair [5] proposed a technique for proving the optimality of Gaussian auxiliaries and used it to establish the capacity region of a two receiver Gaussian vector BC (GVBC) with both private and common messages. Their proof technique serves as a new tool for proving converses in Gaussian settings, instead of applying the traditional EPI.

In this paper, we extend the result in [2] for PD Gaussian BCs to the vector BC case. This is done by extending the technique of concave envelopes [5]. However this extension is not very straightforward. In particular, [5] requires the factorization of concave envelopes of multiletter mutual information terms. We show that this may not hold in the presence of feedback. Our major contribution is a reformulation of the concave envelopes in terms of directed information instead of mutual information. Interestingly, this factorization helps in concluding that the capacity region of a PD-GVBC is not enlarged by feedback. The novelty of the paper also lies in bringing out the fact that factorizing concave envelopes of directed information can handle situations involving feedback, where concave envelopes of mutual information are not factorizable.

We also show that feedback does not enlarge the capacity region of a reversely physically degraded DM broadcast channel (RPDBC), a result of independent interest. The paper is organized as follows. We introduce the system model in Section II. In Section III, we review the notion of concave envelopes in Geng and Nair’s context [5] and the notion of directed information. Section IV describes the feedback capacity region of a PD-GVBC. Section V deals with the feedback capacity region of a RPDBC. Finally Section VI concludes the paper.

II System Model

Consider the broadcast model in Fig 1. Here, $M_{1}$ and $M_{2}$ are two independent messages to the respective receivers. $M_{i}$ is uniformly distributed over $[1:2^{nR_{i}}],\,i\texttt{=}1,2$. The BC $q(\mathbf{y},\mathbf{z}{|}\mathbf{x})$ is physically degraded i.e. $q(\mathbf{y},\mathbf{z}{|}\mathbf{x})=p_{1}(\mathbf{y}{|}\mathbf{x})p_{2}(\mathbf{z}{|}\mathbf{y})$. More specifically, $\textbf{Y}=\textbf{X}+\textbf{N}$ and $\textbf{Z}=\textbf{Y}+\tilde{\textbf{N}}$, where N and $\tilde{\textbf{N}}$ are independent of each other, and of X. There is perfect feedback from both receivers to the encoder.

We will assume without loss of generality that all the random vectors are zero mean. Here $\mathbf{N}\sim\mathcal{N}(\mathbf{0},\mathbf{K})$ and $\tilde{\mathbf{N}}\sim\mathcal{N}(\mathbf{0},\tilde{\mathbf{K}})$. The input is subject to a covariance matrix constraint ${\mathbb{E}}\left[{\mathbf{X}}{{\mathbf{X}}^{\mathbf{T}}}\right]{\preccurlyeq}{{\mathbf{K}}^{{}^{\prime}}}$. A $(2^{nR_{1}},2^{nR_{2}},n,\lambda_{n})$ code for the channel is defined to be a pair of encoder mappings, $\mathbf{X}_{i}=f_{i}(M_{1},M_{2},\mathbf{Y}^{i-1},\mathbf{Z}^{i-1}),\,i\texttt{=}1,2$ and two decoder mappings $d_{i}:{\mathbf{\mathbb{R}}}^{n}\to[1:2^{nR_{i}}],i=1,2$. The probability of error $\lambda_{n}$ is given by

[TABLE]

[TABLE]

[TABLE]

A rate pair $(R_{1},R_{2})$ is said to be achievable if there exists a sequence of $(2^{nR_{1}},2^{nR_{2}},n,\lambda_{n})$ codes with $\lambda_{n}\to 0$ as $n\to\infty$. The capacity region $\mathcal{C}_{VBC}^{fb}$ is defined to be the closure of the set of achievable $(R_{1},R_{2})$ pairs.

El Gamal [1] showed that the capacity region of a physically degraded DM broadcast channel is not enlarged by feedback. Thus the capacity is given by superposition coding:

Theorem 1**.**

[1]** The capacity region of a DM physically degraded BC $q(y,z{|}x)$ with perfect feedback from both decoders is the set of $(R_{1},R_{2})$ pairs such that

[TABLE]

for some $p(u,x)$, where $U$ is an auxiliary random variable with $U\to X\to Y\to Z$, and $|\mathcal{U}|{\leq}\min{{(|\mathcal{X}|,|\mathcal{Y}|,|\mathcal{Z}|)}}+1$.

El Gamal [2] further showed that even in the scalar Gaussian case, feedback does not improve the capacity region if the channel is physically degraded. This result required an adaptation of the EPI.

As seen in Theorem 1, the capacity regions or bounds thereof are often represented in terms of auxiliary random variables. The evaluation of these regions reduce to optimization problems over the joint distribution of input and auxiliary random variables. In Gaussian problems, proving the optimality of Gaussian auxiliaries generally involves an application of the EPI, or its variants [3]. The application of EPI can be cumbersome in some cases. For instance, the converse for the Gaussian Multiple Input Multiple Output (MIMO) BC problem required the introduction of the notion of an enhanced channel [4] in order to make the application of EPI viable. However, even the result of [4] cannot handle a vector Gaussian BC with both private and common messages. The latter problem was recently solved by Geng and Nair [5] by the method of factorization of concave envelopes. But the concave envelopes in [5] do not factorize in the presence of feedback. We reformulate the factorization of concave envelopes in terms of directed information to yield the capacity region of a PD-GVBC with feedback. Our main result is the following theorem:

Theorem 2**.**

The capacity region $\mathcal{C}_{VBC}^{fb}$ of a PD-GVBC with perfect feedback from both the receivers to the transmitter is the union of the set of $(R_{1},R_{2})$ pairs such that

[TABLE]

for some $\textbf{B}_{\textbf{1}},\textbf{B}_{\textbf{2}}\succcurlyeq 0$, with $\textbf{B}_{\textbf{1}}+\textbf{B}_{\textbf{2}}\>{\preccurlyeq}\>\textbf{K}^{{}^{\prime}}$.

Note that the region mentioned above is exactly the region without feedback.

III Concave Envelopes and Directed Information

Let us first review the technique of concave envelopes introduced in [5]. We follow the notation used in that work. Recall that the upper concave envelope of a function $f(x)$ is the smallest concave function $g(x)$ such that $g(x)\geq f(x)$ throughout the domain of $f(x)$. Equivalently, $g(x)$ can be expressed as [5]

[TABLE]

For a BC $q(\mathbf{y},\mathbf{z}{|}\mathbf{x})$, and $\lambda>1$, define the following function of $p(\mathbf{x})$ (the input distribution)

[TABLE]

For $(V,\mathbf{X})$ such that $V\to\mathbf{X}\to(\mathbf{Y},\mathbf{Z})$, define

[TABLE]

Now we denote the upper concave envelope of $s_{\lambda}^{q}(\mathbf{X})$ as

[TABLE]

Applying the definition from Equation (8), we get

[TABLE]

Define a conditional version of the concave envelope

[TABLE]

Since $S_{\lambda}^{q}(\mathbf{X})$ is concave in $p(\mathbf{x})$, by Jensen’s inequality

[TABLE]

For a two-letter broadcast channel $q(\mathbf{y}_{1},\mathbf{z}_{1}{|}\mathbf{x}_{1}){\times}q(\mathbf{y}_{2},\mathbf{z}_{2}{|}\mathbf{x}_{2})$, define similarly a function of $p(\mathbf{x}_{1},\mathbf{x}_{2})$

[TABLE]

Similar definitions apply for the quantities $s_{\lambda}^{q{\times}q}(\mathbf{X}_{1},\mathbf{X}_{2}{|}V)$, $S_{\lambda}^{q{\times}q}(\mathbf{X}_{1},\mathbf{X}_{2})$ and $S_{\lambda}^{q{\times}q}(\mathbf{X}_{1},\mathbf{X}_{2}{|}W)$.

The key technique of [5] was to show that the concave envelope defined over the joint distribution $p(\mathbf{x}_{1},\mathbf{x}_{2})$ of a product broadcast channel $q(\mathbf{y}_{1},\mathbf{z}_{1}{|}\mathbf{x}_{1}){\times}q(\mathbf{y}_{2},\mathbf{z}_{2}{|}\mathbf{x}_{2})$ satisfies the following subadditivity property:

[TABLE]

This in turn leads to the optimality of Gaussian auxiliary random variables in establishing the capacity region of a GVBC.

However, the subadditivity mentioned above may not hold in the presence of feedback. This can be seen by considering an example of a single user channel from $\textbf{X}\to\textbf{Z}$. Consider a two letter channel $p(\textbf{z}_{1}{|}\textbf{x}_{1})\times p(\textbf{z}_{2}{|}\textbf{x}_{2})$. Notice that, due to feedback, though $\textbf{X}_{1}\to\textbf{X}_{2}\to\textbf{Z}_{2}$ holds, $\textbf{Z}_{1}\to\textbf{X}_{1}\to\textbf{X}_{2}$ may not. Hence the term $I(\textbf{X}_{2};\textbf{Z}_{1}{|}\textbf{X}_{1})$ may not be zero and consequently, the subadditivity $I(\textbf{X}_{1},\textbf{X}_{2};\textbf{Z}_{1},\textbf{Z}_{2})\leq I(\textbf{X}_{1};\textbf{Z}_{1})+I(\textbf{X}_{2};\textbf{Z}_{2})$ is not guaranteed to hold. Thus the technique of [5] as such does not extend to the case with feedback.

The main contribution of the current work is the formulation of concave envelopes in terms of directed information, which admit a convenient factorization/subadditivity property over product channels. This in turn is used to prove the optimality of Gaussian auxiliary random variables, establishing the capacity region of a PD-GVBC with feedback. Recall that the notion of directed information between two sequences, introduced by Massey is [6]

[TABLE]

Note that $I(X^{N}\to Y^{N})\neq I(Y^{N}\to X^{N})$ in general. The following lemma relates directed information and mutual information.

Lemma 3**.**

[6]** If $X^{N}$ is the input and $Y^{N}$ is the output of a channel, then

[TABLE]

with equality if and only if the channel is memoryless and used without feedback.

IV PD-GVBC with feedback

In this Section, we denote $s_{\lambda}^{q}(\mathbf{X})$ by $\overrightarrow{s}_{\lambda}^{q}(\mathbf{X})$ and $S_{\lambda}^{q}(\mathbf{X})$ by $\overrightarrow{S}_{\lambda}^{q}(\mathbf{X})$ to emphasize the fact that we are now replacing mutual information by directed information. Similarly, for product channels,

[TABLE]

Let the covariance matrix constraint on the input be ${\mathbb{E}}\left[{\mathbf{X}}{{\mathbf{X}}^{\mathbf{T}}}\right]{\preccurlyeq}{{\mathbf{K}}^{{}^{\prime}}}$. Let us call the region in Theorem 2 as $\mathcal{I}_{{{\mathbf{K}}^{{}^{\prime}}}}$. Clearly $\mathcal{I}_{{{\mathbf{K}}^{{}^{\prime}}}}$ can be achieved by ignoring the feedback and using superposition coding. On the other hand, Theorem 1 suggests that any achievable rate pair $(R_{1},R_{2})\in\mathcal{O}_{{{\mathbf{K}}^{{}^{\prime}}}}$, where

[TABLE]

Here the union is taken over all $p(v,\mathbf{x})$ such that $p(\mathbf{x})$ meets the covariance constraint ${{\mathbf{K}}^{{}^{\prime}}}$, and $V\to\mathbf{X}\to(\mathbf{Y},\mathbf{Z})$. We now focus on showing that ${\mathcal{O}}_{\textbf{K}^{{}^{\prime}}}\subseteq{\mathcal{I}}_{\textbf{K}^{{}^{\prime}}}$. Since both $\mathcal{I}_{{{\mathbf{K}}^{{}^{\prime}}}}$ and $\mathcal{O}_{{{\mathbf{K}}^{{}^{\prime}}}}$ are closed convex sets, they can be expressed as an intersection of supporting hyperplanes. Recall that any closed convex set $\mathcal{S}$ belonging to the positive quadrant can be expressed as

[TABLE]

where

[TABLE]

Using the characterization in equations (20) and (21), it suffices to show that

[TABLE]

From the LHS of (22)

[TABLE]

where the first inequality follows from the outer bound in expression (19) and the last step follows by the definition of $\overrightarrow{s}_{\lambda}^{q}({\textbf{X}}|V)$. We can now prove Theorem 2 by establishing that

[TABLE]

We now propose some factorization properties that hold for product DM channels with feedback.

Property 4**.**

Consider a DM channel $q(\textbf{y}{|}\textbf{x})$ and its two letter extension $q(\textbf{y}_{\textbf{1}}{|}\textbf{x}_{\textbf{1}})\times q(\textbf{y}_{\textbf{2}}{|}\textbf{x}_{\textbf{2}})$. Then the following holds

[TABLE]

Further, equality is attained when $\textbf{Y}_{\textbf{1}}$ and $\textbf{Y}_{\textbf{2}}$ are independent.

Proof.

[TABLE]

The equality is attained when $h(\textbf{Y}_{\textbf{2}}{|}\textbf{Y}_{\textbf{1}})=h(\textbf{Y}_{\textbf{2}})$. ∎

Thus the factorization inequality holds true for a single user channel with feedback. Now we show the factorization of concave envelopes for a two letter broadcast channel. The following lemma holds for such product broadcast channels.

Property 5**.**

For a two letter broadcast channel $q(\textbf{y}_{\textbf{1}},\textbf{z}_{\textbf{1}}{|}\textbf{x}_{\textbf{1}}){\times}q(\textbf{y}_{\textbf{2}},\textbf{z}_{\textbf{2}}{|}\textbf{x}_{\textbf{2}})$,

[TABLE]

Proof.

For any $(V,\textbf{X}_{\textbf{1}},\textbf{X}_{\textbf{2}})$ such that $V\to(\textbf{X}_{\textbf{1}},\textbf{X}_{\textbf{2}})\to(\textbf{Y}_{\textbf{1}},\textbf{Y}_{\textbf{2}},\textbf{Z}_{\textbf{1}},\textbf{Z}_{\textbf{2}})$, we have

[TABLE]

where (a) and (d) follow from the definition of concave envelope, (b) follows from the Markov chain $\textbf{X}_{\textbf{1}}\to\textbf{X}_{\textbf{2}}\to(\textbf{Y}_{\textbf{2}},\textbf{Z}_{\textbf{2}})$, (c) follows from the fact that $\textbf{Z}_{\textbf{1}}$ is a degraded version of $\textbf{Y}_{\textbf{1}}$ and (e) follows from concavity. Now optimizing over all distributions $p(v|\mathbf{x}_{1},\mathbf{x}_{2})$ completes the proof. ∎

Thus we have shown that the concave envelopes in terms of directed information satisfy the factorization inequality (25), (26). This in turn will be used to prove the optimality of Gaussian auxiliaries in achieving the capacity region of a PD-GVBC with feedback.

We also require the following property regarding unitary transformations on a two-letter Gaussian channel.

Property 6**.**

Consider a two letter Gaussian channel with feedback

[TABLE]

where $\textbf{N}_{\textbf{1}}$ and $\textbf{N}_{\textbf{2}}$ are independent and distributed as $\mathcal{N}(\textbf{0},{{\textbf{K}}^{{}^{\prime}}})$. Define

[TABLE]

Then $I(\textbf{X}_{\textbf{1}},\textbf{X}_{\textbf{2}}\to\textbf{Y}_{\textbf{1}},\textbf{Y}_{\textbf{2}})=I(\textbf{U}_{\textbf{1}},\textbf{U}_{\textbf{2}}\to\textbf{V}_{\textbf{1}},\textbf{V}_{\textbf{2}})$.

Proof.

Take $\textbf{W}_{\textbf{1}}\texttt{=}\frac{1}{\sqrt{2}}(\textbf{N}_{\textbf{1}}+\textbf{N}_{\textbf{2}})$ and $\textbf{W}_{\textbf{2}}\texttt{=}\frac{1}{\sqrt{2}}(\textbf{N}_{\textbf{1}}-\textbf{N}_{\textbf{2}})$.

[TABLE]

where (a), (b) and (c) follow since, given $\textbf{X}_{\textbf{2}}$, the uncertainty in $\textbf{Y}_{\textbf{2}}$ is only due to $\textbf{N}_{\textbf{2}}$ and, for Gaussian noise channels, $\textbf{W}_{\textbf{1}}$ and $\textbf{W}_{\textbf{2}}$ are i.i.d. with the same distribution as $\textbf{N}_{\textbf{1}}$. ∎

Now define

[TABLE]

By Proposition $7$ of [5], there exists a pair of random variables $(V^{\dagger},\textbf{X}^{\dagger})$ with $\left|{{\mathcal{V}}^{\dagger}}\right|\leq 1+\frac{d(d+1)}{2}$ and ${\mathbb{E}}\left[{\textbf{X}}^{\dagger}{{\textbf{X}^{\dagger}}^{\textbf{T}}}\right]{\preccurlyeq}{\textbf{K}^{{}^{\prime}}}$ such that (with $d$ being the dimension of all the vectors)

[TABLE]

The following lemma helps in identifying the optimal distribution that achieves $V_{\lambda}^{q}({\textbf{K}^{{}^{\prime}}})$.

Lemma 7**.**

Let $(V^{\dagger},\textbf{X}^{\dagger})\sim p^{\dagger}(v,\textbf{x})$ attain $V_{\lambda}^{q}({\textbf{K}^{{}^{\prime}}})$. Let $\textbf{X}_{\textbf{v}}$ be distributed according to the conditional law $p^{\dagger}(\textbf{X}{|}V=v)$. Let $(V_{1},V_{2},\textbf{X}_{\textbf{1}},\textbf{X}_{\textbf{2}})\sim p(v_{1},v_{2},\textbf{x}_{\textbf{1}},\textbf{x}_{\textbf{2}})$ be such that the marginals $p^{\dagger}(v_{1},\textbf{x}_{\textbf{1}})$ and $p^{\dagger}(v_{2},\textbf{x}_{\textbf{2}})$ attain $V_{\lambda}^{q}({\textbf{K}^{{}^{\prime}}})$. Define

[TABLE]

Then $\textbf{U}_{\textbf{1}}$ and $\textbf{U}_{\textbf{2}}$ attain $V_{\lambda}^{q}({\textbf{K}^{{}^{\prime}}})$.

Proof.

Consider the two letter broadcast channel $q(\textbf{y}_{\textbf{1}},\textbf{z}_{\textbf{1}}{|}\textbf{x}_{\textbf{1}}){\times}q(\textbf{y}_{\textbf{2}},\textbf{z}_{\textbf{2}}{|}\textbf{x}_{\textbf{2}})$. Let $\textbf{Y}^{a}_{\textbf{1}}{|}((V_{1},V_{2})=(v_{1},v_{2}))\sim\frac{1}{\sqrt{2}}({\textbf{Y}}_{v_{1}}+{\textbf{Y}}_{v_{2}})$, $\textbf{Y}^{a}_{\textbf{2}}{|}((V_{1},V_{2})=(v_{1},v_{2}))\sim\frac{1}{\sqrt{2}}({\textbf{Y}}_{v_{1}}-{\textbf{Y}}_{v_{2}})$, $\textbf{Z}^{a}_{\textbf{1}}{|}((V_{1},V_{2})=(v_{1},v_{2}))\sim\frac{1}{\sqrt{2}}({\textbf{Z}}_{v_{1}}+{\textbf{Z}}_{v_{2}})$ and $\textbf{Z}^{a}_{\textbf{2}}{|}((V_{1},V_{2})=(v_{1},v_{2}))\sim\frac{1}{\sqrt{2}}({\textbf{Z}}_{v_{1}}-{\textbf{Z}}_{v_{2}})$. We have

[TABLE]

where (a) follows from Property 6 and

[TABLE]

(b) follows by the definition of concave envelope, (c) follows from Property 5 and (d) follows by definition of $V_{\lambda}^{q}({\textbf{K}^{{}^{\prime}}})$. Since the extremes match, all inequalities are equalities. In particular, (d) is an equality. Thus $\textbf{U}_{\textbf{1}}$ and $\textbf{U}_{\textbf{2}}$ attain $V_{\lambda}^{q}({\textbf{K}^{{}^{\prime}}})$. ∎

The fact in Lemma 7 allows us to identify that Gaussian auxiliaries are indeed optimal. This is stated in the form of the following theorem.

Theorem 8**.**

There exists $\textbf{X}^{\dagger}\sim\mathcal{N}(\textbf{0},\textbf{K}^{\dagger})$, $\textbf{K}^{\dagger}{\preccurlyeq}{\textbf{K}^{{}^{\prime}}}$ such that $V_{\lambda}^{q}({\textbf{K}^{{}^{\prime}}})=\overrightarrow{s}_{\lambda}^{q}(\textbf{X}^{\dagger})$.

Proof.

From Lemma 7, ${\textbf{X}}_{{v_{1}}}+{\textbf{X}}_{{v_{2}}}$ attains $V_{\lambda}^{q}({\textbf{K}^{{}^{\prime}}})$. Thus for any optimal distribution $p_{o}$, the distribution $p_{o}*p_{o}$ works equally well. Continuing this argument, by induction, we can use the Central Limit Theorem to conclude that Gaussian is a maximizer. Let $\textbf{X}^{\dagger}\sim\mathcal{N}(\textbf{0},\textbf{K}^{\dagger})$ be the maximizer, where $\textbf{K}^{\dagger}{\preccurlyeq}{\textbf{K}^{{}^{\prime}}}$. Note that since $v_{1}$ and $v_{2}$ were arbitrary, all the $\textbf{X}_{v}$’s have the same covariance matrix, namely $\textbf{K}^{\dagger}$. ∎

We now complete the proof of Theorem 2 using Theorem 8.

IV-A Proof of Theorem 2

In expression (IV), the first term on the LHS is maximized by $\textbf{X}\sim\mathcal{N}(\textbf{0},{\textbf{K}^{{}^{\prime}}})$. The second term is maximized by $\textbf{U}={\textbf{X}}^{\dagger}\sim\mathcal{N}(\textbf{0},\textbf{K}^{\dagger})$, where $\textbf{K}^{\dagger}{\preccurlyeq}{\textbf{K}^{{}^{\prime}}}$ by Theorem 8. Now let $\textbf{X}={\textbf{X}}^{\dagger}+{\textbf{X}}^{\ast}=\textbf{U}+\textbf{V}$, where $\textbf{U}={\textbf{X}}^{\dagger}\sim\mathcal{N}(\textbf{0},{\textbf{K}}^{\dagger})$ and $\textbf{V}={\textbf{X}}^{\ast}\sim\mathcal{N}(\textbf{0},({\textbf{K}^{{}^{\prime}}}-{\textbf{K}}^{\dagger}))$ are independent. This gives $\textbf{X}\sim\mathcal{N}(\textbf{0},{\textbf{K}^{{}^{\prime}}})$ as desired. Evaluating $I(\textbf{X};\textbf{Z})$ and $\overrightarrow{s}_{\lambda}^{q}(\textbf{X}{|}V)$ with these choices, it can be easily checked that (IV) is indeed satisfied. Hence we have proved the reverse inclusion ${\mathcal{O}}_{{\textbf{K}^{{}^{\prime}}}}\subseteq{\mathcal{I}}_{{\textbf{K}^{{}^{\prime}}}}$. This establishes that the capacity region of a PD-GVBC is not enlarged by feedback. We have also proved that a superposition of Gaussian codes is optimal.

Remark 9**.**

Although our proofs were given for the model $\mathbf{Y}=\mathbf{X}+\mathbf{N}$ and $\mathbf{Z}=\mathbf{Y}+\widetilde{\mathbf{N}}$, it can be easily verified that the proofs go through for a more general model $\mathbf{Y}=G\mathbf{X}+\mathbf{N}$ and $\mathbf{Z}=\mathbf{Y}+\widetilde{\mathbf{N}}$ (where $G$ is an invertible matrix) as well. This justifies our terminology of Gaussian Vector BC. The capacity region in this more general case is given in the form of the following theorem:

Theorem 10**.**

The capacity region $\mathcal{C}_{MVBC}^{fb}$ of a MIMO PD-GBC with perfect feedback from both the receivers to the transmitter is the union of the set of $(R_{1},R_{2})$ pairs such that

[TABLE]

for some $\textbf{B}_{\textbf{1}},\textbf{B}_{\textbf{2}}\succcurlyeq 0$, with $\textbf{B}_{\textbf{1}}+\textbf{B}_{\textbf{2}}\>{\preccurlyeq}\>\textbf{K}^{{}^{\prime}}$.

V Feedback Capacity of Reversely PD DMBC

We now extend our results to a reversely physically degraded DM broadcast channel (RPDBC). The model is as shown in Fig. 2. As seen in the figure, the terminology RPDBC is used to refer to a product of two (inconsistently) degraded DM broadcast channels. El Gamal [7] derived the capacity region of a RPDBC with both private and common messages in the absence of feedback. In general, while the effect of feedback on a RPDBC has not been studied so far in literature, we show that feedback does not enlarge the capacity region of a RPDBC. This enables our results from the previous sections to be adapted so as to establish the capacity region of a Gaussian reversely PD vector BC with feedback. Here, we focus on deriving the discrete memoryless result. The definition of a code, achievable rate and capacity region are similar to [7], except that we take

[TABLE]

for some maps $g_{1}(\cdot)$ and $g_{2}(\cdot)$. Let us denote the capacity region of a RPDBC with both private and common messages with perfect noiseless feedback from all the outputs by $\mathcal{C}_{RBC}^{fb}$.

Theorem 11**.**

[TABLE]

where

[TABLE]

with the union taken over all $p(u_{1},x_{1})p(u_{2},x_{2})$ such that $U_{1}\to X_{1}\to Y_{1}\to Z_{1}$ and $U_{2}\to X_{2}\to Z_{2}\to Y_{2}$, and $|\mathcal{U}_{i}|\leq\min(|\mathcal{X}_{i}|,|\mathcal{Y}_{i}|,|\mathcal{Z}_{i}|),i=1,2$.

Note that the region mentioned above is exactly the region without feedback, which was established in [7].

Proof.

The achievability follows from [7] by ignoring the feedback. Since the steps in the converse proof are similar to [7], we only highlight the differences. It turns out that employing the following auxiliaries

[TABLE]

in the converse proof of [7] suffices to obtain the theorem. Observe that the above choice of auxiliaries ensures both the Markov conditions $U_{1i}\to X_{1i}\to Y_{1i}\to Z_{1i}$ and $U_{2i}\to X_{2i}\to Z_{2i}\to Y_{2i}$. These conditions are not true for the original auxiliaries in [7].

We show the single-letterization of (42), others follow in a similar fashion. Also, we suppress the $n\epsilon_{n}$ terms while applying Fano’s inequality in the expression below.

[TABLE]

The single letterization is completed by letting $n\to\infty$ which makes $\epsilon_{n}\to 0$ and noting that the region in Theorem 11 is convex. ∎

VI Conclusion

We proved that feedback does not enlarge the capacity region of a physically degraded Gaussian vector BC. Our proof does not rely on the use of the EPI or any of its variants, and makes use of factorization of concave envelopes in terms of directed information. It illustrates the utility of the technique of factorization of concave envelopes as a tool for proving converses in Gaussian settings, where previously the application of EPI was the main bottleneck. Our work also brings out the fact that factorizing concave envelopes of directed information can handle situations involving feedback, where concave envelopes of mutual information are not factorizable. We also proved that the capacity region of a reversely physically degraded DM broadcast channel is not enlarged by feedback.

Acknowledgements

The work was supported in part by the Bharti Centre for Communication, IIT Bombay and in part by the Department of Science and Technology (DST), Government of India, under grant SB/S3/EECE/077/2013.