Sum-set Inequalities from Aligned Image Sets: Instruments for Robust GDoF Bounds

TL;DR

This paper introduces new sum-set inequalities derived from aligned image sets, providing robust tools for deriving GDoF bounds in complex wireless networks with multiple antennas and uncertain channel conditions.

Contribution

It generalizes the aligned image sets approach to create sum-set inequalities that aid in GDoF analysis for multi-antenna wireless networks with channel uncertainty.

Findings

Derived tight GDoF bounds for a two-user interference channel with multiple antennas.

Generalized sum-set inequalities applicable to various wireless network configurations.

Provided a new information-theoretic framework for analyzing channel uncertainty effects.

Abstract

We present sum-set inequalities specialized to the generalized degrees of freedom (GDoF) framework. These are information theoretic lower bounds on the entropy of bounded density linear combinations of discrete, power-limited dependent random variables in terms of the joint entropies of arbitrary linear combinations of new random variables that are obtained by power level partitioning of the original random variables. These bounds generalize the aligned image sets approach, and are useful instruments to obtain GDoF characterizations for wireless networks, especially with multiple antenna nodes, subject to arbitrary channel strength and channel uncertainty levels. To demonstrate the utility of these bounds, we consider a non-trivial instance of wireless networks - a two user interference channel with different number of antennas at each node, and different levels of partial channel knowledge available to the transmitters. We obtain tight GDoF characterization for specific instance of this channel with the aid of sum-set inequalities.

Figures

Equations

$${\mathcal{X}}_{{\lambda }_i}$$

$$(X{)}_{{\lambda }_1}$$

$$(X{)}_{{\lambda }_1}^{{\lambda }_2}$$

$$(\mathbf{V}{)}_{{\lambda }_1}$$

$$(\mathbf{V}{)}_{{\lambda }_1}^{{\lambda }_2}$$

$$L_j^b(x_1,x_2,\cdots ,x_k)$$

$$L_j(x_1,x_2,\cdots ,x_k)$$

$$L_j^{b\vec{\gamma }\vec{\delta }}(x_1,x_2,\cdots ,x_k)$$

$$L_j^{\vec{\gamma }\vec{\delta }}(x_1,x_2,\cdots ,x_k)$$

$$L_j^b(V)=\sum _{1\le i\le k}\lfloor g_{j_i}{v}_i\rfloor$$

$$L_j(V)=\sum _{1\le i\le k}\lfloor h_{j_i}{v}_i\rfloor$$

$$\mathcal{T}(A)=\mathcal{T}(B)=\underset{j\in [k]}{\max }\min ({\eta }_j,({\gamma }_j-{\delta }_j{)}^{+}).$$

$$A$$

$$B$$

$$V_{m,n}$$

$$Z$$

$$Z_1$$

$$Z_2$$

$$H(Z\mid \mathcal{G})$$

$$H(Z\mid \mathcal{G},W)$$

$$H(Z|\mathcal{G})$$

$$H\left(\text{leavevmode}to72.44pt\text{vbox}to95.37pt\text{pgfpicture}\text{makeatletter}\text{lower}-20.96512ptto0.0pt\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.4pt\text{pgfsys@invoke}\text{nullfont}to0.0pt\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{0,0,1}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}8001\text{pgfsys@moveto}0.0pt0.0pt\text{pgfsys@moveto}0.0pt0.0pt\text{pgfsys@lineto}0.0pt33.28957pt\text{pgfsys@lineto}25.6073pt33.28957pt\text{pgfsys@lineto}25.6073pt0.0pt\text{pgfsys@closepath}\text{pgfsys@moveto}25.6073pt33.28957pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{0,0,1}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}6001\text{pgfsys@moveto}0.0pt33.28957pt\text{pgfsys@moveto}0.0pt33.28957pt\text{pgfsys@lineto}0.0pt58.89687pt\text{pgfsys@lineto}25.6073pt58.89687pt\text{pgfsys@lineto}25.6073pt33.28957pt\text{pgfsys@closepath}\text{pgfsys@moveto}25.6073pt58.89687pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{1,0,0}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}8100\text{pgfsys@moveto}25.6073pt0.0pt\text{pgfsys@moveto}25.6073pt0.0pt\text{pgfsys@lineto}25.6073pt33.28957pt\text{pgfsys@lineto}51.21461pt33.28957pt\text{pgfsys@lineto}51.21461pt0.0pt\text{pgfsys@closepath}\text{pgfsys@moveto}51.21461pt33.28957pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{1,0,0}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}6100\text{pgfsys@moveto}25.6073pt33.28957pt\text{pgfsys@moveto}25.6073pt33.28957pt\text{pgfsys@lineto}25.6073pt58.89687pt\text{pgfsys@lineto}51.21461pt58.89687pt\text{pgfsys@lineto}51.21461pt33.28957pt\text{pgfsys@closepath}\text{pgfsys@moveto}51.21461pt58.89687pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{pgfsys@moveto}56.33598pt0.71pt\text{pgfsys@lineto}56.33598pt32.57956pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.0-1.01.00.056.33598pt0.71pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.01.0-1.00.056.33598pt32.57956pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.060.06898pt14.07478pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{\lambda }_1\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{pgfsys@moveto}56.33598pt33.99957pt\text{pgfsys@lineto}56.33598pt58.18686pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.0-1.01.00.056.33598pt33.99957pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.01.0-1.00.056.33598pt58.18686pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.060.06898pt43.52322pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{\lambda }_2\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{pgfsys@moveto}0.59999pt-5.12137pt\text{pgfsys@lineto}50.61462pt-5.12137pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}-1.00.00.0-1.00.59999pt-5.12137pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.050.61462pt-5.12137pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.010.5654pt-17.63211pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}Z=L^b\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@moveto}12.80365pt58.89687pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.06.86894pt64.23431pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{X}_1\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@moveto}38.41095pt58.89687pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.032.47624pt64.23431pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{X}_2\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{hss}\text{pgfsys@discardpath}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{hss}\text{lxSVG@closescope}\text{endpgfpicture}\right)$$

$$Z$$

$$Z_1$$

$$Z_2$$

$$Z_l$$

$$m(a)$$

$${\lambda }_1+{\lambda }_2+\cdots +{\lambda }_{(m(s)-1)}$$

$$H(Z\mid \mathcal{G},W)$$

$$H\left(\text{leavevmode}to59.92pt\text{vbox}to214.58pt\text{pgfpicture}\text{makeatletter}\text{lower}-19.82704ptto0.0pt\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.4pt\text{pgfsys@invoke}\text{nullfont}to0.0pt\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{0,0,1}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}8001\text{pgfsys@moveto}0.0pt0.0pt\text{pgfsys@moveto}0.0pt0.0pt\text{pgfsys@lineto}0.0pt59.75052pt\text{pgfsys@lineto}19.91684pt59.75052pt\text{pgfsys@lineto}19.91684pt0.0pt\text{pgfsys@closepath}\text{pgfsys@moveto}19.91684pt59.75052pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{0,0,1}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}6001\text{pgfsys@moveto}0.0pt59.75052pt\text{pgfsys@moveto}0.0pt59.75052pt\text{pgfsys@lineto}0.0pt99.58421pt\text{pgfsys@lineto}19.91684pt99.58421pt\text{pgfsys@lineto}19.91684pt59.75052pt\text{pgfsys@closepath}\text{pgfsys@moveto}19.91684pt99.58421pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{0,0,1}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}5001\text{pgfsys@moveto}0.0pt99.58421pt\text{pgfsys@moveto}0.0pt99.58421pt\text{pgfsys@lineto}0.0pt149.37633pt\text{pgfsys@lineto}19.91684pt149.37633pt\text{pgfsys@lineto}19.91684pt99.58421pt\text{pgfsys@closepath}\text{pgfsys@moveto}19.91684pt149.37633pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{0,0,1}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}2001\text{pgfsys@moveto}0.0pt149.37633pt\text{pgfsys@moveto}0.0pt149.37633pt\text{pgfsys@lineto}0.0pt179.25159pt\text{pgfsys@lineto}19.91684pt179.25159pt\text{pgfsys@lineto}19.91684pt149.37633pt\text{pgfsys@closepath}\text{pgfsys@moveto}19.91684pt179.25159pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{1,0,0}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}8100\text{pgfsys@moveto}19.91684pt0.0pt\text{pgfsys@moveto}19.91684pt0.0pt\text{pgfsys@lineto}19.91684pt59.75052pt\text{pgfsys@lineto}39.83368pt59.75052pt\text{pgfsys@lineto}39.83368pt0.0pt\text{pgfsys@closepath}\text{pgfsys@moveto}39.83368pt59.75052pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{1,0,0}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}6100\text{pgfsys@moveto}19.91684pt59.75052pt\text{pgfsys@moveto}19.91684pt59.75052pt\text{pgfsys@lineto}19.91684pt99.58421pt\text{pgfsys@lineto}39.83368pt99.58421pt\text{pgfsys@lineto}39.83368pt59.75052pt\text{pgfsys@closepath}\text{pgfsys@moveto}39.83368pt99.58421pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{1,0,0}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}5100\text{pgfsys@moveto}19.91684pt99.58421pt\text{pgfsys@moveto}19.91684pt99.58421pt\text{pgfsys@lineto}19.91684pt149.37633pt\text{pgfsys@lineto}39.83368pt149.37633pt\text{pgfsys@lineto}39.83368pt99.58421pt\text{pgfsys@closepath}\text{pgfsys@moveto}39.83368pt149.37633pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{definecolor}[named]pgf@tempcolorrgb\mathrm{1,0,0}\text{pgfsys@invoke}\text{lxSVG@setpatternuncolored@}2100\text{pgfsys@moveto}19.91684pt149.37633pt\text{pgfsys@moveto}19.91684pt149.37633pt\text{pgfsys@lineto}19.91684pt179.25159pt\text{pgfsys@lineto}39.83368pt179.25159pt\text{pgfsys@lineto}39.83368pt149.37633pt\text{pgfsys@closepath}\text{pgfsys@moveto}39.83368pt179.25159pt\text{pgfsys@fillstroke}\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{pgfsys@moveto}43.81699pt0.71pt\text{pgfsys@lineto}43.81699pt59.04051pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.0-1.01.00.043.81699pt0.71pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.01.0-1.00.043.81699pt59.04051pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.047.54999pt27.30527pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{\lambda }_1\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{pgfsys@moveto}43.81699pt60.46053pt\text{pgfsys@lineto}43.81699pt98.8742pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.0-1.01.00.043.81699pt60.46053pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.01.0-1.00.043.81699pt98.8742pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.047.54999pt77.09738pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{\lambda }_2\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{pgfsys@moveto}43.81699pt100.29422pt\text{pgfsys@lineto}43.81699pt148.66632pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.0-1.01.00.043.81699pt100.29422pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.01.0-1.00.043.81699pt148.66632pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.047.54999pt121.91028pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{\lambda }_3\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{pgfsys@moveto}43.81699pt150.08633pt\text{pgfsys@lineto}43.81699pt178.54158pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.0-1.01.00.043.81699pt150.08633pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}0.01.0-1.00.043.81699pt178.54158pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.047.54999pt161.74397pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{\lambda }_4\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@setlinewidth}0.8pt\text{pgfsys@invoke}\text{pgfsys@moveto}0.59999pt-3.98329pt\text{pgfsys@lineto}39.23369pt-3.98329pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}-1.00.00.0-1.00.59999pt-3.98329pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.039.23369pt-3.98329pt\text{pgfsys@invoke}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.04.87494pt-16.49403pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}Z=L^b\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@moveto}9.95842pt179.25159pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.04.02371pt184.58904pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{X}_1\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@moveto}29.87526pt179.25159pt\text{pgfsys@stroke}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@beginscope}\text{pgfsys@invoke}\text{pgfsys@transformcm}1.00.00.01.023.94055pt184.58904pt\text{pgfsys@invoke}\text{definecolor}pgfstrokecolorrgb\mathrm{0,0,0}\text{pgfsys@color@rgb@stroke}000\text{pgfsys@invoke}\text{pgfsys@color@rgb@fill}000\text{pgfsys@invoke}{X}_2\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{hss}\text{pgfsys@discardpath}\text{pgfsys@invoke}\text{lxSVG@closescope}\text{pgfsys@endscope}\text{hss}\text{lxSVG@closescope}\text{endpgfpicture}\right)$$

$$Z(t)$$

$$Z_1(t)$$

$$Z_2(t)$$

$$Z_l(t)$$

$$H(Z^{[n]}\mid W,\mathcal{G})$$

$$\mathcal{T}(Z_{s+1}(t))+\mathcal{T}(Z_{s+2}(t))+\cdots +\mathcal{T}(Z_l(t))$$

Full text

1 Introduction

Originating in additive combinatorics, sum-set inequalities are bounds on the cardinalities of sum-sets (given $X_{1},X_{2}$, the sumset $X_{1}+X_{2}\triangleq\{x_{1}+x_{2}:x_{1}\in X_{1},x_{2}\in X_{2}\}$). Crossing over to network information theory, sum-set inequalities represent bounds on the entropies of sums of random variables, typically expressed in terms of the entropies of the constituent random variables. Prominent examples of such inequalities include Ruzsa’s sum-triangle inequality in additive combinatorics [1] and the entropy power inequality in information theory [2]. Sum-set inequalities are essential to the study of the capacity of wireless interference networks. This is particularly true for the studies of capacity approximations known as generalized degrees of freedom (GDoF) [3] through deterministic models [4] which de-emphasize the additive noise to place the focus exclusively on the interactions between signals. Received signals in wireless networks are comprised of sums (more generally, linear combinations) of codewords from various codebooks, sent from various transmitters. GDoF optimal schemes seek to maximize the entropy of received linear combinations of signals where they are desired, while simultaneously minimizing the entropy of received linear combinations of the same signals where they are undesired (e.g., by zero-forcing or interference alignment [5]). The fundamental constraints on the structure of sum-sets, as revealed by sum-set inequalities are therefore the critical determinants of the GDoF of wireless interference networks. However, in spite of much recent progress in translating sum-set inequalities from additive combinatorics to network information theory [6], the structure of sum sets remains scarcely understood, and continues to be an impediment for GDoF characterizations. In fact, the intricacies of the sum-set structure are such that even a coarse metric like the degrees of freedom (DoF) for constant channel realizations turns out to be sensitive to fragile details of no conceivable practical relevance – e.g., whether the channel coefficients take rational or irrational values [7].

Useful insights need robust models and metrics which respond predominantly to those parameters that are known to be of the greatest practical significance. For wireless interference networks, the most significant aspects include the interplay of spatial dimensions (especially if multiple antennas are involved) with channel strengths and channel uncertainty levels [8]. Fortunately, the GDoF framework incorporates all three – spatial dimensions, channel uncertainties and channel strength levels. Furthermore, the fragile aspects of the GDoF metric may be avoided by restricting channel state information at the transmitters (CSIT) to finite precision.

The study of DoF under finite precision channel knowledge was initiated by Lapidoth et al. in [9], leading to a conjecture on the collapse of DoF. In spite of various attempts at proving or disproving the conjecture the conjecture remained open for a decade. It was ultimately settled using an approach based on a combinatorial accounting of the size of the aligned image sets (AIS) under finite precision channel knowledge, in short the AIS approach in [10]. The AIS approach modeled finite precision channel knowledge as the assumption that from the transmitters’ perspective, all joint and conditional probability density functions of channel coefficients exist and are bounded. The bounded density assumption was found to be compatible with various levels of channel strengths and channel knowledge. The AIS approach was further developed to fully characterize the GDoF of the $2$ user MISO BC (broadcast channel with two antennas at the transmitter and one antenna at each of the two receivers) for arbitrary channel strength levels and arbitrary channel uncertainty levels for each channel coefficient, establishing the GDoF optimality of robust schemes in all cases [11]. It has also led to GDoF characterizations for the $K$ user symmetric IC under finite precision CSIT [12], symmetric instances of $K$ user MISO BC [13], symmetric DoF of interference networks with finite precision CSIT and perfect CSIR [14], and GDoF of $2$ user symmetric MIMO IC with partial CSIT [15]. Indeed, there exists the distant but exciting possibility that the AIS approach may ultimately lead us to the GDoF characterizations of broad classes of wireless networks. If so, then the resulting comprehensive and fundamental understanding of these complex networks – the interplay between spatial dimensions, channel strengths, and channel uncertainty levels – would be invaluable. However, in order to get there, it is evident that a robust understanding of sum-sets will be needed. Specifically, there is the need to identify the key sum-set inequalities for signals subject to arbitrary power levels under the robust bounded density assumption. This is the goal that we pursue in this work.

The paper is organized as follows. Section $2$ provides the necessary definitions. The main results, i.e., the sum-set inequalities are presented in Section 3 through progressively generalized theorems for ease of exposition, starting from a single-letter single-antenna form to the general multi-letter multi-antenna form that is needed to derive GDoF outer bounds for MIMO networks. In Section 4, we present an example to show how these sum-set inequalities allow us to obtain new GDoF characterizations for non-trivial networks under partial CSIT111Channel uncertainty and channel strengths are interchangeable to a certain extent in MIMO interference networks, because the channel uncertainty level governs the strength of residual interference when signals are zero-forced. This is previously noted in [16]. that were previously open. The example is comprised of a $2$ user MIMO interference channel (IC) where the two transmitters are equipped with $M_{1}=5$ and $M_{2}=5$ antennas, their corresponding receivers with $N_{1}=2$ and $N_{2}=3$ antennas, the channel strength parameters are chosen to be $(\alpha_{11},\alpha_{12},\alpha_{21},\alpha_{22})=(1,\frac{3}{4},\frac{2}{3},1)$ and partial CSIT parameters are chosen to be $\beta_{12}=1/4$ and $\beta_{21}=1/3$. Remarkably, building upon these insights, in [17] we have found that the sum-set inequalities allow us to fully characterize the GDoF region of the MIMO IC with arbitrary antenna configurations $(M_{1},M_{2},N_{1},N_{2})$ under arbitrary levels of partial CSIT. Moreover, sum-set inequalities allowed the authors to characterize the full GDoF region of the two user MIMO BC with arbitrary antenna configurations $(M,N_{1},N_{2})$ under arbitrary levels of partial CSIT in [18].

Notation: For $n\in\mathbb{N}$, define the notation $[n]=\{1,2,\cdots,n\}$. The cardinality of a set $A$ is denoted as $|A|$. The notation  $X^{[n]}$ stands  for $\{X(1),X(2),\cdots,X(n)\}$. Moreover, $X_{i}^{[n]}$ also stands for $\{X_{i}(t):\forall t\in[n]\}$. The support of a random variable $X$ is denoted as supp$(X)$. The sets $\mathbb{R}$ and $\mathbb{R}^{n}$ stand for the set of real numbers and the set of all $n$-tuples of real numbers respectively. Moreover, the set $\mathbb{R}^{2+}$ is defined as the set of all pairs of non-negative numbers. If $A$ is a set of random variables, then $H(A)$ refers to the joint entropy of the random variables in $A$. Conditional entropies, mutual information and joint and conditional probability densities of sets of random variables are similarly interpreted. Moreover, we use the Landau $O(\cdot)$ and $o(\cdot)$ notations as follows. For functions $f(x),g(x)$ from $\mathbb{R}$ to $\mathbb{R}$, $f(x)=O(g(x))$ denotes that $\limsup_{x\rightarrow\infty}\frac{|f(x)|}{|g(x)|}<\infty$. $f(x)=o(g(x))$ denotes that $\limsup_{x\rightarrow\infty}\frac{|f(x)|}{|g(x)|}=0$. We use $\mathbb{P}(\cdot)$ to denote the probability function $\mbox{Prob}(\cdot)$. For any real number $x$ we define $\lfloor x\rfloor$ as the largest integer that is smaller than or equal to $x$ when $x>0$, the smallest integer that is larger than or equal to $x$ when $x<0$, and $x$ itself when $x$ is an integer. We also define $(x)^{+}$ as maximum of the number $x$ and [math], i.e., $\max(x,0)$. The number $X_{r,s}$ may be represented as $X_{rs}$ if there is no cause for ambiguity.

2 Definitions

The information theoretic sum-set inequalities that we seek are motivated by the GDoF framework. Since in the next section we present general statements of sum-set inequalities, here we only present definitions needed for Section 3. The definitions needed for the MIMO IC setting that we use as an example, are presented in Section 4.

Definition 1** (Power Levels)**

Consider integer valued variables $X_{i}$ over alphabet $\mathcal{X}_{\lambda_{i}}$,

[TABLE]

where $\bar{P}^{\lambda_{i}}$ is a compact notation for $\left\lfloor\sqrt{P^{\lambda_{i}}}\right\rfloor$. We refer to $P\in\mathbb{R}_{+}$ as power, and are primarily interested in limits as $P\rightarrow\infty$. Quantities that do not depend on $P$ will be referred to as constants. The constant $\lambda_{i}\in\mathbb{R}_{+}$ denotes the power level of $X_{i}$.

We are interested in sum-set inequalities in terms of entropies of random variables such as $X_{i}$, normalized by $\log{\bar{P}}$ as ${P}\rightarrow\infty$, while the power levels $\lambda_{i}$ are held fixed. All the sumset inequalities in this work hold in this asymptotic sense, i.e., while disregarding terms that are negligible relative to $\log({P})$. Such terms are denoted as $o(\log({P}))$ terms.

Definition 2

For any nonnegative real numbers $X$, $\lambda_{1}$ and $\lambda_{2}$, define $(X)_{\lambda_{1}}$ and $(X)^{\lambda_{2}}_{\lambda_{1}}$ as,

[TABLE]

In words, for any $X\in\mathcal{X}_{\lambda_{1}+\lambda_{2}}$, $(X)^{\lambda_{1}+\lambda_{2}}_{\lambda_{1}}$ retrieves the top $\lambda_{2}$ power levels of $X$, while $(X)_{\lambda_{1}}$ retrieves the bottom $\lambda_{1}$ levels of $X$. $(X)^{\lambda_{3}}_{\lambda_{1}}$ retrieves only the part of $X$ that lies between power levels $\lambda_{1}$ and $\lambda_{3}$. Note that $X\in\mathcal{X}_{\lambda}$ can be expressed as $X={\bar{P}^{\lambda_{1}}}{(X)}_{\lambda_{1}}^{\lambda}+{(X)}_{\lambda_{1}}$ for $0\leq\lambda_{1}<\lambda$. Equivalently, suppose $X_{1}\in\mathcal{X}_{\lambda_{1}}$, $X_{2}\in\mathcal{X}_{\lambda_{2}}$, $0<\lambda_{2}$ and $X=X_{1}+X_{2}\bar{P}^{\lambda_{1}}$. Then $X_{1}={(X)}_{\lambda_{1}}$, $X_{2}={(X)}^{\lambda_{1}+\lambda_{2}}_{\lambda_{1}}$. A conceptual illustration of power level partitions is shown in Figure 1.

Definition 3

For the vector ${\bf V}=\begin{bmatrix}v_{1}&v_{2}&\cdots&v_{k}\end{bmatrix}^{T}$, we define $({\bf V})_{\lambda_{1}}$ and $({\bf V})^{\lambda_{2}}_{\lambda_{1}}$ as,

[TABLE]

Definition 4** (Bounded Density Channel Coefficients)**

Bounded density channels are represented by a set of real valued random variables, $\mathcal{G}$ such that the magnitude of each random variable $g\in\mathcal{G}$ is bounded away from zero and infinity, $0<\Delta_{1}\leq|g|\leq\Delta_{2}<\infty$, for some constants $\Delta_{1},\Delta_{2}$, and there exists a finite positive constant $f_{\max}$, such that for all finite cardinality disjoint subsets $\mathcal{G}_{1},\mathcal{G}_{2}$ of $\mathcal{G}$, the joint probability density function of all random variables in $\mathcal{G}_{1}$, conditioned on all random variables in $\mathcal{G}_{2}$, exists and is bounded above by $f_{\max}^{|\mathcal{G}_{1}|}$.

Definition 5** (Arbitrary Channel Coefficients)**

Let $\mathcal{H}$ be a set of arbitrary constant values that are bounded above by $\Delta_{2}$, i.e., if $h\in\mathcal{H}$ then $|h|\leq\Delta_{2}<\infty$.

Definition 6

For real numbers $x_{1}\in\mathcal{X}_{\eta_{1}},x_{2}\in\mathcal{X}_{\eta_{2}},\cdots,x_{k}\in\mathcal{X}_{\eta_{k}}$ and the vectors $\vec{\gamma}=(\gamma_{1},\gamma_{2},\cdots,\gamma_{k})$ and $\vec{\delta}=(\delta_{1},\delta_{2},\cdots,\delta_{k})$ define the notations $L_{j}^{b}(x_{i},1\leq i\leq k)$, $L_{j}(x_{i},1\leq i\leq k)$, $L_{j}^{b\vec{\gamma}\vec{\delta}}(x_{i},1\leq i\leq k)$ and $L_{j}^{\vec{\gamma}\vec{\delta}}(x_{i},1\leq i\leq k)$ to represent,

[TABLE]

for distinct random variables $g_{j_{i}}\in\mathcal{G}$, some arbitrary real valued and finite constants $h_{j_{i}}\in\mathcal{H}$ and some arbitrary non-negative real valued constants $\delta_{i},\gamma_{i}$. For the vector $V=\begin{bmatrix}v_{1}&v_{2}&\cdots&v_{k}\end{bmatrix}^{T}$ we also define the notations $L^{b}_{j}(V)$ and $L_{j}(V)$ to represent,

[TABLE]

for distinct random variables $g_{j_{i}}\in\mathcal{G}$ and $h_{j_{i}}\in\mathcal{H}$.

Note that, the subscript $j$ is used to distinguish among multiple linear combinations, and may be dropped if there is no potential for ambiguity. We refer to the $L^{b}$ functions as bounded density linear combinations.

Definition 7

For the linear combinations $A=L^{b\vec{\gamma}\vec{\delta}}(x_{i},1\leq i\leq k)$ and $B=L^{\vec{\gamma}\vec{\delta}}(x_{i},1\leq i\leq k)$ where $x_{1}\in\mathcal{X}_{\eta_{1}},x_{2}\in\mathcal{X}_{\eta_{2}},\cdots,x_{k}\in\mathcal{X}_{\eta_{k}}$ we define $\mathcal{T}(A)$ and $\mathcal{T}(B)$ as,

[TABLE]

Note that the terminology from Definition (6) is invoked in Definition (7). Figure 2 provides a visual illustration of $L^{\vec{\gamma}\vec{\delta}}$ and $\mathcal{T}(A)$. From the definition of $\mathcal{T}(A)$ and $\mathcal{T}(B)$ in (12), it follows that,

[TABLE]

This is because all elements of $\mathcal{G},\mathcal{H}$ are bounded from above by $\Delta_{2}$.

Definition 8

For any vector $V=\begin{bmatrix}v_{1}&\cdots&v_{k}\end{bmatrix}^{T}$ and non-negative integer numbers $m$ and $n$ less than $k$, define

[TABLE]

Moreover, for the two vectors $V=\begin{bmatrix}v_{1}&\cdots&v_{k_{1}}\end{bmatrix}^{T}$ and $W=\begin{bmatrix}w_{1}&\cdots&w_{k_{2}}\end{bmatrix}^{T}$ define $V\bigtriangledown W$ as $\begin{bmatrix}v_{1}&\cdots&v_{k_{1}}&w_{1}&\cdots&w_{k_{2}}\end{bmatrix}^{T}$.

3 Results

Theorem 1

For $\lambda_{1}\geq\lambda_{2}\geq 0$, consider random variables $X_{1},X_{2}\in\mathcal{X}_{\lambda_{1}+\lambda_{2}}$, all independent of $\mathcal{G}$, and define

[TABLE]

then

[TABLE]

The following remarks place Theorem 1 in perspective and discuss some of its generalizations.

Let $\mathcal{G}(Z)\subset\mathcal{G}$ denote the set of all bounded density channel coefficients that appear in $Z=L^{b}(X_{2},X_{2})$, and let $W$ be a random variable such that conditioned on any $\mathcal{G}_{o}\subset(\mathcal{G}/\mathcal{G}(Z))\cup\{W\}$, the channel coefficients $\mathcal{G}(Z)$ satisfy the bounded density assumption. Then (22) generalizes to the following conditional form.

[TABLE]

The proof presented in Appendix A.1 covers this generalization. In various applications of these sum-set inequalities, the conditioning variable $W$ could represent terms such as $L_{3}(X_{1},X_{2})$, $(X_{1})_{\delta}^{\gamma}$ or $L_{4}^{b}((X_{1})_{\frac{1}{2}},X_{2})$. 2. 2.

A typical restriction in information theoretic sum-set inequalities is the independence of random variables. In contrast, note that the statement of Theorem 1 also holds for dependent random variables. 3. 3.

Since the linear combining coefficients $h_{i}$ involved in $L_{1}$ and $L_{2}$ can take arbitrary (including zero) values, several specializations of Theorem 1 follow immediately, e.g.,

[TABLE]

Figure 3 visually illustrates these inequalities in terms of the power levels.

Theorem 1 also holds if $L_{1},L_{2}$ are replaced with bounded density linear combinations, i.e., $L_{1}^{b},L_{2}^{b}$. 5. 5.

While in the GDoF framework, Theorem 1 is typically used when $\lambda_{1}\geq\lambda_{2}$ as assumed, it is possible to generalize the result of Theorem 1 to allow $\lambda_{2}\geq\lambda_{1}$. In that case, the inequality (22) becomes $H(Z\mid W,\mathcal{G})\geq H(Z_{1},Z_{2}\mid W)-(\lambda_{2}-\lambda_{1})^{+}\log(\bar{P})+o(\log{\bar{P}})$. The proof presented in Appendix A.1 covers this generalization. 6. 6.

The result of Theorem 1 lends itself to extensive generalizations in terms of the number of random variables, and the number of power level partitions. Such a generalization is presented in the following theorem.

Theorem 2

Consider $M$ non-negative numbers $\lambda_{1},\cdots,\lambda_{M}$ and random variables $X_{j}\in\mathcal{X}_{\lambda_{1}+\lambda_{2}+\cdots+\lambda_{M}}$, $j\in[N]$ independent of $\mathcal{G}$, and define

[TABLE]

$I_{1},I_{2},\cdots,I_{l}\subset[M]$* such that $\forall a,b\in[M]$, $a<b\Rightarrow m(a)\geq m(b)$, where we define*

[TABLE]

If for each $s\in\{1,2,\cdots,l-1\}$,

[TABLE]

then,

[TABLE]

Recall that for any real number $x$, we define $(x)^{+}=\max(x,0)$.

Theorem 1 is recovered as a special case of Theorem 2 if $M=N=2$, $I_{1}=\{2\}$, $I_{2}=\{1,2\}$, $\delta_{kij}=0$ and $\gamma_{kij}=\max_{q\in[M]}\lambda_{q}$ for any $k,i,j\in\{1,2\}$. 8. 8.

While applying Theorem 2 in the GDoF framework, a multi-letter extension is required. Such a generalization is presented in the following theorem. The same applies for extensions to complex valued random variables which can be obtained along the same lines as previous bounds based on the AIS approach, e.g., Section VII in [10].

Theorem 3

Consider $M$ non-negative numbers $\lambda_{1},\cdots,\lambda_{M}$ and random variables $X_{j}(t)\in\mathcal{X}_{\lambda_{1}+\lambda_{2}+\cdots+\lambda_{M}}$, $j\in[N]$, $t\in\mathbb{N}$ independent of $\mathcal{G}$, and define

[TABLE]

The channel uses are indexed by t$$\in\mathbb{N}. $I_{1},I_{2},\cdots,I_{l}$ are subsets of $\{1,2,\cdots,M\}$ such that $m(a)\geq m(b)$ whenever $a,b\in\{1,2,\cdots,M\}$ and $a<b$, then

[TABLE]

if for each $s\in\{1,2,\cdots,l-1\}$,

[TABLE]

Note that, for any $i\in[l]$ the set $I_{i}$ indicates what power levels are used by each $Z_{i}(t)$. For instance $I_{3}=\{1\}$ enforces $Z_{3}(t)$ to be a linear combination of bottom $\lambda_{1}$ part of $X_{j}(t)$ for all $j\in[N]$, i.e., $Z_{3}(t)=L_{3}^{\vec{\gamma}_{3}\vec{\delta}_{3}}(t)((X_{j}(t))_{\lambda_{1}},j\in[N])$. 9. 9.

While applying Theorem 3 in the GDoF framework, a multi-antenna extension is required. The results of Theorem 3 can be generalized as follows,

Theorem 4

Consider $KM$ non-negative numbers $\{\lambda_{km}:k\in[K],m\in[M]\}$ and random variables $X_{j}(t)\in\mathcal{X}_{\max_{k\in[K]}\{\lambda_{k,1}+\lambda_{k,2}+\cdots+\lambda_{k,M}\}}$, $j\in[N]$, $t\in\mathbb{N}$, independent of $\mathcal{G}$, and $\forall k\in[K],K\leq N$, define

[TABLE]

The channel uses are indexed by $t\in\mathbb{N}$. $I_{kk^{\prime}}\subset[M],k\in[K],k^{\prime}\in[l_{k}],$ such that $i<j\Rightarrow m(k,i)\geq m(k,j)$, where

[TABLE]

If for all $k\in[K]$ and for each $s\in\{1,2,\cdots,l_{k}-1\}$,

[TABLE]

then

[TABLE]

See Appendix A.2 for the proof of Theorem 4. Note that the proof of Theorem 4 also proves Theorem 2 and Theorem 3 which may be recovered as specializations of Theorem 4.

A visual illustration of an application of Theorem 4 is provided in Figure 5. 10. 10.

The results of Theorem 1 and its generalization in Theorem 4 can be further combined with sub-modularity properties of the entropy function222If $\Omega$ is a finite set, a submodular function is a set function $f:2^{\Omega}\rightarrow\mathbb{R}$, where $2^{\Omega}$ denotes the power set of $\Omega$, which satisfies the following property;

For every $S,T\subseteq\Omega$ we have that $f(S)+f(T)\geq f(S\cup T)+f(S\cap T)$ [19]. to obtain a variety of sum-set inequalities specialized for different GDoF settings. 11. 11.

To show how the new sum-set inequalities presented in Theorem 4 are useful to obtain tight GDoF bounds in conjunction with submodularity properties of entropy, an example that arise in the context of the $2$ user MIMO IC is presented in Section 4.

4 GDoF Outer Bound for a $2$ User MIMO IC under Partial CSIT

In this section, as an example of the use of the sum-set inequalities, we obtain a tight GDoF outer bound for a non-trivial two user MIMO IC setting with asymmetric antenna configuration and asymmetric partial CSIT. Specifically, we consider the two user MIMO IC with $(M_{1},M_{2},N_{1},N_{2})=(5,5,2,3)$ as shown in Figure 6. We assume, $(\alpha_{11},\alpha_{12},\alpha_{21},\alpha_{22})=(1,\frac{3}{4},\frac{2}{3},1)$ and $\beta_{12}=\frac{1}{4},\beta_{21}=\frac{1}{3}$ and derive a tight GDoF bound for this channel using our sum-set inequalities. Achievability for this case is already known from [17] and [20].

4.1 The Channel

The channel model for the two user $(M_{1},M_{2},N_{1},N_{2})=(5,5,2,3)$ MIMO IC with  $(\alpha_{11},\alpha_{12},\alpha_{21},\alpha_{22})=(1,\frac{3}{4},\frac{2}{3},1)$ is defined by the following input-output equations.

[TABLE]

Here, $\mathbf{X}_{1}(t)=[{X}_{11}(t)\ {X}_{12}(t)\ {X}_{13}(t)\ {X}_{14}(t)\ {X}_{15}(t)]^{T}$ and $\mathbf{X}_{2}(t)=[{X}_{21}(t)\ {X}_{22}(t)\ {X}_{23}(t)\ {X}_{24}(t)\ {X}_{25}(t)]^{T}$ are the ${5\times 1}$ signal vectors sent from the first and second transmitters respectively, normalized so that each is subject to unit power constraint. $\mathbf{Y}_{1}(t)=[{Y}_{11}(t)\ {Y}_{12}(t)]^{T}$ and $\mathbf{Y}_{2}(t)=[{Y}_{21}(t)\ {Y}_{22}(t)\ {Y}_{23}(t)]^{T}$ are the ${2\times 1}$ and ${3\times 1}$ received signal vectors at the first and second receivers, respectively. $\mathbf{\Gamma}_{1}(t)$ and $\mathbf{\Gamma}_{2}(t)$ are the ${2\times 1}$ and ${3\times 1}$ vectors whose components are zero-mean unit-variance additive white Gaussian noise (AWGN). The $N_{r}\times M_{s}$ matrix ${\bf G}_{rs}(t)$ is the channel fading coefficient matrix between the $r$-th receiver and the $s$-th transmitter for any $r,s\in\{1,2\}$. The entry in the $n$-th row and $m$-th column of the matrix ${\bf G}_{rs}(t)$ is ${G}_{rsnm}(t)$.

4.1.1 Partial CSIT

Under partial CSIT, the channel coefficients are represented as

[TABLE]

Recall that $G_{rsnm}(t)$ is the channel fading coefficient between the $n$-th antenna of the $r$-th receiver and the $m$-th antenna of the $s$-th transmitter. $\hat{G}_{rsnm}(t)$ is the channel estimate and $\tilde{G}_{rsnm}(t)$ is the estimation error term. To avoid degenerate conditions, for each $N_{r}\times M_{s}$ channel matrix ${\bf G}_{rs}(t)$, we require that all its $N_{r}\times N_{r}$ submatrices are non-singular, i.e., their determinants are bound away from zero. To this end, for all $t\in[n],~{}r,s\in\{1,2\}$, and for all choices of $N_{r}$ transmit antenna indices $\{m_{1},m_{2},\cdots,m_{N_{r}}:m_{i}\in[M_{s}]\}$ define the determinant $D(t)$ as

[TABLE]

Then we require that there exists a positive constant $\Delta_{1}>0$, such that $|D(t)|\geq\Delta_{1}$, for all $t\in[n],~{}r,s\in\{1,2\},\{m_{1},m_{2},\cdots,m_{N_{r}}:m_{i}\in[M_{s}]\}.$ The channel variables $\hat{G}_{rsnm}(t),\tilde{G}_{rsnm}(t)$ are distinct random variables drawn from the set $\mathcal{G}$. The realizations of $\hat{G}_{rsnm}(t)$ are known to the transmitter, but the realizations of $\tilde{G}_{rsnm}(t)$ are not available to the transmitter. We also assume that the channel coefficients $|{G}_{rsnm}(t)|$ are bounded away from zero, i.e.,

[TABLE]

Note that under the partial CSIT model, the variance of the channel coefficients $G_{rsnm}(t)$ behaves as $\sim P^{-\beta_{rs}}$ and the peak of the probability density function behaves as $\sim\sqrt{P^{\beta_{rs}}}$.

For any $r,s\in\{1,2\}$, in order to span the full range of partial channel knowledge at the transmitters, the corresponding range of $\beta_{rs}$ parameters, assumed throughout this work, is $0\leq\beta_{rs}\leq 1$. $\beta_{rs}=0$ and $\beta_{rs}=1$ correspond to the two extremes where the CSIT is essentially absent, or perfect, respectively. Note that the value of $\beta_{11}$ and $\beta_{22}$ will not affect the GDoF.

4.1.2 GDoF

The definitions of achievable rates $R_{i}(P)$ and capacity region $\mathcal{C}(P)$ are standard. The DoF region is defined as

[TABLE]

4.2 Channel Model

The channel model is derived similar to the channel model for the general two user IC with arbitrary number of antennas in [12]. We will avoid repetition of explanations for those steps that are essentially identical to [12], and focus instead on the deviations from the original proof. As in [12], the starting point is to bound the problem with deterministic model, such that a GDoF outer bound on the deterministic model is also a GDoF outer bound for the original problem. Since the derivation of the deterministic model is essentially identical to [12], here we simply state the resulting equivalent deterministic model.

4.2.1 Equivalent Deterministic Model

As in [12], without loss of generality for DoF characterizations, we will use the deterministic model for the equivalent channel.

[TABLE]

for all $t\in[n]$. $\bar{\mathbf{X}}_{1a}(t)$, $\bar{{X}}_{1b}(t)$, $\bar{{X}}_{1c}(t)$, $\bar{\mathbf{X}}_{2a}(t)$, $\bar{\mathbf{X}}_{2c}(t)$ and $\bar{\mathbf{Y}}_{1}(t)$ are defined as,

[TABLE]

and $\bar{X}_{1m}(t),\bar{X}_{2m}(t)\in\{0,1,\cdots,{\bar{P}}\}$, $\forall m\in[5]$.

4.3 GDoF region of the two user MIMO IC

Theorem 5

The GDoF region of the two user  $(M_{1},M_{2},N_{1},N_{2})=(5,5,2,3)$ MIMO IC with $(\alpha_{11},\alpha_{12},\alpha_{21},\alpha_{22})=(1,\frac{3}{4},\frac{2}{3},1)$ and $\beta_{12}=\frac{1}{4},\beta_{21}=\frac{1}{3}$, is as follows

[TABLE]

Note that, these bounds turns out to be tight, i.e., the achievability and the outer bounds coincide with each others, see [17].

Proof of Theorem 5 is relegated to Appendix A.3 and is straightforward except for the following lemma which is the main novelty of the outer bound proof. It is in deriving this key lemma that we require both the sum-set inequalities of Theorem 4 and the sub-modularity of entropy functions.

Lemma 1

For the two user $(M_{1},M_{2},N_{1},N_{2})=(5,5,2,3)$ MIMO IC with $(\alpha_{11},\alpha_{12},\alpha_{21},\alpha_{22})=(1,\frac{3}{4},\frac{2}{3},1)$ and $\beta_{12}=\frac{1}{4},\beta_{21}=\frac{1}{3}$ levels of partial CSIT, we have,

[TABLE]

See Figure 7 for the comparison of the two sides of (58).

See Appendix B for proof of Lemma 1.

5 Conclusion

We present a class of sum-set inequalities for bounded set linear combinations of random variables typically encountered in the GDoF framework. The bounds are obtained by building upon the aligned image sets (AIS) approach. Through an example, we showed that these inequalities are useful for obtaining tight GDoF bounds for MIMO interference networks with arbitrary antenna configurations and arbitrary levels of channel uncertainty for each channel. Indeed, we expect these inequalities to be broadly useful for obtaining tight GDoF bounds for MIMO wireless interference and broadcast networks under varying levels of channel strengths and channel uncertainty.

Appendix A Proof of Theorems 1, 2, 3, 4 and 5

A.1 Proof of Theorem 1

A.1.1 Sketch of the proof

Let us start with a summary of the Aligned Image Sets approach that we use in this proof. We are only interested in maximum of difference of entropies of $Z^{\prime}=(Z_{1},Z_{2})$ and $Z$ conditioned on $W,\mathcal{G}$, i.e., $H(Z^{\prime}\mid W)-H(Z\mid W,\mathcal{G})$. Following directly along the AIS approach [10], from the functional dependence argument it follows that without loss of generality $Z$ can be assumed to be a function of $Z^{\prime},W,\mathcal{G}$. So, it follows that,

[TABLE]

where $(\ref{ew1})$ follows from chain rule and $(\ref{ew2})$ is true as $Z$ is a function of $Z^{\prime},W,\mathcal{G}$. Thus, the difference of entropies is equal to $H(Z^{\prime}|Z,W,\mathcal{G})$. Now, for a given $W$ and channel realization $\mathcal{G}$, define aligned image set $S_{\nu}(W,\mathcal{G})$ as the set of all $Z^{\prime}$ which result in the same $Z$. In the other words, since $Z$ is a function of $Z^{\prime},W,\mathcal{G}$, we define the set $S_{\nu}(W,\mathcal{G})$ as the set of all values of $Z^{\prime}$ which produce the same value for $Z$, as is produced by $Z^{\prime}=\nu$. Since uniform distribution maximizes the entropy,

[TABLE]

where $\mathcal{W}$ is support of the random variable $W$. (64) comes from the Jensen’s Inequality. Thus, the difference of the entropies is bounded by the log of expected value of cardinality of the aligned image set. Now, the most crucial step is to bound the cardinality of $S_{\nu}(W,\mathcal{G})$ where we need to use Bounded Density Assumption of $\mathcal{G}$ to bound the cardinality of $S_{\nu}(W,\mathcal{G})$. So, from the equation (64), \mbox{E}_{\mathcal{G}}\{|S_{\nu}(W,\mathcal{G})|\mid{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}W=w}\} is what needed to be calculated. Expected value of size of the cardinality of aligned image set is equal to the summation of probability of alignment over all $Z^{\prime}$, or in the other words,

[TABLE]

where $P_{a}$ is defined as the probability that $Z^{\prime}$ and $\nu$ correspond to the same $Z$. In the proof, we prove that \mbox{E}_{\mathcal{G}}\{|S_{\nu}(W,\mathcal{G})|\mid{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}W=w}\} is bounded by $(c_{1}+c_{2}\bar{P}^{{(\lambda_{2}-\lambda_{1})}^{+}})(c_{3}+c_{4}\log{\bar{P}})$ from above for constants $c_{1},c_{2},c_{3}$ and $c_{4}$. So, from the inequality (64), we have,

[TABLE]

for some constant $c_{5}$ if $\lambda_{2}\leq\lambda_{1}$. As $\log{(\log{\bar{P}})}=o(\log{\bar{P}})$, (22) is concluded. The detailed arguments are presented next.

A.1.2 Functional Dependence $Z(Z^{\prime},W,\mathcal{G})$

We start by showing that there is no loss of generality in the assumption that $(X_{1},X_{2})$ is a function of $Z^{\prime}$ and $W$, and therefore $Z$ is a function of $Z^{\prime},W,\mathcal{G}$. Recall that $(X_{1},X_{2})$ is independent of $\mathcal{G}$. However, there may be multiple values of $(X_{1},X_{2})$ that cast the same image in $Z^{\prime},W$. So the mapping from $Z^{\prime},W$ to $(X_{1},X_{2})$ is in general random. Let us denote it by $\mathcal{L}$ i.e.

[TABLE]

In general, because the mapping may be random, $\mathcal{L}$ is a random variable. Conditioning cannot increase entropy, therefore,

[TABLE]

Let $L_{o}\in\mathcal{L}$ be the mapping that minimizes the entropy term. Fix this as the deterministic mapping,

[TABLE]

so that now $Z$ is a function of $Z^{\prime},W,\mathcal{G}$, and since $Z^{\prime}$ is a function of $(X_{1},X_{2})$, $Z$ is equivalently a function of $(X_{1},X_{2},W,\mathcal{G})$. Based on convenience, we may indicate the functional dependence in any of these forms as

[TABLE]

We note that the choice of mapping does not affect the positive entropy term $H(Z^{\prime}\mid W)$ but it minimizes $H(Z\mid W,\mathcal{G})$.

A.1.3 Definition of Aligned Image Sets

The aligned image set containing the codeword $\nu\in\mbox{supp}(Z^{\prime})$ for a given $W=w$ and realization $\mathcal{G}=G$ is defined as the set of all values of $Z^{\prime}$ that produces the same $Z$ value as is produced by $Z^{\prime}=\nu$. Mathematically,

[TABLE]

Since we will need the average (over $\mathcal{G}$) of the cardinality of an aligned image set, E$|S_{\nu}(W,\mathcal{G})|$, it is worthwhile to point out that the cardinality $|S_{\nu}(W,\mathcal{G})|$ as a function of $\mathcal{G}$, is a bounded simple function, and therefore measurable.333A simple function is a finite sum of indicator functions of measurable sets [21]. It is bounded because its values are restricted to the set of natural numbers not greater than $c\bar{P}^{\lambda_{2}+\max(\lambda_{1},\lambda_{2})}$, where $c$ depends on coefficients of linear combinations $L_{1}$ and $L_{2}$. Following the same steps as [10] it is a simple function too.

A.1.4 Bounding the Probability of Image Alignment

From (65), we have $\mbox{E}_{\mathcal{G}}\{\left|S_{\nu}(W,\mathcal{G})\right|\mid W=w\}=\sum_{Z^{\prime}}P_{a}(Z^{\prime})$. Given $W=w,\mathcal{G}$, consider two distinct realizations of $(Z_{1},Z_{2})$, say $(z_{1},z_{2})$, and $(z^{\prime}_{1},z^{\prime}_{2})$, which are produced by two distinct realizations of $(X_{1},X_{2})$, denoted as $(\mu_{1},\mu_{2})$ and $(\nu_{1},\nu_{2})$. For $i\in\{1,2\}$, define $\mu_{1i}$, $\mu_{2i}$, $\nu_{1i}$ and $\nu_{2i}$ as $(\mu_{i})_{\lambda_{1}}$, $(\mu_{i})_{\lambda_{1}}^{\lambda_{1}+\lambda_{2}}$, $(\nu_{i})_{\lambda_{1}}$, and $(\nu_{i})_{\lambda_{1}}^{\lambda_{1}+\lambda_{2}}$ respectively.

[TABLE]

We wish to bound the probability that the images of these two codewords align, or in other words $Z(z_{1},z_{2},W,\mathcal{G})=Z(z^{\prime}_{1},z^{\prime}_{2},W,\mathcal{G})$,

[TABLE]

defining $C$ as $g_{2}(\mu_{22}-\nu_{22})\bar{P}^{\lambda_{1}}-g_{2}(\mu_{12}-\nu_{12})$ we have

[TABLE]

So for fixed values of $g_{2}$ the random variable $g_{1}(\mu_{21}-\nu_{21})\bar{P}^{\lambda_{1}}+g_{1}(\mu_{11}-\nu_{11})$ must take values within an interval of length no more than $4$. If $|\mu_{21}-\nu_{21}|+|\mu_{11}-\nu_{11}|\neq 0$, then $g_{1}$ must take values in an interval of length no more than $\frac{4}{|(\mu_{21}-\nu_{21})\bar{P}^{\lambda_{1}}+\mu_{11}-\nu_{11}|}$, the probability of which is no more than $\frac{4f_{\max}}{|(\mu_{21}-\nu_{21})\bar{P}^{\lambda_{1}}+\mu_{11}-\nu_{11}|}$. Note that the integral of any real-valued measurable function $h(x)$ over any measurable set $S$ can be bounded above by $\max_{x\in\mathcal{R}}h(x)$ times the measure of the set $S$, which for the interval $I$ reduces to the length of the interval $I$ [21]. Similarly, for fixed values of $g_{1}$ probability of alignment will be bounded by $\frac{4f_{\max}}{|(\mu_{22}-\nu_{22})\bar{P}^{\lambda_{1}}+\mu_{12}-\nu_{12}|}$. As $(z_{1},z_{2})$, and $(z^{\prime}_{1},z^{\prime}_{2})$ are two distinct realizations of $(Z_{1},Z_{2})$, at least one of $\mu_{ij}-\nu_{ij}$ for $i,j\in\{1,2\}$ is nonzero. So, it can be concluded that the probability is no more than $P_{a}(z^{\prime}_{1},z^{\prime}_{2})$ where $P_{a}(z^{\prime}_{1},z^{\prime}_{2})$ is defined as follows.

[TABLE]

A.1.5 Bounding the Average Size of Aligned Image Sets

From (65) we have to compute the following summation,

[TABLE]

Note that, from (73), and (74) the terms $|z_{1}-z^{\prime}_{1}|$, and $|z_{2}-z^{\prime}_{2}|$ can be bounded from above by $3+\lfloor 2\Delta_{2}\bar{P}^{\lambda_{2}}\rfloor$, and $5+\lfloor 4\Delta_{2}\bar{P}^{\max(\lambda_{1},\lambda_{2})}\rfloor$, respectively as $|h_{2j}|$ and $|h^{\prime}_{ij}|$ are less than $\Delta_{2}$ for all $i,j\in\{1,2\}$ and, $|\mu_{1j}-\nu_{1j}|$ and $|\mu_{2j}-\nu_{2j}|$ are also less than $\bar{P}^{\lambda_{1}}$ , $\bar{P}^{\lambda_{2}}$ respectively. Using the definition of Aligned Image Sets from A.1.3, we have,

[TABLE]

where the sets $S_{1}$, $S^{\prime}_{1}$ and $S_{2}$ are defined as $\{0,1,\cdots,3+\lfloor 2\Delta_{2}\bar{P}^{\lambda_{2}}\rfloor\}$, $\{h_{o},\cdots,3+\lfloor 2\Delta_{2}\bar{P}^{\lambda_{2}}\rfloor\}$ and $\{0,1,\cdots,5+\lfloor 4\Delta_{2}\bar{P}^{\max(\lambda_{1},\lambda_{2})}\rfloor\}$, respectively. For simplicity we defined $h_{o}$ as the constant $4+\lfloor|h_{21}|+|h_{22}|\rfloor$. Now, let us bound each term in (80) separately. Note that our ultimate goal is to prove $\mbox{E}_{\mathcal{G}}\{\left|S_{z_{1},z_{2}}(\mathcal{G})\right|\}\leq\bar{P}^{(\lambda_{2}-\lambda_{1})^{+}}(c_{3}+c_{4}{(\log{(\bar{P})})})$ for some constants $c_{3}$ and $c_{4}$, which along with (64) leads to the conclusion (22) when $\lambda_{2}<\lambda_{1}$.

Let us compute the first summation in (80).

[TABLE]

(81) follows by bounding the terms $|(\mu_{2i}-\nu_{2i})\bar{P}^{\lambda_{1}}+\mu_{1i}-\nu_{1i}|$ from below by $(|\mu_{2i}-\nu_{2i}|-1)\bar{P}^{\lambda_{1}}$ if $|\mu_{2i}-\nu_{2i}|>1$. (1) is breaking the summation into multiplication of two summations, and (LABEL:poi11) is true because $P_{a}$ is bounded by one, so the summation of $h_{o}$ such terms can be at most $h_{o}$. Finally, (1) follows by bounding $|z_{1}-z^{\prime}_{1}|$ from (73) and (74) as,

[TABLE]

where $I$ is a random variable which takes values in the interval $(-2,2)$. (85) is true as the partial sum of harmonic series can be bounded above by logarithmic function, i.e., $\sum_{i=1}^{n}\frac{1}{i}\leq 1+\ln{(n)}$. 2. 2.

The second term in (80), i.e., $\sum_{|\mu_{22}-\nu_{22}|\notin\{0,1\},|z_{1}-z^{\prime}_{1}|\in S_{1},|z_{2}-z^{\prime}_{2}|\in S_{2}}P_{a}$ is bounded similarly by the exact term in equation (85) as the inequalities (81)-(85) remain true whether the summation is over $|\mu_{21}-\nu_{21}|\notin\{0,1\}$ or $|\mu_{22}-\nu_{22}|\notin\{0,1\}$. 3. 3.

Finally, the third term in (80) is bounded from above by splitting the summation into four summations where in each summation the terms $|\mu_{21}-\nu_{21}|$ and $|\mu_{22}-\nu_{22}|$ are fixed to either zero or one. First of all let us write $z_{2}-z^{\prime}_{2}$ from $(\ref{eqrs1})$, and $(\ref{eqrs2})$ as,

[TABLE]

where $I$ is a random variable depending on $\mu_{ij},\nu_{ij},\forall i,j\in\{1,2\}$ which takes values in the interval $(-4,4)$.

[TABLE]

where the function $sgn(x)$ is defined as $1$ if $x\geq 0$ and $-1$ if $x<0$. The numbers $\hat{r}$, $\hat{s}$, $u_{11}$, $u_{12}$, $P_{r}$ and $P_{t}$ are also defined as $2r-1$, $2s-1$, $sgn(\mu_{11}-\nu_{11})$, $sgn(\mu_{12}-\nu_{12})$, $4+\lfloor 4\Delta_{2}\rfloor+\lfloor 4\Delta_{2}\bar{P}^{\lambda_{1}}\rfloor$, and $r\hat{r}u_{11}h^{\prime}_{11}\bar{P}^{\lambda_{1}}+s\hat{s}u_{12}h^{\prime}_{12}\bar{P}^{\lambda_{1}}+rh^{\prime}_{21}+sh^{\prime}_{22}$. The set $S_{3}$ is defined as the set of integer numbers $\{1,2,\cdots,P_{r}\}$. (LABEL:ppjj) is derived by replacing $|\mu_{21}-\nu_{21}|,|\mu_{22}-\nu_{22}|\in\{0,1\}$ in $P_{a}$. (90) follows as,

[TABLE]

where $(\ref{gfds})$ is true as the $\hat{r},\hat{s},u_{11},u_{12}\in\{-1,1\}$. (93) is derived by replacing $z_{2}-z^{\prime}_{2}$ from (88), (93) follows by breaking summation into two summations, and, (93) is true as minimum of any two numbers can be bounded above by one of them. Note that the first summation in (93) can be at most $11$ as $|z_{2}-z^{\prime}_{2}-I-P_{t}|<1$ is true only if $-5+P_{t}<z_{2}-z^{\prime}_{2}<P_{t}+5$. Moreover, $-5+P_{t}<z_{2}-z^{\prime}_{2}<P_{t}+5$ can be true for at most $11$ integer numbers of $|z_{2}-z^{\prime}_{2}|$. So, the first summation is at most 11. Second summation in (93) is bounded as,

[TABLE]

where (97) is concluded from the following two points,

1. (a)

Each summand in the left summation (97) is reciprocal of a positive integer number, and reciprocal of any positive integer number, i.e., $n$ can be repeated in the left summation at most $18$ times as $\lfloor|P_{t}+I-(z_{2}-z^{\prime}_{2})|\rfloor=n$ can have at most $18$ solutions in the set of $z_{2}-z^{\prime}_{2}\in\mathcal{Z}$ for any fixed integer number $n$.

[TABLE]

(99), (100) can be true only for $18$ integers of $z_{2}-z^{\prime}_{2}$ in the set of $z_{2}-z^{\prime}_{2}\in\mathcal{Z}$ for any fixed integer number of $n$. So, any reciprocal of any positive integer number, i.e., $n$ can be repeated at most 18 times in the left summation. 2. (b)

$\hat{z}$ can only get integer numbers from the set $S_{3}$, as ${\lfloor|P_{t}+I-(z_{2}-z^{\prime}_{2})|\rfloor}$ is bounded from above by $4+\lfloor 4\Delta_{2}\rfloor+\lfloor 4\Delta_{2}\bar{P}^{\lambda_{1}}\rfloor$.

Finally, (95) is true as the partial sum of harmonic series can be bounded above by logarithmic function.

A.1.6 Combining the Bounds to Complete the Proof

Now, from (80), (85), and (95) since constant terms and $\log{\log{{P}}}$ are $o(\log{{P}})$, we have,

[TABLE]

as (101) is true for all $W=w$, from (65) we have,

[TABLE]

Note that, (22) is concluded when $\lambda_{2}\leq\lambda_{1}$.

A.2 Proof of Theorem 2, 3 and 4

In this section we only present proof of Theorem 4 as Theorem 2 and 3 are obtained as special cases of Theorem 4.

Recall that,

[TABLE]

The channel uses are indexed by $t\in\mathbb{N}$. $I_{kk^{\prime}}\subset[M],k\in[K],k^{\prime}\in[l_{k}],$ such that $i<j\Rightarrow m(k,i)\geq m(k,j)$, where

[TABLE]

If for all $k\in[K]$ and for each $s\in\{1,2,\cdots,l_{k}-1\}$,

[TABLE]

then

[TABLE]

Note that, (107) is equivalent to,

[TABLE]

where $\gamma_{krk^{\prime}j}$ and $\delta_{krk^{\prime}j}$ are elements of the vectors $\vec{\gamma}_{kr}$, $\vec{\delta}_{kr}$. Without loss of generality we assume $K<N$ since for $K\geq N$ the left hand side of (108) is equal to $H(X_{1}^{[n]},\cdots,X_{N}^{[n]}\mid W,\mathcal{G})$. Therefore, (108) is immediate. Moreover, we assume $\delta_{krk^{\prime}j}\leq\gamma_{krk^{\prime}j}$ for any $k\in[K],r\in[l_{k}],j\in[N],k^{\prime}\in I_{kr}$ 444Note that $(x)_{\delta}^{\gamma}=0$ if $\gamma\leq\delta$, see Definition 6..

A.2.1 Sketch of the proof

The first steps to prove (108) follows from from the same lines of proof of Theorem 1 and it is straightforward based on it, see A.1.1. To avoid repetition we only go over the parts that are different from proof of Theorem 1. Similar to proof of Theorem 1, we are only interested in maximum of difference of entropies of ${Z^{\prime}}^{n}=(Z_{11}^{[n]},\cdots,Z_{Kl_{K}}^{[n]})$ and $Z^{n}=(Z_{1}^{[n]},\cdots,Z_{K}^{[n]})$ conditioned on $W$ and $\mathcal{G}$, i.e., $H({Z^{\prime}}^{n}\mid W)-H(Z^{n}\mid W,\mathcal{G})$. Similar to proof of Theorem 1, from the functional dependence argument it follows that without loss of generality $Z^{n}$ can be made a function of ${Z^{\prime}}^{n},W,\mathcal{G}$. For given $W$ and channel realization $\mathcal{G}$, define aligned image set $S_{\nu^{n}}(W,\mathcal{G})$ as the set of all ${Z^{\prime}}^{n}$ which result in the same $Z^{n}$. Thus, we have,

[TABLE]

where (113) comes from the Jensen’s Inequality. Thus, the difference of the entropies is bounded by the log of expected value of cardinality of the aligned image set. Similar to proof of Theorem 1, the key step is to bound the cardinality of $S_{\nu^{n}}(W,\mathcal{G})$ where we need to use Bounded Density Assumption of $\mathcal{G}$ to bound the cardinality of $S_{\nu^{n}}(W,\mathcal{G})$. So, from (113), $\mbox{E}_{\mathcal{G}}\{|S_{\nu^{n}}(W,\mathcal{G})|\mid W=w\}$ is what needed to be calculated. Expected value of size of the cardinality of aligned image set is equal to the summation of probability of alignment over all ${Z^{\prime}}^{n}$, or in the other words,

[TABLE]

where $\mathbb{P}({Z^{\prime}}^{n})$ is defined as the probability that ${Z^{\prime}}^{n}$ and $\nu^{n}$ correspond to the same $Z^{n}$. In the proof, we prove that for any $w\in\mathcal{W}$, $\mbox{E}_{\mathcal{G}}\{|S_{\nu^{n}}(W,\mathcal{G})|\mid W=w\}$ is bounded by ${(c_{1}+c_{2}\log{\bar{P}})}^{Kn}$ from above for some positive constants $c_{1},c_{2}$. Note that, $\mathcal{W}$ was defined as the support of $W$. So, from the inequality (64), we have,

[TABLE]

for some positive constant $c_{3}$. As $\log{(\log{\bar{P}})}=o(\log{\bar{P}})$, (43) is concluded. The detailed arguments are presented next.

A.2.2 Bounding the Probability of Image Alignment

Given  $\mathcal{G}$ and $W=w$,  consider  two  distinct  instances  of  ${Z^{\prime}}^{n}$  denoted as $\mu^{[n]}=(\mu_{11}^{[n]},\cdots,\mu_{Kl_{K}}^{[n]})$ and $\nu^{[n]}=(\nu_{11}^{[n]},\cdots,\nu_{Kl_{K}}^{[n]})$ produced by corresponding realizations of codewords $(X_{1}^{n},X_{2}^{n},\cdots,X_{N}^{n})$ denoted by $(E_{1}^{n},E_{2}^{n},\cdots,E_{N}^{n})$ and $(F_{1}^{n},F_{2}^{n},\cdots,F_{N}^{n})$, respectively. For any $k\in[K]$, $l\in[l_{k}]$, $t\in[n]$, the random variables $\mu_{kl}(t)$ and $\nu_{kl}(t)$ are derived as,

[TABLE]

In the next step we bound $\mathbb{P}(\mu^{[n]}\in\mathcal{S}_{\nu^{[n]}})$ from above. We wish to bound the probability that the images of these two codewords align, or in other words $Z^{n}({\mu}^{n},W,\mathcal{G})=Z^{n}({\nu}^{n},W,\mathcal{G})$. Thus, for any $k\in[K]$ and $t\in[n]$ we have,

[TABLE]

where (120) follows from (119) as for any real number $x$, $|x-\lfloor x\rfloor|<1$. For any $k\in[K],t\in[n],l\in\{N\}$ and any fixed values of $g_{k1}(t),\cdots,g_{k(l-1)}(t),g_{k(l+1)}(t),\cdots,g_{kM}(t)$ the random variable $g_{kl}(t)\left(E_{l}(t)-F_{l}(t)\right)$ must take values within an interval of length no more than $2{N}$. Therefore, for any $k\in[K],t\in[n],l\in[N]$ if $E_{l}(t)\neq F_{l}(t)$, then $g_{kl}(t)$ must take values in an interval of length no more than $\frac{2{N}}{|E_{l}(t)-F_{l}(t)|}$, the probability of which is no more than $\frac{2{N}f_{\max}}{|E_{l}(t)-F_{l}(t)|}$. The probability of alignment is bounded by

[TABLE]

A.2.3 Bounding the Average Size of Aligned Image Sets

From (114) we have to compute the following summation,

[TABLE]

for some positive constants $c_{1}$ and $c_{2}$. ${\mathcal{Z}^{\prime}}^{n}$ and $\mathcal{Z}_{kl}^{[n]}$ are defined as the support of the random variable ${\mu}^{n}$ and $\mu_{kl}^{[n]}$ for any $k\in[K]$ and $l\in[l_{K}]$. Note that, from (113) and (126), the bound (116) is obtained. Therefore, (43) is concluded.

A.2.4 Proof of (126) for $K=1$ and $n=1$

First let us prove the bound (126) when $K=1$ and $n=1$. Without loss of generality let us drop the time index $(t)$ and assume $k=1$. Thus, our goal is to prove that,

[TABLE]

where $\mathcal{Z}_{kl}$ is defined as the set $\{0\}\cup[2MN+\lfloor MN\Delta_{2}{\bar{P}}^{\max_{j\in[N],k^{\prime}\in I_{kl}}\min(\lambda_{kk^{\prime}},\gamma_{klk^{\prime}j}-\delta_{klk^{\prime}j})}\rfloor]$ for any $k\in[K]$ and $l\in[l_{K}]$ 555Note that from (117) and (118), we have,

$\displaystyle|\mu_{kl}(t)-\nu_{kl}(t)|$ $\displaystyle\leq$ $\displaystyle 2MN+MN\Delta_{2}{\bar{P}}^{\max_{j\in[N],k^{\prime}\in I_{kl}}\min(\lambda_{kk^{\prime}},\gamma_{klk^{\prime}j}-\delta_{klk^{\prime}j})}.$

. For any $l\in[l_{1}]$, the random variables $\mu_{1l}$ and $\nu_{1l}$ are derived from (117) and (118) as,

[TABLE]

In the next step to compute the summation (152). First of all, we define $\breve{\Delta}_{lij}$ as,

[TABLE]

and define the number $l^{\star}$ as the smallest integer where

[TABLE]

Consider the following two cases.

$l^{\star}$ doesn’t exist.

If there doesn’t exist any $l\in[l_{1}]$, $i\in I_{1l}$ and $j\in[N]$ satisfying the condition (131), i.e., $\forall l,i,j,l\in[l_{1}],i\in I_{1l},j\in[N],\breve{\Delta}_{lij}\in\{-1,0,1\}$, then each of the variables $|\mu_{1l}-\nu_{1l}|$ is bounded by $MN\Delta_{2}$ as we have

[TABLE]

for any $l\in[l_{k}]$. Therefore, (152) is true as the summation in (152) is the summation of positive numbers less than $2Nf_{\max}$ over at most ${(2MN\Delta_{2})}^{l_{1}}$ numbers. 2. 2.

$1\leq l^{\star}\leq l_{1}$.

Any number $X\in\mathcal{X}_{\delta}$ can be written as $(X)_{a}^{\delta}\bar{P}^{a}+(X)^{a}_{b}\bar{P}^{b}+X_{b}$ for any non-negative numbers $a,b$ less than $\delta$. Thus, the term $|E_{j}-F_{j}|$ can be rewritten as,

[TABLE]

where $J_{ij}$ is defined as,

[TABLE]

Moreover, define $J^{\star}_{ij}$ as the set of $\{-\bar{P}^{\min(\lambda_{1i},\gamma_{1l^{\star}ij})},0,\bar{P}^{\min(\lambda_{1i},\gamma_{1l^{\star}ij})}\}$. (134) is true for any non-negative real number $x$ as both of the random variables $(E_{j})_{x}$ and $(F_{j})_{x}$ are positive numbers less than $\bar{P}^{x}$. Since (134) is true for any $i\in I_{1l^{\star}}$, we have,

[TABLE]

where (137) is true as for any two non-negative real-valued functions $f(x)$ and $g(x)$ and the set $S\subseteq\mathbb{R}$ we have,

[TABLE]

The left side of (152) is bounded as,

[TABLE]

where $J$ and $\hat{J}_{ij}$ are defined as $L_{1l^{\star}}(\hat{J}_{ij})=\sum_{i\in I_{1l^{\star}},j\in[N]}\lfloor h_{l^{\star}ij}\hat{J}_{ij}\rfloor$ and $\hat{J}_{ij}=\underset{J_{ij}\in J^{\star}_{ij}}{\operatorname{argmin}}|J_{ij}-\breve{\Delta}_{l^{\star}ij}|$ (Note  that  $J$  is a constant number).  $\bar{\mathcal{Z}}_{1l^{\star}}$  is  defined as the set of $[MN+\lfloor 2MN\Delta_{2}{\bar{P}}^{\max_{j\in[N],k^{\prime}\in I_{1l^{\star}}}\min(\lambda_{1k^{\prime}},\gamma_{1l^{\star}k^{\prime}j}-\delta_{1l^{\star}k^{\prime}j})}\rfloor]$. $h_{l^{\star}ij}$ are also defined as the coefficients of the linear combination $L_{1l^{\star}}$. Note that for $l_{1}=1$, the product $\prod_{l=2}^{l_{1}}\mid{\mathcal{Z}_{1l}}\mid$ is the empty product, with the value 1. Note that the denominator in (LABEL:bmn5) cannot be zero as from (131) we have, $|\breve{\Delta}_{l^{\star}ij}|\geq 2$. (LABEL:bmn5) yields from (134) and (144) is true as from (128) and (129), we have,

[TABLE]

(149) is true as $J$ is equal to $\sum_{i\in I_{1l^{\star}},j\in[N]}\lfloor h_{l^{\star}ij}\hat{J}_{ij}\rfloor$. Thus from (149), the inequality (144) is concluded as,

[TABLE]

(144) is obtained from (132). (146) follows similar to (94) and (146) is concluded as the partial sum of harmonic series can be bounded above by logarithmic function i.e., $\sum_{i=1}^{n}\frac{1}{n}\leq 1+\ln{n}$. Finally, (147) is obtained from (109) setting $k=1$,

[TABLE]

A.2.5 Proof of (126)

Now we prove the bound (126) for the general $K$ and $n$. Our goal is to prove that,

[TABLE]

Similar to A.2.4 let us define $\breve{\Delta}_{klij}(t)$, $J_{kij}(t)$, $J^{\star}_{kij}(t)$, $\hat{J}_{kij}(t)$ and $J_{k}(t)$ as

[TABLE]

and define the number $l^{\star}_{k}(t)$ as the smallest integer where

[TABLE]

Therefore, similar to A.2.4 the left side of (152) is bounded as,

[TABLE]

where (161) follows from interchange of the summation and the product 666 Note that for the arbitrary functions $f_{1}(x),f_{2}(x),\cdots,f_{n}(x)$ and the arbitrary sets of numbers $S_{1},S_{2},\cdots,S_{n}$ we have,

$\displaystyle\sum_{a_{1}\in S_{1},a_{2}\in S_{2},\cdots,a_{n}\in S_{n}}\prod_{t=1}^{n}f_{t}(a_{t})$

$\displaystyle=$ $\displaystyle\sum_{a_{1}\in S_{1}}\sum_{a_{2}\in S_{2}}\cdots\sum_{a_{n}\in S_{n}}\prod_{t=1}^{n}f_{t}(a_{t})$

(164)

$\displaystyle=$ $\displaystyle\sum_{a_{1}\in S_{1}}f_{1}(a_{1})\times\sum_{a_{2}\in S_{2}}f_{2}(a_{2})\times\cdots\times\sum_{a_{n}\in S_{n}}f_{n}(a_{n})$

(165)

$\displaystyle=$ $\displaystyle\prod_{t=1}^{n}\sum_{a_{t}\in S_{t}}f_{t}(a_{t})$

(166) .

A.3 Proof of Theorem 5

Consider the two user $(M_{1},M_{2},N_{1},N_{2})=(5,5,2,3)$ MIMO IC with $(\alpha_{11},\alpha_{12},\alpha_{21},\alpha_{22})=(1,\frac{3}{4},\frac{2}{3},1)$ and $\beta_{12}=\frac{1}{4},\beta_{21}=\frac{1}{3}$ levels of partial CSIT. The bounds $d_{1}\leq 2$ and $d_{2}\leq 3$ follow from the single user bounds. So, let us prove the bound $d_{1}+d_{2}\leq 3+\frac{7}{9}$ with the aid of sum-set inequalities.

Writing Fano’s Inequality for the first receiver we have,

[TABLE]

Multiplying the number $3$ to (167) we have,

[TABLE]

where (172) follows from Definition 2 and (172) is true from the chain rule. (172) is concluded as the entropy of a random variable is bounded by logarithm of the cardinality of it, i.e., $H((\bar{\bf Y}^{[n]}_{1})_{\frac{2}{3}}\mid(\bar{\bf Y}^{[n]}_{1})_{\frac{2}{3}}^{1},\bar{\mathbf{X}}^{[n]}_{1},\mathcal{G})\leq\frac{4}{3}n\log{\bar{P}}$, $H(\bar{\bf Y}^{[n]}_{1}\mid\mathcal{G})\leq 2n\log{\bar{P}}$. (172) is true as for any random variable $t$ and independent random variables $w_{1}$ and $w_{2}$ we have, $I(t;w_{1})\leq I(t;w_{1}\mid w_{2})$. As a result, we have $I((\bar{\bf Y}^{[n]}_{1})_{\frac{2}{3}}^{1};\bar{\mathbf{X}}^{[n]}_{1}\mid\mathcal{G})\leq I((\bar{\bf Y}^{[n]}_{1})_{\frac{2}{3}}^{1};\bar{\mathbf{X}}^{[n]}_{1}\mid\bar{\mathbf{X}}^{[n]}_{2},\mathcal{G})$. (173) yields from summation of (172) and (58) from Lemma 1. 2. 2.

Writing Fano’s Inequality for the second receiver we have,

[TABLE]

Let us remind from (50) that the received signal at the second receiver, i.e., $\bar{\mathbf{Y}}_{2}(t)$ is expressed as,

[TABLE]

summing (2) twice and (175) together, we have,

[TABLE]

(178) is concluded as the entropy of a random variable is bounded by logarithm of the cardinality of it, i.e., $H(\bar{\bf Y}^{[n]}_{2}\mid\mathcal{G})\leq{3}n\log{\bar{P}}$. (179) follows from (176) and (181) yields from the chain rule. (182) is concluded similar to (178) as $H((\bar{\bf X}^{[n]}_{2c})_{\frac{1}{2}}\mid(\bar{\bf X}^{[n]}_{2c})^{1}_{\frac{1}{2}})\leq\frac{3}{2}n\log{\bar{P}}$. Finally, (181) is obtained as,

[TABLE]

(183) yields from (49). (184) follows from independence of $(\bar{{\bf X}}^{[n]}_{1c})^{1}_{\frac{2}{3}}$ and $\bar{\mathbf{X}}^{[n]}_{2}$, and (185) is true as any $3$ components of $\bar{\bf Y}^{[n]}_{2}$ is a bounded density linear combination of random variables including components of and $(\bar{{\bf X}}^{[n]}_{1c})^{1}_{\frac{2}{3}}$ from (50). To further clarify it, let us present the following illustration of Theorem 4. For random variable $W$ independent of $\mathcal{G}$, any $n$-letter real-valued random variables $r_{1}^{[n]},r_{2}^{[n]},r_{3}^{[n]}$ independent of $\mathcal{G}$ and $n$-letter real-valued random variable $s_{1}^{[n]},s_{2}^{[n]},s_{3}^{[n]},s_{4}^{[n]}$ where for any $t\in[n]$,

[TABLE]

we have

[TABLE]

(185) is concluded from (190) 777 This also could be concluded from Theorem 1 in [10].. 3. 3.

Summing (173) and (182) together we have,

[TABLE]

Dividing (191) by $3\log{\bar{P}}$, $d_{1}+d_{2}\leq 3+\frac{7}{9}$ is concluded.

In order to prove the region (57), the bound $\frac{d_{1}}{2}+\frac{d_{2}}{3}\leq{\frac{3}{2}}$ is proved as follows.

Writing Fano’s Inequality for the first and second receivers we have,

[TABLE]

summing (192) three times and (193) twice we have,

[TABLE]

where (195) is true as the entropy of a random variable is bounded by logarithm of the cardinality of it, i.e., $H(\bar{\bf Y}^{[n]}_{1}\mid\mathcal{G})\leq 2n\log{\bar{P}}$. (197) is concluded from sub-modularity properties of entropy function, i.e., for any $m$ random variables $\{X_{1},X_{2},\cdots,X_{m}\}$ where we define $X_{k+m}$ as $X_{k}$ for positive numbers of $k$ we have,

[TABLE]

if $n\leq m$. To prove (198), consider any of the three entropies $H(\bar{Y}^{[n]}_{21},\bar{Y}^{[n]}_{22}\mid\bar{\mathbf{X}}^{[n]}_{1},\mathcal{G})$, $H(\bar{Y}^{[n]}_{21},\bar{Y}^{[n]}_{23}\mid\bar{\mathbf{X}}^{[n]}_{1},\mathcal{G})$ and $H(\bar{Y}^{[n]}_{22},\bar{Y}^{[n]}_{23}\mid\bar{\mathbf{X}}^{[n]}_{1},\mathcal{G})$. For instance, consider $H(\bar{Y}^{[n]}_{21},\bar{Y}^{[n]}_{23}\mid\bar{\mathbf{X}}^{[n]}_{1},\mathcal{G})$ and bound it as,

[TABLE]

(200) yields from the chain rule and (202) is true as conditioned on $\bar{\mathbf{X}}^{[n]}_{1}$, the random variables $(\bar{Y}_{21}(t))_{\frac{1}{2}}^{1}$ and $(\bar{Y}_{23}(t))_{\frac{1}{2}}^{1}$ are bounded density linear combinations of random variables $(\bar{{X}}^{[n]}_{23})^{1}_{\frac{1}{2}}$, $(\bar{{X}}^{[n]}_{24})^{1}_{\frac{1}{2}}$ and $(\bar{{X}}^{[n]}_{25})^{1}_{\frac{1}{2}}$ while $\bar{Y}_{11}(t)$ and $\bar{Y}_{12}(t)$ are bounded density linear combinations of random variables $(\bar{{X}}^{[n]}_{21})^{1}_{\frac{1}{4}}$, $(\bar{{X}}^{[n]}_{22})^{1}_{\frac{1}{4}}$, $(\bar{{X}}^{[n]}_{23})^{1}_{\frac{1}{2}}$, $(\bar{{X}}^{[n]}_{24})^{1}_{\frac{1}{2}}$ and $(\bar{{X}}^{[n]}_{25})^{1}_{\frac{1}{2}}$, see (50). Thus, (200) is concluded similar to (185). Dividing (198) by $6\log{\bar{P}}$, $\frac{d_{1}}{2}+\frac{d_{2}}{3}\leq{\frac{3}{2}}$ is concluded.

Appendix B Proof of Lemma 1

As $(\bar{\mathbf{X}}^{[n]}_{2c})^{1}_{\frac{1}{2}}$ is independent from $\bar{\mathbf{X}}^{[n]}_{1}$, (58) can be written as,

[TABLE]

(203) follows from the chain rule, i.e.,

[TABLE]

Starting from the left side of (203) we have,

[TABLE]

(205) follows from the definition of $\bar{\mathbf{X}}^{[n]}_{2c}$. (209) and (209) is true from the chain rule. (209) follows similar to (197) from sub-modularity properties of entropy function. Finally, (210) is true from Theorem 3 as any of the three entropies in (209) are less than $H(\bar{\bf Y}_{1}^{[n]}\mid\bar{\mathbf{X}}^{[n]}_{1},\mathcal{G})+n~{}o~{}(\log{\bar{P}})$, i.e.,

[TABLE]

To further illuminate how (211) is concluded from Theorem 4, define $Z_{1}(t),Z_{2}(t),Z_{11}(t),Z_{12}(t),Z_{21}(t),Z_{22}(t)$ and $W$ from (49) for all $t\in[n]$ as,

[TABLE]

where $\lambda_{1},\lambda_{2}$ and the singleton sets of $I_{1},I_{2}$ are derived as

[TABLE]

So, the condition (37) is satisfied and (211) is concluded from Theorem 4, i.e.,

[TABLE]