Ergodic MIMO Mutual Information: Twenty Years After Emre Telatar

TL;DR

This paper revisits Telatar's 1999 integral representation of MIMO mutual information, explicitly evaluates it as a finite sum involving exponential integrals, and connects it to recent quantum information theory developments.

Contribution

It provides an explicit finite-sum evaluation of the expected mutual information for MIMO channels, enhancing the classical integral representation with closed-form expressions.

Findings

Explicit finite sum representation derived

Involves exponential integral functions

Connects classical information theory with quantum information developments

Abstract

In the celebrated work of Emre Telatar in the year 1999 (14274 citations to date), it was shown that the expected value of the mutual information \begin{equation*} \mathrm{I} = \ln\det\left( \mathbf{I}_m + \frac{1}{t} \mathbf{HH}^{\dagger} \right) \end{equation*} of an $m\times n$ MIMO Rayleigh channel matrix $\mathbf{H}$ with a SNR $1/t$ can be represented as an integral involving Laguerre polynomials. We show, in this work, that Telatar's integral representation can be explicitly evaluated to a finite sum of the form \begin{equation*} \mathbb{E}\!\left[\mathrm{I}\right]=\sum_{k=0}^{n+m-3}a_{k}t^{k}+\rm e^{t}~\text{Ei}(-t)\sum_{k=0}^{n+m-2}b_{k}t^{k},, \end{equation*} where $\text{Ei}(-t)$ is the exponential integral and $a_{k}$, $b_{k}$ are known constants that do not dependent on $t$. The renewed interest in this classical information theory problem came from, quite surprisingly, the recent development in quantum information theory.

Tables

Table 1. TABLE I: Ergodic Mutual Information ( 7 ): Some Examples

$m=2$	$n=2$	$1-t-e^t\text{Ei}\left(-t\right)\left(2+t^2\right)$
	$n=4$	$\frac{1}{6}\left(20-6t-t^2-t^3-e^t\text{Ei}\left(-t\right)(12-12t+6t^2+2t^3+t^4)\right)$
	$n=6$	$\frac{1}{120}\left(524-180t+48t^2-8t^3-3t^4-t^5-e^t\text{Ei}\left(-t\right)(240-240t+120t^2-40t^3+10t^4+4t^5+t^6)\right)$
$m=4$	$n=4$	$\frac{1}{36}\left(156-156t-96t^2-56t^3-11t^4-t^5-e^t\text{Ei}\left(-t\right)(144+216t^2+144t^3+66t^4+12t^5+t^6)\right)$
	$n=6$	$\frac{1}{720}(5544-1440t-720t^2-1600t^3-756t^4-186t^5-21t^6-t^7-e^t\text{Ei}(-t)\times$
		$(2880-2880t+1440t^2+1920t^3+2200t^4+924t^5+206t^6+22t^7+t^8))$

Equations

$$\mathrm{I}=\ln \det \left({\mathbf{I}}_m+\frac{1}{t}{\mathbf{HH}}^{†}\right)$$

$$\mathbb{E}\left[\mathrm{I}\right]=\sum _{k=0}^{n+m-3}{a}_k{t}^k+{\mathrm{e}}^t\text{Ei}(-t)\sum _{k=0}^{n+m-2}{b}_k{t}^k,$$

$$\mathrm{I}=\ln \det \left({\mathbf{I}}_m+\frac{1}{t}{\mathbf{HH}}^{†}\right),$$

$$\mathrm{I}=\sum _{i=1}^m\ln \left(1+\frac{1}{t}{\lambda }_i\right).$$

$$\mathbb{E}\left[\mathrm{I}\right]=m\mathbb{E}\left[\ln \left(1+\frac{1}{t}\lambda \right)\right],$$

$$p(\lambda )=\frac{1}{m}{\mathrm{e}}^{-\lambda }{\lambda }^{n-m}\sum _{k=0}^{m-1}\frac{k!}{(n-m+k)!}{\left(L_k^{(n-m)}(\lambda )\right)}^2$$

$${\int }_0^{\infty }{x}^{\alpha }{\mathrm{e}}^{-x}{L}_k^{(\alpha )}(x)L_l^{(\alpha )}(x)\mathrm{d}x=\frac{(\alpha +k)!}{k!}{\delta }_{kl},$$

$$\mathbb{E}\left[\mathrm{I}\right]=\sum _{k=0}^{m-1}\frac{k!}{(n-m+k)!}{\int }_0^{\infty }\ln \left(1+\frac{1}{t}\lambda \right)\times$$

$${\mathrm{e}}^{-\lambda }{\lambda }^{n-m}{\left(L_k^{(n-m)}(\lambda )\right)}^2\mathrm{d}\lambda .$$

$$a_k$$

$$b_k$$

$$c_{ij}$$

$$\mathbb{E}\left[\mathrm{I}\right]=\sum _{k=0}^{n+m-3}{a}_k{t}^k+{\mathrm{e}}^t\text{Ei}(-t)\sum _{k=0}^{n+m-2}{b}_k{t}^k,$$

$$\text{Ei}(x)=-{\int }_{-x}^{\infty }\frac{{\mathrm{e}}^{-s}}{s}\mathrm{d}s,$$

$$\mathrm{I}=-m\ln t+\sum _{i=1}^m\ln \left(t+{\lambda }_i\right)$$

$$\mathbb{E}\left[\mathrm{I}\right]=-m\ln t+m{\int }_0^{\infty }\ln \left(t+\lambda \right)p(\lambda )\mathrm{d}t.$$

$$p(\lambda )$$

$$\mathbb{E}\left[\mathrm{I}\right]=-m\ln t+\frac{m!}{(n-1)!}\left(A_{m-1,m-1}\left(t\right)-A_{m-2,m}\left(t\right)\right),$$

$$A_{s,t}\left(t\right)$$

$${\int }_t^{\infty }{x}^k{\mathrm{e}}^{-x}\ln x\mathrm{d}x=\Gamma \left(k+1,t\right)\ln t+k!\sum _{s=0}^k\frac{\Gamma \left(s,t\right)}{s!},$$

$$\Gamma \left(s,t\right)=(s-1)!{\mathrm{e}}^{-t}\sum _{i=0}^{s-1}\frac{t^i}{i!}$$

$$L_k^{(\alpha )}(x)=\sum _{i=0}^k(-1{)}^i(\genfrac{}{}{0ex}{}{\alpha +k}{k-i})\frac{x^i}{i!},$$

$$A_{s,t}\left(t\right)$$

$$\mathbb{E}\left[\mathrm{I}\right]$$

$$m$$

$$\sum _{k=0}^N\left(\genfrac{}{}{0ex}{}{N}{k}\right)(-t{)}^{-k}\Gamma (k+1,t)$$

$$\sum _{k=0}^N\frac{(-N{)}_k(a{)}_k}{k!(b{)}_k}$$

$$\mathbb{E}\left[\mathrm{I}\right]$$

$$-\sum _{i=0}^{2m-2}\sum _{j=0}^i{c}_{ij}\sum _{k=0}^{i+n-m}\left(\genfrac{}{}{0ex}{}{i+n-m}{k}\right)(-t{)}^{i+n-m-k}k!$$

$$=$$

$$-\sum _{k=0}^{n-m}\frac{(-t{)}^k}{k!}\sum _{j=0}^{m-1}\sum _{i=j}^{2m-2}(i+n-m)!c_{ij}-$$

$${\mathrm{e}}^t\sum _{i=0}^{2m-2}\sum _{j=0}^i{c}_{ij}\sum _{k=0}^{i+n-m}\left(\genfrac{}{}{0ex}{}{i+n-m}{k}\right)\times$$

$$\sum _{j=0}^{m-1}\sum _{i=-n+m+1}^{2m-2}{c}_{ij}\underset{ϵ\to 0}{\lim }\frac{(i+n-m)!-ϵ!(i+n-m-ϵ)!}{ϵ}$$

Full text

[TABLE]

of an $m\times n$ MIMO Rayleigh channel matrix $\mathbf{H}$ with a SNR $1/t$ can be represented as an integral involving Laguerre polynomials. We show, in this work, that Telatar’s integral representation can be explicitly evaluated to a finite sum of the form

[TABLE]

where $\text{Ei}(-t)$ is the exponential integral and $a_{k}$, $b_{k}$ are known constants that do not dependent on $t$. The renewed interest in this classical information theory problem came from, quite surprisingly, the recent development in quantum information theory.

I Introduction

The notion of mutual information is among the most important quantities in information theory. Its supremum value provides the fundamental performance measure, the channel capacity. For Multiple-Input-Multiple-Output (MIMO) Rayleigh fading channel, that corresponds to a rich scattering environment, it is a well-known result [1, Theorem 1] that the ergodic capacity is achieved when the input follows a complex Gaussian distribution with zero mean and covariance proportional to an identity matrix. The operational meaning of ergodic capacity is that the transmission of each codeword has experienced sufficient many channel realizations, i.e., a fast fading scenario. Mathematically, the ergodic capacity of Rayleigh fading channels is given by the expected value of the mutual information as [1]

[TABLE]

where $1/t$ is the Signal-to-Noise Ratio (SNR) and the entries of the $m\times n$ channel matrix $\mathbf{H}$ spanned by the pathes between $n$ transmit and $m$ receive antennas are independent and follow a complex Gaussian distribution with zero mean and unit variance. Here, $\ln(\cdot)$ is the natural logarithm with the unit of (1) being nats/second/Hz, $\det(\cdot)$ is the matrix determinant, and it can be assumed without loss of generality that $m\leq n$.

Denote the eigenvalues of the Hermitian matrix $\mathbf{HH}^{{\dagger}}$ by $0<\lambda_{m}<\dots<\lambda_{2}<\lambda_{1}<\infty$, the mutual information is rewritten as

[TABLE]

By the symmetry of (2) in all the eigenvalues, the ergodic mutual information is evaluated as

[TABLE]

where the right-hand-side expectation is now taken over the density of an arbitrary eigenvalue (a.k.a. the one-point density) given by [2, 1]

[TABLE]

with $\lambda\in[0,\infty)$. Here, $L_{k}^{(\alpha)}(x)$ is the Laguerre orthogonal polynomial of degree $k$ in $x$ that satisfies the orthogonality relation [2]

[TABLE]

where $\delta_{kl}$ is the Kronecker delta function. Inserting (4) into (3), one arrives at an integral representation of the ergodic mutual information of MIMO Rayleigh channels as

[TABLE]

In fact, the above result was stated as Theorem 2 in the seminal work by Telatar [1].

In this paper, we show that Telatar’s representation (I) can be calculated to an explicit form involving, up to a factor ${\rm e}^{t}~{}\text{Ei}(-t)$, finite polynomials in $t$ as presented in the next section. The corresponding proof will be outlined in Section III. In Section IV, we point out some further work that may be carried out based on the obtained results.

II Main Result

The main result of this paper is summarized in the following proposition.

Proposition 1**.**

The ergodic mutual information of MIMO Rayleigh fading channels with $n$ transmit antennas, $m$ receive antennas, and any SNR value $1/t$, i.e., the expected value of the random variable (1), is given by

[TABLE]

where

[TABLE]

is the exponential integral function. The constants $a_{k}$ and $b_{k}$ are given by (9) and (10), respectively, as shown on top of this page.

Proposition 1 reveals that the ergodic mutual information of MIMO Rayleigh channels consists of finite polynomials in the inverse of SNR $t$ with a common factor ${\rm e}^{t}~{}\text{Ei}(-t)$ independent of the channel dimensions. The degree of the polynomials depends essentially on the sum of the channel dimensions $m+n$. The coefficients of the polynomials $a_{k}$ and $b_{k}$ in turn are given by finite sums of factorial functions. Clearly, these coefficients can be efficiently computed for any dimensions $m$ and $n$ with no special functions involved. The unveiled simple structure (7) of the ergodic mutual information is far from clear a priori. The functional dependence on the SNR is explicitly displayed. This facilitates further analysis on the statistical behavior of mutual information when the SNR is modeled as a random variable. The SNR variations may result from, for example, transceiver schemes or physical channels.

Before proving the Proposition 1, we list in Table I some examples computed from the derived formula (7) of different channel dimensions as shown on top of the next page. We also point out that extensive simulation has been performed that numerically verified the main result (7). Interested readers may contact the author to obtain the code implemented in Mathematica.

III Proof of the Main Result

The proof of Proposition 1 involves two steps with the starting point being Telatar’s representation (I). The first step is to manipulate the integrand of (I) to a form such that the integral can be conveniently calculated. The second step is to simplify, with the help of special function theory, the multiple nested summations produced in the first step. This two-step procedure has been recently utilized in solving a related problem111Instead of mutual information, the random variable of interest in [3] is $\sum_{i=1}^{m}\lambda_{i}\ln\lambda_{i}$ known as the von Neumann entropy. Here, the presence of $\lambda_{i}$ in front of the logarithm is not essential, which is absorbed in the density (4). Thus, the considered mutual information (2) is a more generalized quantity, i.e., a one parameter $t$ deformation to the von Neumann entropy, cf. (12). in quantum information theory [3].

For the first step, we start by rewriting (2) as

[TABLE]

so that by (3) the expected value becomes

[TABLE]

To proceed, we use the fact that the one-point density (4) utilized in Telatar’s result (I) admits a more convenient form [2] with no summation involved

[TABLE]

Inserting (14) into (13) and by changing the variable $x\to x-t$, we have

[TABLE]

where we denote

[TABLE]

Since the product $(x-t)^{n-m}L_{s}^{(n-m+1)}(x-t)L_{t}^{(n-m+1)}(x-t)$ is a polynomial in $x$, to evaluate (16) it remains to see if an integral of the form $\int_{t}^{\infty}x^{k}{\rm e}^{-x}\ln x\,\mathrm{d}x$ can be calculated. This turns out to be possible, and the result is given in the lemma below.

Lemma 1**.**

For any non-negative integer $k$, we have

[TABLE]

where

[TABLE]

defines the incomplete Gamma function for a positive integer $s$.

The proof of Lemma 1 is in the Appendix. By Lemma 1 and the series form of the Laguerre polynomial

[TABLE]

the integral (16) is computed as in (20) shown on top of the next page.

Inserting (20) into (15) and by factoring out from $A_{m-1,m-1}$ and $A_{m-2,m}$ the common term $c_{ij}$ as shown in (11), we obtain

[TABLE]

We have so far carried out the first step of the proof, i.e., computing the integral in Telatar’s formula (I). The second step is to simplify the obtained expression (21). We first show that the two terms involving $\ln t$ in (21) cancel, i.e., the equality below holds

[TABLE]

The above equality is established by sequentially applying the summation formulas below

[TABLE]

where the first formula follows from the definition (18) and the second one is known as the Chu-Vandermonde identity (see, e.g., [3]) with $(a)_{n}=\Gamma(a+n)/\Gamma(a)$ denoting the Pochhammer’s symbol. With the result (22) and the fact that $\Gamma(0,t)=-\text{Ei}(-t)$ (cf. (8) and (35)), we can write (21) as

[TABLE]

where the coefficients $a_{k}$ and $b_{k}$ are now related to the coefficients of the first and second term in the last bracket in (25), respectively. Putting aside the factor ${\rm e}^{t}\text{Ei}(-t)$, the coefficients $b_{k}$ shown in (10) are identified from (25) as

[TABLE]

where the last equality follows by changing orders of the summations. Due to the presence of the factor $1/(m-1-j)!$ in $c_{ij}$, the upper bound of the sum over $j$ in (27) becomes $m-1$. Further simplification is possible by splitting the sum over $k$ into two sums over $k=0,\dots,n-m$ and $k=n-m+1,\dots,n+m-2$, where the sum over $i$ in the first $k$-sum now starts from $j$ due to the term $1/(i-j)!$ in $c_{ij}$. These lead (27) to

[TABLE]

where we have used (22). From (28), the coefficients of $t^{k}$ can be read off as shown in (10). To identify the coefficients $a_{k}$, we repeatedly change the order of summations and apply the Chu-Vandermonde identity (24) to the corresponding term in (25) as

[TABLE]

From the above equation, the coefficients $a_{k}$ for $k\neq 0$ can be read off as shown in (9). For $k=0$, the equation (29) is indeterminant, which is resolved by using l*′*Hôpital’s rule as

[TABLE]

where, in addition to adjust the summation bound according to $c_{ij}$, we have also utilized the definition of digamma function

[TABLE]

and its finite sum form for positive arguments (see, e.g., [3])

[TABLE]

with $\gamma\approx 0.5772$ being Euler’s constant. Indeed, (30) is the coefficient $a_{0}$ as claimed in (9). This completes the proof of Proposition 1.

IV Conclusion and Future Work

We have revisited the problem of finding a closed-form formula for the ergodic mutual information of MIMO Rayleigh channels initially studied by Telatar twenty years ago. Interestingly, our motivation came from the recent progress of a similar problem in quantum information theory. We have shown that the seemingly complicated end-result by Telatar in fact admits a simple structure. The main result of this work lies on explicitly determining this simple structure, which has been summarized in Proposition 1.

We finish this paper by mentioning some future research directions based on the derived results. Firstly, as a direct application we could analyze the ergodic mutual information for models with SNR variation, where the effective SNR is modeled as a random variable. The analysis is possible as the revealed dependence on SNR $1/t$ in (7) is explicit, where, conditioned on any $t$, operations such as integration and differentiation can be easily performed. Besides modeling of physical channels, the need to take into account SNR variation comes from transceiver schemes such as modulation, coding, or channel estimation.

The considered ergodic mutual information corresponds to the first moment, whereas higher moments of mutual information are needed to describe the outage probability relevant to slow or block fading. In particular, finding the exact second moment (fluctuation) remains a long-standing open problem. Our preliminary study indicates that the higher moments also admit a similar finite polynomial structure as (7), where the difference is that the coefficients including $a_{k}$, $b_{k}$ and the common factor ${\rm e}^{t}~{}\text{Ei}(-t)$ become more complicated.

Last but not least, the proposed analytical framework can be also utilized to calculating the moments of mutual information of other MIMO channel models. For example, we expect that the Jacobi MIMO channel relevant to MIMO optical communications can be similarly studied, where, instead of the Laguerre polynomials, the Jacobi polynomials will be utilized.

Proof of Lemma 1

Denote the integral in (17) by $g(k)$, integration by parts gives

[TABLE]

where $f^{\prime}\left(x\right)=\,\mathrm{d}f(x)/\,\mathrm{d}x$ and we have used the integral form of incomplete Gamma function

[TABLE]

Equation (34) implies a recurrence relation of $g(k)$,

[TABLE]

By iterating $k-1$ times the above relation, we arrive at

[TABLE]

where the second equality is obtained by (18). Using integration by parts, the integral $g(0)$ is calculated as

[TABLE]

where we have used the definition (35) for $k=0$. Inserting (40) into (37), we arrive at the claimed result (17).