A double EP-based proposal for turbo equalization

TL;DR

This paper introduces a double expectation propagation (EP) based turbo equalizer that reduces iteration count and computational complexity, offering three implementation variants and demonstrating improved performance over existing EP-based methods.

Contribution

It proposes a novel double EP-based equalizer combining multiple approaches to enhance turbo equalization efficiency and reduce complexity compared to prior EP-based solutions.

Findings

Reduces the number of iterations needed for turbo equalization.

Achieves computational complexity twice that of linear MMSE.

Shows improved performance over existing EP-based proposals.

Abstract

This letter deals with the application of the expectation propagation (EP) algorithm to turbo equalization. The EP has been successfully applied to obtain either a better approximation at the output of the equalizer or at the output of the channel decoder to better initialize the Gaussian prior used by the equalizer. In this letter we combine both trends to propose a novel double EP-based equalizer that is able to decrease the number of iterations needed, reducing the computational complexity to twice that of the linear MMSE. This novel equalizer is developed in three different implementations: a block design that exploits the whole vector of observations, a Wiener filter-type approach that just uses the observations within a predefined window and a Kalman smoothing filter-type approach that emulates the BCJR behavior. Finally, we include some experimental results to compare the three different implementations and to detail their improvements with respect to other EP-based proposals in the literature.

Equations

$$\underset{\mathbf{y}}{\underbrace{\left[\begin{array}{c}{y}_1\\ \\ ⋮\\ \\ y_{\mathtt{N}+\mathtt{L}-1}\end{array}\right]}}=\underset{\mathbf{H}}{\underbrace{\left[\begin{array}{cccc}{h}_1& & & 0\\ ⋮& \ddots & & \\ h_{\mathtt{L}}& \ddots & & h_1\\ & \ddots & & ⋮\\ 0& & & h_{\mathtt{L}}\end{array}\right]}}\underset{\mathbf{u}}{\underbrace{\left[\begin{array}{c}{u}_1\\ \\ ⋮\\ \\ u_{\mathtt{N}}\end{array}\right]}}+\underset{\mathbf{w}}{\underbrace{\left[\begin{array}{c}{w}_1\\ \\ ⋮\\ \\ w_{\mathtt{N}+\mathtt{L}-1}\end{array}\right]}}$$

$$p(\mathbf{u}|\mathbf{y},\mathbf{H})\propto p(\mathbf{y}|\mathbf{u},\mathbf{H})\prod _{k=1}^{\mathtt{N}}p(u_k)$$

$$t_k^{[\ell ]}(u_k)=\mathcal{C}\mathcal{N}\left(u_k:{\mu }_{t_k}^{[\ell ]},{\sigma }_{t_k}^{2[\ell ]}\right),$$

$$q^{[\ell ]}(\mathbf{u})$$

$$q^{[\ell ]}(u_k)\sim \mathcal{C}\mathcal{N}\left(u_k:{\mu }_k^{[\ell ]},{\sigma }_k^{2[\ell ]}\right)$$

$${\mu }_k^{[\ell ]}$$

$${\sigma }_k^{2[\ell ]}$$

$$q_E^{[\ell ]}(u_k)=q^{[\ell ]}(u_k)/t_k^{[\ell ]}(u_k)=\mathcal{C}\mathcal{N}\left(u_k:{\mu }_{E_k}^{[\ell ]},{\sigma }_{E_k}^{2[\ell ]}\right)$$

$${\mu }_{E_k}^{[\ell ]}=\frac{{\mu }_k^{[\ell ]}{\sigma }_{t_k}^{2[\ell ]}-{\mu }_{t_k}^{[\ell ]}{\sigma }_k^{2[\ell ]}}{{\sigma }_{t_k}^{2[\ell ]}-{\sigma }_k^{2[\ell ]}},{\sigma }_{E_k}^{2[\ell ]}=\frac{{\sigma }_k^{2[\ell ]}{\sigma }_{t_k}^{2[\ell ]}}{{\sigma }_{t_k}^{2[\ell ]}-{\sigma }_k^{2[\ell ]}}.$$

$${\hat{p}}^{[\ell ]}(u_k)\propto q_E^{[\ell ]}(u_k)p(u_k),$$

$${\sigma }_{t_k,new}^{2[\ell +1]}=\frac{{\sigma }_{{\hat{p}}_k}^{2[\ell ]}{\sigma }_{E_k}^{2[\ell ]}}{{\sigma }_{E_k}^{2[\ell ]}-{\sigma }_{{\hat{p}}_k}^{2[\ell ]}},{\mu }_{t_k,new}^{[\ell +1]}=\frac{{\mu }_{{\hat{p}}_k}^{[\ell ]}{\sigma }_{E_k}^{2[\ell ]}-{\mu }_{E_k}^{[\ell ]}{\sigma }_{{\hat{p}}_k}^{2[\ell ]}}{{\sigma }_{E_k}^{2[\ell ]}-{\sigma }_{{\hat{p}}_k}^{2[\ell ]}}.$$

$${\sigma }_{t_k}^{2[\ell +1]}$$

$${\mu }_{t_k}^{[\ell +1]}$$

$${\sigma }_{t_k}^{2[\ell +1]}={\sigma }_{t_k}^{2[\ell ]},{\mu }_{t_k}^{[\ell +1]}={\mu }_{t_k}^{[\ell ]}.$$

$$t_k^{[1]}(u_k)=\text{mbox}{Proj}_G[p(u_k)]\sim \mathcal{C}\mathcal{N}\left(u_k:{\mu }_{t_k}^{[1]},{\sigma }_{t_k}^{2[1]}\right)$$

$$t_k^{[1]}(u_k)=\frac{\text{mbox}{Proj}_G[{\hat{p}}^{[S+1]}(u_k)]}{q_E^{[S+1]}(u_k)}\sim \mathcal{C}\mathcal{N}\left(u_k:{\mu }_{t_k}^{[1]},{\sigma }_{t_k}^{2[1]}\right),$$

$${\mu }_{t_k}^{[1]}$$

$${\sigma }_{t_k}^{2[1]}$$

Full text

I Introduction

Current digital transmissions are corrupted by intersymbol interference (ISI) introduced by the dispersive nature of the channels, which negatively affects the received signal. This corrupted signal is processed by the equalizer, that provides an estimation of the transmitted symbols [1]. These estimations can be probabilistic, resulting in a high benefit for modern channel decoders. In addition, the equalizer and channel decoder can exchange information to improve the estimation, which is known as turbo equalization [2].

One optimal solution used in turbo equalization is the BCJR [3], that obtains the maximum a posteriori (MAP) probabilities for each transmitted symbol. However, its computational cost increases exponentially with the number of symbols of the constellation and/or the length of the channel, becoming intractable for large channels or high-order modulations. In this situation, some approximate inference techniques, such as the linear minimum mean square error (LMMSE), are employed.

In turbo equalization, the LMMSE obtains a tractable Gaussian approximation for the a posteriori probabilities (APP) by assuming a Gaussian distribution for the prior according to the channel decoder output. It can be developed in a block [4], Wiener filter-type (WF) [5] and Kalman smoothing (KS) [6] implementations. However, its performance is far from optimal. The expectation propagation (EP) [7, 8] algorithm is a Bayesian inference technique that has been recently applied to turbo equalization to improve the LMMSE performance in its block [9, 10], WF [10] and KS [11, 12, 13, 14] implementations. The EP shares the structure of the LMMSE, where the estimated Gaussian priors depend on the observations, hence the equalizer is non linear.

In [13] the EP is developed from a message passing point of view to better approximate with Gaussians the discrete outputs of the channel decoder. To avoid negative variances, they are set to their absolute values. However, this equalizer does not improve the equalization step by itself since it boils down to the LMMSE for standalone equalization, i.e., if no turbo equalization is carried out. In contrast, in [9, 11, 10, 12, 14] the output of the decoder is directly projected into the family of Gaussians, as the turbo LMMSE does, i.e., the EP is not applied at the output of the decoder. Instead, the EP is used to obtain a better Gaussian approximation for the extrinsic distribution at the output of the equalizer, before sending it to the channel decoder. This equalizer improves the performance of the LMMSE and the EP in [13], either as standalone or turbo equalization. In other words, in [9, 11, 10, 12, 14] the EP is used at an inner loop while in [13] it is used at an outer loop, as will be explained in Section III.

In this letter we take advantage of both trends and combine them into a novel double EP-based equalizer. It applies the EP algorithm twice, within the inner and outer loops, outperforming the LMMSE either as standalone or turbo equalization. Also, we improve the control of negative variances proposed in [13] at the outer loop by setting them to the moments of the information at the output of the channel decoder in case of negative values. In terms of bit error rate (BER), this novel equalizer outperforms previous EP-based proposals. Furthermore, it exhibits a significant reduction in complexity. Finally, we include some experimental results to show the improvements of this novel double EP equalizer with respect to the others found in the literature [9, 11, 10, 12, 13]. These experiments also include a comparison between the three different implementations (block, WF and KS) of an EP-based equalizer after averaging over different random channels.

II System Model

A sequence of information bits, $\mathbf{a}=[a_{1},...,a_{{K}}]^{\top}$ where $a_{i}\in\{0,1\}$, is encoded into the codeword $\mathbf{b}$. This codeword is partitioned into $\mathtt{N}$ blocks of length $Q=\log_{2}(\mathtt{M})$ as $\mathbf{b}=[\mathbf{b}_{1},...,\mathbf{b}_{\mathtt{N}}]^{\top}$, where $\mathbf{b}_{k}=[b_{k,1},...,b_{k,Q}]$. Each block $\mathbf{b}_{k}$ is then modulated into a symbol that belongs to a complex $\mathtt{M}$-ary constellation with alphabet $\mathcal{A}$ and mean transmitted symbol energy $E_{s}$. This yields the vector of symbols $\mathbf{u}=[u_{1},...,u_{\mathtt{N}}]^{\top}$ that is transmitted over a channel $\mathbf{h}=[h_{1},...,h_{\mathtt{L}}]$ and it is corrupted with additive white Gaussian noise (AWGN) whose variance, $\sigma_{w}^{2}$, is known. The received signal is given by

[TABLE]

where ${w_{k}}\sim\mathcal{CN}\!\!\left({{w_{k}}:{{0},\sigma_{w}^{2}}}\right)$. This signal is received by the equalizer that estimates the posterior probability of the transmitted symbol vector as

[TABLE]

where $p(\mathbf{y}|\mathbf{u},\mathbf{H})=\mathcal{CN}\!\!\left({\mathbf{y}:{\mathbf{H}\mathbf{u},\sigma_{w}^{2}{\mathbf{I}}}}\right)$ is the likelihood and $p(u_{k})$ is the information on the priors. If the output of the channel decoder is not available, we may assume equiprobable symbols, which is equivalent to setting the prior to a uniform distribution. If available, the information at the output of the channel decoder is fed back to the equalizer and used to update the priors. Note that the computational complexity of (2) is proportional to $\mathtt{M}^{\mathtt{L}}$ due to the discrete nature of the symbols.

In turbo equalization, the equalizer and channel decoder usually exchange extrinsic information. Hence, the equalizer outputs an extrinsic distribution that it is then demapped and given to the channel decoder as extrinsic log-likelihood ratios (LLRs), $L_{E}(b_{k,j})$.

III Double EP turbo equalizer

The EP algorithm is a Bayesian framework used to approximate a non-tractable distribution, such as (2), with exponential distributions. Recently research works successfully apply this tool to develop a turbo equalizer [9, 11, 10, 12, 13, 14]. In [9, 11, 10, 12], the EP is used to better approximate the posterior (or extrinsic) distribution at the output of the equalizer. To that end, the non Gaussian factors in (2) are replaced by Gaussians, denoted as

[TABLE]

yielding the following Gaussian posterior distribution

[TABLE]

The marginal of (4) yields another Gaussian distribution, that we will denote as

[TABLE]

where [10]

[TABLE]

$\mathbf{h}_{\mathbf{k}}$ is the $k$-th column of the matrix $\mathbf{H}$ defined in (1), ${\boldsymbol{\Sigma}_{t}^{[\ell]}}=\mathrm{diag}([\sigma_{t_{1}}^{2[\ell]},\ldots,\sigma_{t_{\mathtt{N}}}^{2[\ell]}])$ and $\boldsymbol{\upmu}_{t}^{[\ell]}=[\mu_{t_{1}}^{[\ell]},\ldots,\mu_{t_{\mathtt{N}}}^{[\ell]}]^{\top}$.

From (5), one can also compute the extrinsic distribution as

[TABLE]

where

[TABLE]

The moments of the Gaussian factors in (4) are updated in parallel and iteratively by matching the moments of the discrete posterior,

[TABLE]

and the approximated one, $q_{E}^{[\ell]}(u_{k})t_{k}^{[\ell+1]}(u_{k})$. A detailed explanation of this procedure is described in Alg. 1. It also includes a damping factor ($\beta$), a minimum allowed variance ($\epsilon$) and a control of negative variances to improve convergence and control instabilities. This procedure is repeated iteratively and we will refer to it as inner loop.

After $S$ iterations of the previous EP procedure, the extrinsic distributions, $q_{E}^{[S+1]}(u_{k})$, are sent to the channel decoder, whose output is fed back to the equalizer and used to update the information on the priors, $p(u_{k})$. This procedure is repeated along ${T}$ iterations and we will refer to it as outer loop. Since the information provided by the channel decoder, $p(u_{k})$, is discrete, the first step of the equalizer is to find an initial Gaussian approximation, $t_{k}^{[1]}(u_{k})$. In [9, 11, 10, 12], this Gaussian approximation is obtained by projecting $p(u_{k})$ into the family of Gaussians, as the turbo LMMSE does, i.e.,

[TABLE]

where $\mu_{t_{k}}^{[1]}=\mathbb{E}_{p}[u_{k}]$ and $\sigma_{t_{k}}^{2[1]}=\mathbb{E}_{p}[(u_{k}-\mu_{t_{k}}^{[1]})^{2}]$.

In this manuscript, we propose a different Gaussian approximation for $p(u_{k})$ that has the same computational complexity as (15) and more accurate results. Specifically, we apply EP at the output of the decoder, as introduced by the proposal in [13]. Since we have already proposed the use of EP in the inner loop, we are proposing a second EP that takes as a starting point the extrinsic distribution that was given to the channel decoder at the previous turbo iteration, $q_{E}^{[S+1]}(u_{k})$, yielding

[TABLE]

where

[TABLE]

and $(\mu_{\widehat{p}_{k}}^{[S+1]},\sigma_{\widehat{p}_{k}}^{2[S+1]})$ are the moments of $\widehat{p}^{[S+1]}(u_{k})$, defined in (10). If the variances in (18) lead to negative values we replace the moments of (16) by the results of the method in (15). The whole procedure is detailed in Alg. 2.

Note that in this proposal the EP is applied twice, as showed in Fig. 1. We first use EP in an inner loop ($\ell=1,..,S$) to obtain a Gaussian extrinsic distribution at the output of the equalizer, $q_{E}^{[\ell]}(u_{k})$. This use of EP is plotted as a gray block named EP1. Then, a second EP is used within an outer loop ($t=0,...,{T}$) to find an initial Gaussian approximation, $t_{k}^{[1]}(u_{k})$, for the discrete information at the output of the channel decoder, see EP2 in Fig. 1. At this point, it is important to remark the difference with previous proposals, where the EP is applied just once. In [9, 11, 10, 12], the EP2 block is replaced by a projection into a Gaussian distribution, as described in (15). On the other hand, in the proposal in [13], named BP-EP, the block EP1 does not appear, i.e., BP-EP can be viewed as a particularization of the scheme in Fig. 1 where $S$ is set to 0, boiling down to the LMMSE for standalone equalization. Also, the control of negatives variances is different since BP-EP takes the absolute values for the negative variances.

IV Three different implementations

The computational complexity of Alg. 2 is dominated by the computation of the extrinsic distribution, $q_{E}^{[\ell]}(u_{k})$, at steps 4 and 6. This distribution has to be recomputed along a few EP iterations and turbo iterations, yielding a final complexity of ${\mathcal{O}}(K\varepsilon_{eq})$, where $K=(S+1)({T}+1)$ and $\varepsilon_{eq}$ denotes the cost of computing $q_{E}^{[\ell]}(u_{k})$ within the equalizer.

In its block implementation, the estimation of the transmitted symbols takes into account the whole vector of observations [9, 10] and the moments of $q_{E}^{[\ell]}(u_{k})$ are obtained with (6) and (7). The inversion in (7) can be solved with $\varepsilon_{eq}=\mathtt{L}\mathtt{N}^{2}$ if the banded-structure of the matrix is exploited [9]. Hence, the final complexity is ${\mathcal{O}}(K\mathtt{L}\mathtt{N}^{2})$.

To reduce the computational complexity of the block proposal, a KS approach is proposed in [11] and improved in [12]. It merges both forward and backward estimations into a smoothing one, emulating the BCJR behavior. This proposal exhibits the same performance as its block counterpart with linear complexity in the frame length, $\varepsilon_{eq}=\mathtt{N}{\mathtt{L}^{2}}$. Its final complexity is ${\mathcal{O}}(K\mathtt{N}{\mathtt{L}^{2}})$. The closed-form expression for $q_{E}^{[\ell]}(u_{k})$ to be used at step 4 and 6 of Alg. 2 is detailed in eqn. (3.40) of [14]. We will denote this proposal as double KS EP (D-KSEP). As discussed in Section III, if $S=0$ and step 3 is replaced by taking the absolute value of $\sigma_{t_{k}}^{2[1]}$, then it yields the BP-EP [13]. Its computational complexity is ${\mathcal{O}}({(T+1)}\mathtt{N}\mathtt{L}^{2})$.

Finally, a WF approach can be exploited to reduce the computational complexity to be quadratic in the length of a to-be-predefined window, $\mathtt{W}$, i.e., $\varepsilon_{eq}=\mathtt{N}\mathtt{W}^{2}$, yielding a complexity ${\mathcal{O}}(K\mathtt{N}\mathtt{W}^{2})$ [10]. However, its performance degrades in comparison to the block or KS designs since it just uses $\mathtt{W}$ observations [14]. In this implementation, eqn. (28) of [10] is used at steps 4 and 6 of Alg. 2 to compute $q_{E}^{[\ell]}(u_{k})$. We will denote this proposal as double filter EP (D-FEP).

The number of EP iterations, $S$, and of turbo iterations, ${T}$, must be set to speed up convergence while minimizing $K$. Convergence is also driven by the rest of EP parameters, $\beta$ and $\epsilon$. In [10], the EP parameters are optimized to $\epsilon=10^{-8}$, $\beta=\min(\exp^{t/1.5}/10,0.7)$, $S=3$ and ${T}=5$. For the double-EP algorithms proposed, we adopt these values for $\beta$ and $\epsilon$ while the proposed improvement in the estimation of the probabilities at the output of the channel decoder allows to reduce the number of inner iterations to $S=1$.

V Experimental results

In Fig. 2 we compare the BER of several equalizers after the channel decoder for (a) a $64$-QAM and (b) a $128$-QAM. The results are averaged over $100$ random channels and $10^{4}$ random encoded words of length ${V}=4096$ (per channel realization). Each channel tap is zero mean Gaussian independently distributed. The absolute value of LLRs given to the decoder is limited to $5$ to avoid very confident probabilities. A (3,6)-regular low-density parity-check (LDPC) of rate $1/2$ is used, for a maximum of $100$ iterations. First, we include in dash-dotted the LMMSE in its block implementation. Then we depict in dashed lines the BER of previous inner EP approaches with $S=3$ and ${T}=5$, i.e. $K=24$: the block EP (BEP) ($\circ$) [10], WF EP (FEP) ($\diamond$) [10] and KS EP (KSEP) ($\circ$) [12]. It has been checked that smaller values of $S$ and/or ${T}$ lead to a degradation of BER. Note that the BEP and KSEP are depicted with the same marker as they exhibit the same BER [12]. Also, we simulate the BP-EP proposal [13] until convergence, resulting $T=8$ ($K=9$) for 64-QAM and $T=11$ ($K=12$) for 128-QAM, in dotted lines. Finally, we include (in solid) the BER for the double-EP solutions proposed: D-BEP ($\circ$), D-FEP ($\diamond$) and D-KSEP ($\circ$). We use $S=1$, ${T}=5$, i.e. $K=12$. D-BEP and D-KSEP are depicted as one, since they have the same BER. These ones are also included (as $\square$) for ${T}=3$, i.e. $K=8$.

It can be observed that previous inner EP methods exhibit a remarkable 3-6 dB gain with respect to the block LMMSE. Compared with the BP-EP they have gains up to 3 dB with quite stable convergence. The instabilities of the BP-EP at large $E_{b}/N_{0}$ are the result of a poor control of negative variances and not performing a damping. However, the BP-EP has half or less of the computational complexity of the KSEP, with $K=24$.

The novel double EP equalizers achieve the same or better BER than their inner counterparts, reducing by half the number of iterations needed and hence the computational complexity. In Fig. 2.a the D-KSEP, with $K=8$, has a 0.5 dB gain compared with the BP-EP, with $K=9$, while we have a 1 dB gain if we further iterate the D-KSEP ($K=12$). For 128-QAM in Fig. 2.b, the D-KSEP has 2-3 dB gains compared with the BP-EP at the same number of iterations. Furthermore, the D-KSEP does not exhibit instabilities at high $E_{b}/N_{0}$.

The D-FEP improves the performance of the FEP but it is far from the one of other approaches. Also, it presents instabilities at large $E_{b}/N_{0}$ and 128-QAM. These drawbacks can be mitigated by using longer windows, i.e. a larger number of observations as input, at the cost of increasing its complexity.

VI Conclusions

In this letter we propose a new double EP-based equalizer where the EP algorithm is applied twice. First, it is used to improve the output of the equalizer, even in the case of no feedback from the channel decoder. Then, the EP is applied to the discrete outputs of the channel decoder, providing a more accurate initialization for the priors used by the turbo equalizer. This novel approach can be exploited in block, WF and KS implementations of the equalizer. The experimental results included show that the proposed equalizer improves or achieves the same performance of FEP, BEP and KSEP equalizers [10, 12] with half their computational complexity. It also outperforms the LMMSE, with just twice its complexity, and other EP-based solutions, such as the BP-EP [13].