Techniques for improving the finite length performance of sparse superposition codes

TL;DR

This paper explores techniques to enhance the finite length error performance of sparse superposition codes with AMP decoding, including power allocation, parameter tuning, and outer codes, demonstrating improvements over standard LDPC-coded modulation.

Contribution

It introduces novel methods for finite length optimization of sparse superposition codes, including an iterative power allocation algorithm and the use of partial outer codes.

Findings

Improved finite length error performance with proposed techniques.

Steep waterfall error curves achieved with outer codes.

Competitive performance compared to WiMAX LDPC codes.

Abstract

Sparse superposition codes are a recent class of codes introduced by Barron and Joseph for efficient communication over the AWGN channel. With an appropriate power allocation, these codes have been shown to be asymptotically capacity-achieving with computationally feasible decoding. However, a direct implementation of the capacity-achieving construction does not give good finite length error performance. In this paper, we consider sparse superposition codes with approximate message passing (AMP) decoding, and describe a variety of techniques to improve their finite length performance. These include an iterative algorithm for SPARC power allocation, guidelines for choosing codebook parameters, and estimating a critical decoding parameter online instead of pre-computation. We also show how partial outer codes can be used in conjunction with AMP decoding to obtain a steep waterfall in the error performance curves. We compare the error performance of AMP-decoded sparse superposition codes with coded modulation using LDPC codes from the WiMAX standard.

Figures

Equations

$$y=x+w,$$

$$R=\frac{\log (M^L)}{n}=\frac{L\log M}{n}.$$

$$z^t$$

$${\beta }_i^{t+1}$$

$${\eta }_i^t(s)=\sqrt{n{P}_{\ell }}\frac{\exp \left(s_i\frac{\sqrt{n{P}_{\ell }}}{{\tau }_t^2}\right)}{{\sum }_{j\in \sec (i)}\exp \left(s_j\frac{\sqrt{n{P}_{\ell }}}{{\tau }_t^2}\right)},1\le i\le ML.$$

$${\tau }_0^2={\sigma }^2+P,{\tau }_t^2={\sigma }^2+P(1-x({\tau }_{t-1})),t\ge 1,$$

$$\begin{array}{r}x(\tau ):=\sum _{\ell =1}^L\frac{P_{\ell }}{P}\mathbb{E}\left[\frac{e^{\frac{\sqrt{n{P}_{\ell }}}{\tau }(U_1^{\ell }+\frac{\sqrt{n{P}_{\ell }}}{\tau })}}{e^{\frac{\sqrt{n{P}_{\ell }}}{\tau }(U_1^{\ell }+\frac{\sqrt{n{P}_{\ell }}}{\tau })}+{\sum }_{j=2}^M{e}^{\frac{\sqrt{n{P}_{\ell }}}{\tau }{U}_j^{\ell }}}\right].\end{array}$$

$${\mathcal{E}}_{\text{sec}}=\frac{1}{L}\sum _{\ell =1}^{L}1\left\{{\hat{\beta }}_{\ell }\ne {\beta }_{\ell }\right\}$$

$$x(\tau )$$

$$x(\tau )\approx \sum _{\ell =1}^L\frac{P_{\ell }}{P}1\left\{L{P}_{\ell }>2R{\tau }^2\ln 2\right\}.$$

$$P_{\ell }=\frac{P(2^{2\mathcal{C}/L}-1)}{1-2^{-2\mathcal{C}}}{2}^{-2\mathcal{C}\ell /L},\ell \in [L],$$

$$P_{\ell }=\{\begin{array}{ll}\kappa \cdot 2^{-2a\mathcal{C}\ell /L}& 1\le \ell \le fL,\\ \kappa \cdot 2^{-2a\mathcal{C}f}& fL+1\le \ell \le L,\end{array}$$

$$P_{\ell }=\frac{2R{\tau }_0^2\ln 2}{L},1\le \ell \le \frac{L}{B}.$$

$${\mathcal{E}}_{\sec }^{\text{SE}}:=1-\frac{1}{L}\sum _{\ell =1}^L\mathbb{E}\left[\frac{e^{\frac{\sqrt{n{P}_{\ell }}}{{\tau }_T}\left(U_1^{\ell }+\frac{\sqrt{n{P}_{\ell }}}{{\tau }_T}\right)}}{e^{\frac{\sqrt{n{P}_{\ell }}}{{\tau }_T}\left(U_1^{\ell }+\frac{\sqrt{n{P}_{\ell }}}{{\tau }_T}\right)}+{\sum }_{j=2}^M{e}^{\frac{\sqrt{n{P}_{\ell }}}{{\tau }_T}{U}_j^{\ell }}}\right],$$

$$\begin{array}{rl}& \mathbb{P}\left({\mathcal{E}}_{\sec }>{\mathcal{E}}_{\sec }^{\text{SE}}+ϵ\right)\le K_T{e}^{\frac{-{\kappa }_{T}L}{(\log M{)}^{2T-1}}{\left(\frac{ϵ\ln (1+\text{snr})}{4(1+\text{snr})}-f(M)\right)}^2},\end{array}$$

$${\hat{\tau }}_t^2=\frac{\Vert z^t{\Vert }^2}{n}=\frac{1}{n}\sum _{i=1}^n{z}_i^2.$$

$$s^t:={\beta }^t+A^{\ast }{z}^t\approx \beta +{\tau }_{t}Z,$$

$${\bar{\mathcal{E}}}_{\sec }$$

$${\bar{\mathcal{E}}}_{\text{cw}}$$

$$\left\{\sqrt{n{P}_{\ell }}+\sigma Z_{\ell ,1}\le \sigma Z_{\ell ,2}\right\}\cup \dots \cup \left\{\sqrt{n{P}_{\ell }}+\sigma Z_{\ell ,1}\le \sigma Z_{\ell ,M}\right\}.$$

$$\begin{split}P_{\text{err},\ell}&=1-\mathbb{P}\left(\sqrt{nP_{\ell}}+\sigma Z_{\ell,1}>\sigma Z_{\ell,j},\ 2\leq j\leq M\right)\\ &=1-\int_{\mathbb{R}}\prod_{j=2}^{M}\mathbb{P}\left(Z_{\ell,j}<\frac{\sqrt{nP_{\ell}}}{\sigma}+u\,\Big{|}\,Z_{\ell,1}=u\right)\phi(u)du\\ &=1-\mathbb{E}_{U}\left[\Phi\left(\frac{\sqrt{nP_{\ell}}}{\sigma}+U\right)\right]^{M-1},\end{split}$$

$$L_{LDPC}=\frac{n_{LDPC}}{\log M}\text{and}{L}_{protected}=\frac{k_{LDPC}}{\log M},$$

$$L_{parity}=\frac{n_{LDPC}-k_{LDPC}}{\log M}=\frac{(5120-4096)}{8}=128,$$

$$L_{user}=L-L_{parity}=1024-128=896,$$

$$L_{protected}=\frac{k_{LDPC}}{\log M}=\frac{4096}{8}=512,$$

$$L_{LDPC}=L_{protected}+L_{parity}=512+128=640,$$

$$L_{unprotected}=L_{user}-L_{protected}=L-L_{LDPC}=384.$$

Full text

I Introduction

We consider communication over the memoryless additive white Gaussian noise (AWGN) channel given by

[TABLE]

where the channel output $y$ is the sum of the channel input $x$ and independent zero-mean Gaussian noise $w$ of variance $\sigma^{2}$. There is an average power constraint $P$ on the input, so a length-$n$ codeword $(x_{1},\ldots,x_{n})$ has to satisfy $\frac{1}{n}\sum_{i=1}^{n}x_{i}^{2}\leq P$. The goal is to build computationally efficient codes that have low probability of decoding error at rates close to the AWGN channel capacity $\mathcal{C}=\frac{1}{2}\log(1+\textsf{snr})$. Here snr denotes the signal-to-noise ratio $P/\sigma^{2}$.

Though it is well known that Shannon-style i.i.d. Gaussian codebooks can achieve very low probability of error at rates approaching the AWGN capacity [1], this approach has been largely avoided in practice due to the high decoding complexity of unstructured Gaussian codes. Current state of the art approaches for the AWGN channel such as coded modulation [2, 3] typically involve separate coding and modulation steps. In this approach, a binary error-correcting code such as an LDPC or turbo code is first used to generate a binary codeword from the information bits; the code bits are then modulated with a standard scheme such as quadrature amplitude modulation. Though these schemes have good empirical performance, they have not been proven to be capacity-achieving for the AWGN channel.

Sparse Superposition Codes or Sparse Regression Codes (SPARCs) were recently proposed by Barron and Joseph [4, 5] for efficient communication over the AWGN channel. In [5], they introduced an efficient decoding algorithm called “adaptive successive decoding” and showed that it achieved near-exponential decay of error probability (with growing block length), for any fixed rate $R<\mathcal{C}$. Subsequently, an adaptive soft-decision successive decoder was proposed in [6, 7], and Approximate Message Passing (AMP) decoders were proposed in [8, 9, 10, 11]. The adaptive soft-decision decoder in [7] as well as the AMP decoder in [11] were proven to be asymptotically capacity-achieving, and have superior finite length performance compared to the original adaptive successive decoder of [5].

The above results mainly focused on characterizing the error performance of SPARCs in the limit of large block length. In this work, we describe a number of code design techniques for improved finite length error performance. Throughout the paper, we focus on AMP decoding due to its ease of implementation. However, many of the code design ideas can also be applied to the adaptive soft-decision successive decoder in [6, 7]. A hardware implementation of the AMP decoder was recently reported in [12, 13]. We expect that the techniques proposed in this paper can be used to reduce the complexity and optimize the decoding performance in such implementations.

In the remainder of this section, we briefly review the SPARC construction and the AMP decoder from [11], and then list the main contributions of this paper. A word about notation before we proceed. Throughout the paper, we use $\log$ to denote logarithms with base $2$, and $\ln$ to denote natural logarithms. For a positive integer $N$, we use $[N]$ to denote the set $\{1,\ldots,N\}$. The transpose of a matrix $A$ is denoted by $A^{*}$, and the indicator function of an event $\mathcal{E}$ by $\mathbf{1}\{\mathcal{E}\}$.

I-A The sparse superposition code

A SPARC is defined in terms of a design matrix $A$ of dimension $n\times ML$. Here $n$ is the block length, and $M,L$ are integers which are specified below in terms of $n$ and the rate $R$. As shown in Fig. 1, the design matrix $A$ has $L$ sections with $M$ columns each. In the original construction of [4, 5] and in the theoretical analysis in [6, 7, 11, 14], the entries of $A$ are assumed to be i.i.d. Gaussian $\sim\mathcal{N}(0,1/n)$. For our empirical results, we use a random Hadamard-based construction for $A$ that leads to significantly lower encoding and decoding complexity [9, 10, 11].

Codewords are constructed as sparse linear combinations of the columns of $A$. In particular, a codeword is of the form $A\beta$, where $\beta=(\beta_{1},\ldots,\beta_{ML})^{*}$ is a length $ML$ column vector with the property that there is exactly one non-zero $\beta_{j}$ for the section $1\leq j\leq M$, one non-zero $\beta_{j}$ for the section $M+1\leq j\leq 2M$, and so forth. The non-zero value of $\beta$ in each section $\ell$ is set to $\sqrt{nP_{\ell}}$, where $P_{1},\ldots,P_{L}$ are pre-specified positive constants that satisfy $\sum_{\ell=1}^{L}P_{\ell}=P$, the average symbol power allowed.

Both $A$ and the power allocation $\{P_{1},\ldots,P_{L}\}$ are known to both the encoder and decoder in advance. The choice of power allocation plays a crucial role in determining the error performance of the decoder. Without loss of generality, we will assume that the power allocation is non-increasing across sections. Two examples of power allocation are:

• •

Flat power allocation, where $P_{\ell}=\frac{P}{L}$ for all $\ell$. This choice was used in [4] to analyze the error performance with optimal (least-squares) decoding.

• •

Exponentially decaying power allocation, where $P_{\ell}\propto 2^{-{2\mathcal{C}\ell/L}}$. This choice was used for the asymptotically capacity-achieving decoders proposed in [5, 7, 11].

At finite block lengths both these power allocations could be far from optimal and lead to poor decoding performance. One of the main contributions of this paper is an algorithm to determine a good power allocation for the finite-length AMP decoder based only on $R$, $P$, $\sigma^{2}$.

Rate: As each of the $L$ sections contains $M$ columns, the total number of codewords is $M^{L}$. With the block length being $n$, the rate of the code is given by

[TABLE]

In other words, a SPARC codeword corresponding to $L\log M$ input bits is transmitted in $n$ channel uses.

Encoding: The input bitstream is split into chunks of $\log M$ bits. A chunk of $\log M$ input bits can be used to index the location of the non-zero entry in one section of $\beta$. Hence $L$ successive chunks determine the message vector $\beta$, with the $\ell$th chunk of $\log M$ input bits determining the non-zero location in section $\ell$, for $1\leq\ell\leq L$.

Approximate Message Passing (AMP) decoder: The AMP decoder produces iteratively refined estimates of the message vector, denoted by $\beta^{1},\beta^{2},\ldots,\beta^{T}$, where $T$ is the (pre-specified) number of iterations. Starting with $\beta^{0}=0$, for $t=0,1,\ldots,T-1$ the AMP decoder generates

[TABLE]

where

[TABLE]

Here the notation $j\in\sec(i)$ refers to all indices $j$ in the same section as $i$. (Note that there are $M$ indices in each section.) At the end of each step $t$, $\beta_{i}^{t}/\sqrt{nP_{\ell}}$ may be interpreted as the updated posterior probability of the $i$th entry being the non-zero one in its section.

The constants $\tau_{t}^{2}$ are specified by the following scalar recursion called “state evolution” (SE):

[TABLE]

where

[TABLE]

In (6), $\{U^{\ell}_{j}\}$ are i.i.d. $\mathcal{N}(0,1)$ random variables for $j\in[M],\ \ell\in[L]$. The significance of the SE parameters $\tau_{t}^{2}$ is discussed in Section II. In Section IV, we use an online approach to accurately compute the $\tau_{t}^{2}$ values rather than pre-computing them via (6).

At the end of $T$ iterations, the decoded message vector $\widehat{\beta}$ is produced by setting the maximum value in section $\ell$ of $\beta^{T}$ to $\sqrt{nP_{\ell}}$ and the remaining entries to zero, for $1\leq\ell\leq L$.

Error rate of the AMP decoder: We measure the section error rate $\mathcal{E}_{\text{sec}}$ as

[TABLE]

Assuming a uniform mapping between the input bitstream and the non-zero locations in each section, each section error will cause approximately half of the bits it represents to be incorrect, leading to a bit error rate $\mathcal{E}_{\text{ber}}\approx\frac{1}{2}\mathcal{E}_{\text{sec}}$.

Another figure of merit is the codeword error rate $\mathcal{E}_{\text{cw}}$, which estimates the probability $\mathbb{P}(\widehat{\beta}\neq\beta)$. If the SPARC is used to transmit a large number of messages (each via a length $n$ codeword), $\mathcal{E}_{\text{cw}}$ measures the fraction of codewords that are decoded with one or more section errors. The codeword error rate is insensitive to where and how many section errors occur within a codeword when it is decoded incorrectly.

At finite code lengths, the choice of a good power allocation crucially depends on whether we want to minimize $\mathcal{E}_{\text{sec}}$ or $\mathcal{E}_{\text{cw}}$. As we will see in the next section, a power allocation that yields reliably low section error rates may result in a high codeword error rate, and vice versa. In this paper, we will mostly focus on obtaining the best possible section error rate, since in practical applications a high-rate outer code could readily correct a small fraction of section errors to give excellent codeword error rates as well. Further, the bit error rate (which is approximately half the section error rate) is useful to compare with other channel coding approaches, where it is a common figure of merit.

I-B Organization of the paper and main contributions

In the rest of the paper, we describe several techniques to improve the finite length error performance and reduce the complexity of AMP decoding. The sections are organized as follows.

• •

In Section II, we introduce an iterative power allocation algorithm that gives improved error performance with fewer tuning parameters than other power allocation schemes.

• •

In Section III, we analyze the effects of the code parameters $L,M$ and the power allocation on error performance and its concentration around the value predicted by state evolution.

• •

In Section IV, we describe how an online estimate of the key SE parameter $\tau_{t}^{2}$ improves error performance and allows a new early-stopping criterion. Furthermore, the online estimate enables us to accurately estimate the actual section error rate at the end of the decoding process.

• •

In Section V, we derive simple expressions to estimate $\mathcal{E}_{\text{sec}}$ and $\mathcal{E}_{\text{cw}}$ given the rate and power allocation.

• •

In Section VI we compare the error performance of AMP-decoded SPARCs to LDPC-based coded modulation schemes used in the WiMAX standard.

• •

In Section VII, we describe how partial outer codes can be used in conjunction with AMP decoding. We propose a three-stage decoder consisting of AMP decoding, followed by outer code decoding, and finally, AMP decoding once again. We show that by covering only a fraction of sections of the message $\beta$ with an outer code, the three-stage decoder can correct errors even in the sections not covered by the outer code. This results in bit-error curves with a steep waterfall behavior.

The main technical contributions of the paper are the iterative power allocation algorithm (Section II) and the three-stage decoder with an outer code (Section VII). The other sections describe how various choices of code parameters influence the finite length error performance, depending on whether the objective is to minimize the section error rate or the codeword error rate. We remark that the focus in this paper is on improving the finite length performance using the standard SPARC construction with power allocation. Optimizing the finite length performance of spatially-coupled SPARCs considered in [10, 9] is an interesting research direction, but one that is beyond the scope of this paper.

II Power Allocation

Before introducing the power allocation scheme, we briefly give some intuition about the AMP update rules (2)–(4), and the SE recursion in (5)–(6). The update step (3) to generate each estimate of $\beta$ is underpinned by the following key property: after step $t$, the “effective observation” $\beta^{t}+A^{*}z^{t}$ is approximately distributed as $\beta+\tau_{t}Z$, where $Z$ is standard normal random vector independent of $\beta$. Thus $\tau_{t}^{2}$ is the effective noise variance at the end of step $t$. Assuming that the above distributional property holds, $\beta^{t+1}$ is just the Bayes-optimal estimate of $\beta$ based on the effective observation. The entry $\beta^{t+1}_{i}$ is proportional to the posterior probability of the $i$th entry being the non-zero entry in its section.

We see from (5) that the effective noise variance $\tau_{t}^{2}$ is the sum of two terms. The first is the channel noise variance $\sigma^{2}$. The other term $P(1-x(\tau_{t-1}))$ can be interpreted as the interference due to the undecoded sections in $\beta^{t}$. Equivalently, $x(\tau_{t-1})$ is the expected power-weighted fraction of sections which are correctly decodable at the end of step $t$.

The starting point for our power allocation design is the following result from [11], which gives analytic upper and lower bounds for $x(\tau)$ of (5).

Lemma 1**.**

[14, Lemma 1(b)]** Let $\nu_{\ell}:=\frac{LP_{\ell}}{R\tau^{2}\ln 2}$. For sufficiently large $M$, and for any $\delta\in(0,1)$,

[TABLE]

where $\kappa_{1},\kappa_{2}$ are universal positive constants.

As the constants $\kappa_{1},\kappa_{2}$ in (8)–(9) are not precisely specified, for designing power allocation schemes, we use the following approximation for $x(\tau)$:

[TABLE]

This approximate version, which is increasingly accurate as $L,M$ grow large, is useful for gaining intuition about suitable power allocations. Indeed, if the effective noise variance after step $t$ is $\tau_{t}^{2}$, then (10) says that any section $\ell$ whose normalized power $LP_{\ell}$ is larger than the threshold $2R\tau^{2}_{t}\ln 2$ is likely to be decodable correctly in step $(t+1)$, i.e., in $\beta^{t+1}$, the probability mass within the section will be concentrated on the correct non-zero entry. For a given power allocation, we can iteratively estimate the SE parameters $(\tau_{t}^{2},x(\tau_{t}^{2}))$ for each $t$ using the lower bound in (10). This provides a way to quickly check whether or not a given power allocation will lead to reliable decoding in the large system limit. For reliable decoding at a given rate $R$, the effective noise variance given by $\tau_{t}^{2}=\sigma^{2}+P(1-x(\tau_{t-1}))$ should decrease with $t$ until it reaches a value close to $\sigma^{2}$ in a finite number of iterations. Equivalently, $x(\tau_{t})$ in (6) should increase to a value very close to $1$.

For a rate $R<\mathcal{C}$, there are infinitely many power allocations for which (10) predicts successful decoding in the large system limit. However, as illustrated below, their finite length error performance may differ significantly. Thus the key question addressed in this section is: how do we choose a power allocation that gives the lowest section error rate?

The exponentially-decaying power allocation given by

[TABLE]

was proven in [11] to be capacity-achieving in the large system limit, i.e., it was shown that the section error rate $\mathcal{E}_{\text{sec}}$ of the AMP decoder converges almost surely to [math] as $n\to\infty$, for any $R<\mathcal{C}$. However, it does not perform well at practical block lengths, which motivated the search for alternatives. We now evaluate it in the context of (10) to better explain the development of a new power allocation scheme.

Given a power allocation, using (10) one can compute the minimum required power for any section $\ell\in[L]$ to decode, assuming that the sections with higher power have decoded correctly. The dashed lines in Figure 2 shows the minimum power required for each section to decode (assuming the exponential allocation of (11) for the previous sections), for $R=\mathcal{C}$ and $R=0.7\mathcal{C}$. The figure shows that the power allocation in (11) matches (up to order $\frac{1}{L}$ terms) with the minimum required power when $R=\mathcal{C}$. However, for $R=0.7\mathcal{C}$, we see that the exponentially-decaying allocation allocates significantly more power to the earlier sections than the minimum required, compared to later sections. This leads to relatively high section error rates, as shown in Figure 6.

Figure 2 shows that the total power allocated by the minimal power allocation at $R=0.7\mathcal{C}$ is significantly less than the available power $P$. Therefore, the key question is: how do we balance the allocation of available power between the various sections to minimize the section error rate? Allocating excessive power to the earlier sections ensures they decode reliably early on, but then there will not be sufficient power left to ensure reliable decoding in the final sections. This is the reason for the poor finite length performance of the exponentially-decaying allocation. Conversely, if the power is spread too evenly then no section particularly stands out against the noise, so it is hard for the decoding to get started, and early errors can cause cascading failures as subsequent sections are also decoded in error.

This trade-off motivated the following modified exponential power allocation proposed in [11]:

[TABLE]

where the normalizing constant $\kappa$ is chosen to ensure that $\sum_{\ell=1}^{L}P_{\ell}=P$. In (12), the parameter $a$ controls the steepness of the exponential allocation, while the parameter $f$ flattens the allocation after the first fraction $f$ of the sections. Smaller choices of $a$ lead to less power allocated to the initial sections, making a larger amount available to the later sections. Similarly, smaller values of $f$ lead to more power allocated to the final sections. See Figure 3 for an illustration.

While this allocation improves the section error rate by a few orders of magnitude (see [11, Fig. 4]), it requires costly numerical optimization of $a$ and $f$. A good starting point is to use $a=f=R/\mathcal{C}$, but further optimization is generally necessary. This motivates the need for a fast power allocation algorithm with fewer tuning parameters.

II-A Iterative power allocation

We now describe a simple parameter-free iterative algorithm to design a power allocation. The $L$ sections of the SPARC are divided into $B$ blocks of $L/B$ sections each. Each section within a block is allocated the same power. For example, with $L=512$ and $B=32$, there are $32$ blocks with $16$ sections per block. The algorithm sequentially allocates power to each of the $B$ blocks as follows. Allocate the minimum power to the first block of sections so that they can be decoded in the first iteration when $\tau_{0}^{2}=\sigma^{2}+P$. Using (10), we set the power in each section of the first block to

[TABLE]

Using (10) and (5), we then estimate $\tau_{1}^{2}=\sigma^{2}+(P-BP_{1})$. Using this value, allocate the minimum required power for the second block of sections to decode, i.e., $P_{\ell}=2\ln 2R\tau_{1}^{2}/L$ for $\frac{L}{B}+1\leq\ell\leq\frac{2L}{B}$. If we sequentially allocate power in this manner to each of the $B$ blocks, then the total power allocated by this scheme will be strictly less than $P$ whenever $R<\mathcal{C}$. We therefore modify the scheme as follows.

For $1\leq b\leq B$, to allocate power to the $b$th block of sections assuming that the first $(b-1)$ blocks have been allocated, we compare the two options and choose the one that allocates higher power to the block: i) allocating the minimum required power (computed as above) for the $b$th block of sections to decode; ii) allocating the remaining available power equally to sections in blocks $b,\ldots,B$, and terminating the algorithm. This gives a flattening in the final blocks similar to the allocation in (12), but without requiring a specific parameter that determines where the flattening begins. The iterative power allocation routine is described in Algorithm 1. Figure 4 shows a toy example building up the power allocation for $B=5$, where flattening is seen to occur in step 4. Figure 5 shows a more realistic example with $L=512$ and $R=0.7\mathcal{C}$.

Choosing $B$: By construction, the iterative power allocation scheme specifies the number of iterations of the AMP decoder in the large system limit. This is given by the number of blocks with distinct powers; in particular the number of iterations (in the large system limit) is of the order of $B$. For finite code lengths, we find that it is better to use a termination criterion for the decoder based on the estimates generated by the algorithm. This criterion is described in Sec. IV. This data-driven termination criterion allows us to choose the number of blocks $B$ to be as large as $L$. We found that choosing $B=L$, together with the termination criterion in Sec. IV, consistently gives a small improvement in error performance (compared to other choices of $B$), with no additional time or memory cost.

Additionally, with $B=L$, it is possible to quickly determine a pair $(a,f)$ for the modified exponential allocation in (12) which gives a nearly identical allocation to the iterative algorithm. This is done by first setting $f$ to obtain the same flattening point found in the iterative allocation, and then searching for an $a$ which matches the first allocation coefficient $P_{1}$ between the iterative and the modified exponential allocations. Consequently, any simulation results obtained for the iterative power allocation could also be obtained using a suitable $(a,f)$ with the modified exponential allocation, without having to first perform a costly numerical optimization over $(a,f)$.

Figure 6 compares the error performance of the exponential and iterative power allocation schemes discussed above for different values of $R$ at $\textsf{snr}=7,15,31$. The iterative power allocation yields significantly improved $\mathcal{E}_{\text{sec}}$ for rates away from capacity when compared to the original exponential allocation, and additionally outperforms the modified exponential allocation results reported in [11].

For the experiments in Figure 6, the value for $R$ used in constructing the iterative allocation (denoted by $R_{PA}$) was optimized numerically. Constructing an iterative allocation with $R=R_{PA}$ yields good results, but due to finite length concentration effects, the $R_{PA}$ yielding the smallest average error rate may be slightly different from the communication rate $R$. The effect of $R_{PA}$ on the concentration of error rates is discussed in Section III-B. We emphasize that this optimization over $R_{PA}$ is simpler than numerically optimizing the pair $(a,f)$ for the modified exponential allocation. Furthermore, guidelines for choosing $R_{PA}$ as a function of $R$ are given in Section III-B.

III Error Concentration Trade-offs

In this section, we discuss how the choice of SPARC design parameters can influence the trade-off between the ‘typical’ value of section error rate and concentration of actual error rates around the typical values. The typical section error rate refers to that predicted by state evolution (SE). Indeed, running the SE equations (5)–(6) until convergence gives the following prediction for the section error rate:

[TABLE]

where $\tau_{T}^{2}$ denotes the value in the final iteration. The concentration refers to how close the SE prediction $\mathcal{E}^{\text{SE}}_{\sec}$ is to the observed section error rate.

As we describe below, the choice of SPARC parameters $(L,M)$ and the power allocation both determine a trade-off between obtaining a low value for $\mathcal{E}^{\text{SE}}_{\sec}$, and concentration of the actual section error rate around $\mathcal{E}^{\text{SE}}_{\sec}$. This trade-off is of particular interest when applying an outer code to the SPARC, as considered in Section VII, which may be able to reliably handle only a small number of section errors.

III-A Effect of $L$ and $M$ on concentration

Recall from (1) that the code length $n$ at a given rate $R$ is determined by the choice of $L$ and $M$ according to the relationship $nR=L\log M$. In general, $L$ and $M$ may be chosen freely to meet a desired rate and code length.

To understand the effect of increasing $M$, consider Figure 7 which shows the error performance of a SPARC with $R=1.5,L=1024$, as we increase the value of $M$. From (1), the code length $n$ increases logarithmically with $M$. We observe that the section error rate (averaged over $200$ trials) decreases with $M$ up to $M=2^{9}$, and then starts increasing. This is in sharp contrast to the SE prediction (13) (plotted using a dashed line in Figure 7) which keeps decreasing as $M$ is increased.

This divergence between the actual section error rate and the SE prediction for large $M$ is due to large fluctuations in the number of section errors across trials. Recent work on the error exponent of SPARCs with AMP decoding shows that the concentration of error rates near the SE prediction is strongly dependent on both $L$ and $M$. For $R<\mathcal{C}$, [14, Theorem 1] shows that for any $\epsilon>0$, the section error rate $\mathcal{E}_{\text{sec}}$ satisfies

[TABLE]

where $T$ is the number of iterations until state evolution convergence, $\kappa_{T},K_{T}$ are constants depending on $T$, and $f(M)=\frac{M^{-\kappa_{2}\delta^{2}}}{\delta\sqrt{\ln M}}$ is a quantity that tends to zero with growing $M$. For any power allocation, $T$ increases as $R$ approaches $C$. For example, $T\propto 1/\log(\mathcal{C}/R)$ for the exponential power allocation. We observe that the deviation probability bound on the RHS of (14) depends on the ratio $L/(\log M)^{2T-1}$.

In our experiments, $T$ is generally on the order of a few tens. Therefore, keeping $L$ constant, the probability of large deviations from the SE prediction $\mathcal{E}^{\text{SE}}_{\sec}$ increases with $M$. This leads to the situation shown in Figure 7, which shows that the SE prediction $\mathcal{E}^{\text{SE}}_{\sec}$ continues to decrease with $M$, but beyond a certain value of $M$, the observed average section error rate becomes progressively worse due to loss of concentration. This is caused by a small number of trials with a very large number of section errors, even as the majority of trials experience lower and lower error rates as $M$ is increased. This effect can be clearly seen in Figure 8, which compares the histogram of section error rates over $200$ trials for $M=64$ and $M=4096$. The distribution of errors is clearly different, but both cases have the same average section error rate due to the poorer concentration for $M=4096$.

To summarize, given $R,\textsf{snr}$, and $L$, there is an optimal $M$ that minimizes the empirical section error rate. Beyond this value of $M$, the benefit from any further increase is outweighed by the loss of concentration. For a given $R$, values of $M$ close to $L$ are a good starting point for optimizing the empirical section error rate, but obtaining closed-form estimates of the optimal $M$ for a given $L$ is still an open question.

For fixed $L,R$, the optimal value of $M$ increases with snr. This effect can be seen in the results of Figure 12, where there is an inversion in the order of best-performing $M$ values as $E_{b}/N_{0}$ increases. This is because as snr increases, the number of iterations $T$ for SE to converge decreases. A smaller $T$ mitigates the effect of larger $M$ in the large deviations bound of (14). In other words, a larger snr leads to better error rate concentration around the SE prediction, so larger values of $M$ are permissible before the performance starts degrading.

III-B Effect of power allocation on concentration

The non-asymptotic bounds on $x(\tau)$ in Lemma 1 indicate that at finite lengths, the minimum power required for a section $\ell$ to decode in an iteration may be slightly different than that indicated by the approximation in (10). Recall that the iterative power allocation algorithm in Section II-A was designed based on (10). We can compensate for the difference between the approximation and the actual value of $x(\tau)$ by running the iterative power allocation in Algorithm 1 using a modified rate $R_{\text{PA}}$ which may be slightly different from the communication rate $R$. The choice of $R_{\text{PA}}$ directly affects the error concentration. We now discuss the mechanism for this effect and give guidelines for choosing $R_{\text{PA}}$ as a function of $R$.

If we run the power allocation algorithm with $R_{\text{PA}}>R$, from (10) we see that additional power is allocated to the initial blocks, at the cost of less power for the final blocks (where the allocation is flat). Consequently, it is less likely that one of the initial sections will decode in error, but more likely that some number of the later sections will instead. Figure 9 (bottom) shows the effect of choosing a large $R_{\text{PA}}=1.06R$: out of a total of $1000$ trials, there were no trials with more than 7 sections decoded in error (the number of sections $L=1024$); however, relatively few trials ($29\%$) have zero section errors.

Conversely, choosing $R_{\text{PA}}<R$ allocates less power to the initial blocks, and increases the power in the final sections which have a flat allocation. This increases the likelihood of the initial section being decoded in error; in a trial when this happens, there will be a large number of section errors. However, if the initial sections are decoded correctly, the additional power in the final sections increases the probability of the trial being completely error-free. Thus choosing $R_{PA}<R$ makes completely error-free trials more likely, but also increases the likelihood of having trials with a large number of sections in error. In Figure 9 (top), the smaller $R_{\text{PA}}=0.98R$ gives zero or one section errors in the majority ($81\%$) of cases, but the remaining trials typically have a large number of sections in error.

To summarize, the larger the $R_{PA}$, the better the concentration of section error rates of individual trials around the overall average. However, increasing $R_{PA}$ beyond a point just increases the average section error rate because of too little power being allocated to the final sections.

For different values of the communication rate $R$, we empirically determined an $R_{\text{PA}}$ that gives the lowest average section error rate, by starting at $R_{\text{PA}}=R$ and searching the neighborhood in steps of $0.02R$. Exceptionally, at low rates (for $R\leq 1$), the optimal $R_{\text{PA}}$ is found to be [math], leading to a completely flat power allocation with $P_{\ell}=\frac{P}{L}$ for all $\ell$. We note from (10) that for $1\geq R>\frac{P}{2\tau_{0}^{2}\ln 2}$, the large system limit theory does not predict that we can decode any of the $L$ sections — this is because no section is above the threshold in the first iteration of decoding. However, in practice, we observe that some sections will decode initially (due to the correct column being aligned favorably with the noise vector), and this reduces the threshold enough to allow subsequent decoding to continue in most cases. For $R\leq 1$, when $R_{\text{PA}}$ closer to $R$ is used, the lower power in later sections hinders the finite length decoding performance.

We found that the value of $\frac{R_{PA}}{R}$ that minimizes the average section error rate increases with $R$. In particular, the optimal $\frac{R_{PA}}{R}$ was [math] for $R\leq 1$; the optimal $\frac{R_{PA}}{R}$ for $R=1.5$ was close to 1, and for $R=2$, the optimal $\frac{R_{PA}}{R}$ was between $1.05$ and $1.1$. Though this provides a useful design guideline, a deeper theoretical analysis of the role of $R_{PA}$ in optimizing the finite length performance is an open question.

Finally, a word of caution when empirically optimizing $R_{\text{PA}}$ to minimize the average section error rate. Due to the loss of concentration as $R_{PA}$ is decreased below $R$, care must be taken to run sufficient trials to ensure that a rare unseen trial with many section errors will not catastrophically impact the overall average section error rate. For example, in one scenario with $L=1024,M=512,\textsf{snr}=15,R=1.4,R_{\text{PA}}=1.316$, we observed 192 trials with errors out of 407756 trials, but only 4 of these trials had more than one error, with between 400 to 600 section errors in those 4 cases. The average section error rate was $5.6\text{\times}{10}^{-6}$. With fewer trials, it is possible that no trials with a large number of section errors would be observed, leading to an estimated error rate an order of magnitude better, at around $4.6\text{\times}{10}^{-7}$.

IV Online Computation of $\tau_{t}^{2}$ and Early Termination

Recall that the update step (4) of the AMP decoder requires the SE coefficients $\tau_{t}^{2}$, for $t\in[T]$. In the standard implementation [11], these coefficients are computed in advance using the SE equations (5)–(6). The total number of iterations $T$ is also determined in advance by computing the number of iterations required the SE to converge to its fixed point (to within a specified tolerance). This technique produced effective results, but advance computation is slow as each of the $L$ expectations in (6) needs to be computed numerically via Monte-Carlo simulation, for each $t$. A faster approach is to compute the $\tau_{t}^{2}$ coefficients using the asymptotic expression for $x(\tau)$ given in (10). This gives error performance nearly identical to the earlier approach with significant time savings, but still requires advance computation. Both these methods are referred to as “offline” as the $\tau_{t}^{2}$ values are computed a priori.

A simple way to estimate $\tau_{t}^{2}$ online during the decoding process is as follows. In each step $t$, after producing $z^{t}$ as in (2), we estimate

[TABLE]

The justification for this estimate comes from the analysis of the AMP decoder in [11, 14], which shows that for large $n$, $\widehat{\tau}_{t}^{2}$ is close to $\tau_{t}^{2}$ in (5) with high probability. In particular, [14] provides a concentration inequality for $\widehat{\tau}_{t}^{2}$ similar to (14). We note that such a similar online estimate has been used previously in various AMP and GAMP algorithms [8, 9, 10, 15]. Here, we show that in addition to being fast, the online estimator permits an interpretation as a measure of SPARC decoding progress and provides a flexible termination criterion for the decoder. Furthermore, the error performance with the online estimator was observed to be the same or slightly better than the offline methods.

Recall from the discussion at the beginning of Section II that in each step, we have

[TABLE]

where $Z$ is a standard normal random vector independent of $\beta$. Starting from $\tau_{0}^{2}=\sigma^{2}+P$, a judicious choice of power allocation ensures that the SE parameter $\tau_{t}^{2}$ decreases with $t$, until it converges at $\tau_{T}^{2}=\sigma^{2}$ in a finite number of iterations $T$.

However, at finite lengths there are deviations from this trajectory of $\tau_{t}^{2}$ predicted by SE, i.e., the variance of the effective noise vector $(s^{t}-\beta)$ may deviate from $\tau_{t}^{2}$. The online estimator $\widehat{\tau}_{t}^{2}$ is found to track $\mathrm{Var}(s^{t}-\beta)=\lVert s^{t}-\beta\rVert^{2}/n$ very accurately, even when this variance deviates significantly from $\tau_{t}^{2}$. This effect can be seen in Figure 10, where 16 independent decoder runs are plotted and compared with the SE trajectory for $\tau_{t}^{2}$ (dashed line). For the 15 successful runs, the empirical variance $\mathrm{Var}(s^{t}-\beta)$ approaches $\sigma^{2}=1$ along different trajectories depending on how the decoding is progressing. In the unsuccessful run, $\mathrm{Var}(s^{t}-\beta)$ converges to a value much larger than $\sigma^{2}$.

In all the runs, $\widehat{\tau}^{2}_{t}$ is indistinguishable from $\mathrm{Var}(s^{t}-\beta)$. This indicates that we can use the final value $\widehat{\tau}_{T}^{2}$ to accurately estimate the power of the undecoded sections — and thus the number of sections decoded correctly — at runtime. Indeed, $(\widehat{\tau}^{2}_{T}-\sigma^{2})$ is an accurate estimate of the total power in the incorrectly decoded sections. This, combined with the fact that the power allocation is non-increasing, allows the decoder to estimate the number of incorrectly decoded sections.

Furthermore, we can use the change in $\widehat{\tau}_{t}^{2}$ between iterations to terminate the decoder early. If the value $\widehat{\tau}_{t}^{2}$ has not changed between successive iterations, or the change is within some small threshold, then the decoder has stalled and no further iterations are worthwhile. Empirically we find that a stopping criterion with a small threshold (e.g., stop when $\lvert\widehat{\tau}^{2}_{t}-\widehat{\tau}^{2}_{t-1}\rvert<P_{L}$) leads to no additional errors compared to running the decoder for the full iteration count, while giving a significant speedup in most trials. Allowing a larger threshold for the stopping criterion gives even better running time improvements. This early termination criterion based on $\widehat{\tau}_{t}^{2}$ gives us flexibility in choosing the number of blocks $B$ in the iterative power allocation algorithm of Section II-A. This is because the number of AMP iterations is no longer tied to $B$, hence $B$ can be chosen as large as desired.

To summarize, the online estimator $\widehat{\tau}_{t}^{2}$ provides an estimate of the noise variance in each AMP iteration that accurately reflects how the decoding is progressing in that trial. It thereby enables the decoder to effectively adapt to deviations from the $\tau_{t}^{2}$ values predicted by SE. This explains the improved performance compared to the offline methods of computing $\tau_{t}^{2}$. More importantly, it provides an early termination criterion for the AMP decoder as well as a way to track decoding progress and predict the number of section errors at runtime.

V Predicting $\mathcal{E}_{\text{sec}}$, $\mathcal{E}_{\text{ber}}$ and $\mathcal{E}_{\text{cw}}$

For a given power allocation $\{P_{\ell}\}$ and reasonably large SPARC parameters $(n,M,L)$, it is desirable to have a quick way to estimate the section error rate and codeword error rate, without resorting to simulations. Without loss of generality, we assume that the power allocation is asymptotically good, i.e., the large system limit SE parameters (computed using (10)) predict reliable decoding, i.e., the SE converges to $x_{T}=1$ and $\tau_{T}^{2}=\sigma^{2}$ in the large system limit. The goal is to estimate the finite length section error rate $\mathcal{E}_{\text{sec}}$.

One way to estimate $\mathcal{E}_{\text{sec}}$ is via the state evolution prediction (13), using $\tau_{T}=\sigma$. However, computing (13) requires computing $L$ expectations, each involving a function of $M$ independent standard normal random variables. The following result provides estimates of $\mathcal{E}_{\text{sec}}$ and $\mathcal{E}_{\text{cw}}$ that are as accurate as the SE-based estimates, but much simpler to compute.

Proposition 1**.**

Let the power allocation $\{P_{\ell}\}$ be such that the state evolution iteration using the asymptotic approximation (10) converges to $\tau_{T}^{2}=\sigma^{2}$. Then, under the idealized assumption that $\beta^{T}+A^{*}z^{T}=\beta+\tau_{T}Z$ (where $Z$ is a standard normal random vector independent of $\beta$), we have the following. The probability of a section (chosen uniformly at random) being incorrectly decoded is

[TABLE]

The probability of the codeword being incorrectly decoded is

[TABLE]

In both expressions above, $U$ is a standard normal random variable, and $\Phi(.)$ is the standard normal cumulative distribution function.

Proof:

As $\tau_{T}^{2}=\sigma^{2}$, the effective observation in the final iteration has the representation $\beta+\sigma Z$. The denoising function $\eta^{T}$ generates a final estimate based on this effective observation, and the index of the largest entry in each section is chosen to form the decoded message vector $\widehat{\beta}$. Consider the decoding of section $\ell$ of $\beta$. Without loss of generality, we can assume that the first entry of the section is the non-zero one. Using the notation $\beta_{\ell,j}$ to denote the $j$th entry of the section $\beta_{\ell}$, we therefore have $\beta_{\ell,1}=\sqrt{nP_{\ell}}$, and $\beta_{\ell,j}=0$ for $2\leq j\leq M$. As the effective observation for section $\ell$ has the representation $(\beta^{T}+A^{*}z^{T})_{\ell}=\beta_{\ell}+\sigma Z_{\ell}$, the section will be incorrectly decoded if and only if the following event occurs:

[TABLE]

Therefore, the probability that the $\ell$th section is decoded in error can be computed as

[TABLE]

where $\phi$ and $\Phi$ denote the density and the cumulative distribution function of the standard normal distribution, respectively. In the second line of (19), we condition on $Z_{\ell,1}$ and then use the fact that $Z_{\ell,1},\ldots,Z_{\ell,M}$ are i.i.d. $\sim\mathcal{N}(0,1)$.

The probability of a section chosen uniformly at random being incorrectly decoded is $\frac{1}{L}\sum_{\ell=1}^{L}P_{\text{err},\ell}$. The probability of codeword error is one minus the probability that no section is in error, which is given by $1-\prod_{\ell=1}^{L}(1-P_{\text{err},\ell})$. Substituting for $P_{\text{err},\ell}$ from (19) yields the expressions in (17) and (18).

∎

The section error rate and codeword error rate can be estimated using the idealized expressions in (17) and (18). This still requires computing $L$ expectations, but each expectation is now a function of a single Gaussian random variable, rather than the $M$ independent ones in the SE estimate. Thus we reduce the complexity by a factor of $M$ over the SE approach; evaluations of $\bar{\mathcal{E}}_{\sec}$ and $\bar{\mathcal{E}}_{\text{cw}}$ typically complete within a second.

Figure 7 shows $\bar{\mathcal{E}}_{\sec}$ alongside the SE estimate $\mathcal{E}^{\text{SE}}_{\sec}$ for $L=1024$, and various values of $M$. We see that both these estimates match the simulation results closely up to a certain value of $M$. Beyond this point, the simulation results diverge from theoretical estimates due to lack of concentration in section error rates across trials, as described in Sec. III-A. Figure 11 compares the idealized codeword error probability in (18) with that obtained from simulations. Here, there is a good match between the estimate and the simulation results as the concentration of section error rates across trials plays no role — any trial with one or more section errors corresponds to one codeword error.

VI Comparison with Coded Modulation

In this section, we compare the performance of AMP-decoded SPARCs against coded modulation with LDPC codes. Specifically, we compare with two instances of coded modulation with LDPC codes from the WiMax standard IEEE 802.16e: 1) A $16$-QAM constellation with a rate $\frac{1}{2}$ LDPC code for an overall rate $R=1$ bit/channel use/real dimension, and 2) A $64$-QAM constellation with a rate $\frac{1}{2}$ LDPC code for an overall rate $R=1.5$ bits/channel use/real dimension. (The spectral efficiency is $2R$ bits/s/Hz.) The coded modulation results, shown in dashed lines in Figures 12 and 13, are obtained using the CML toolkit [16] with LDPC code lengths $n=576$ and $n=2304$.

Each figure compares the bit error rates (BER) of the coded modulation schemes with various SPARCs of the same rate, including a SPARC with a matching code length of $n=2304$. Using $P=E_{b}R$ and $\sigma^{2}=\frac{N_{0}}{2}$, the signal-to-noise ratio of the SPARC can be expressed as $\frac{P}{\sigma^{2}}=\frac{2RE_{b}}{N_{0}}$. The SPARCs are implemented using Hadamard-based design matrices, power allocation designed using the iterative algorithm in Sec. II-A with $B=L$, and online $\widehat{\tau}_{t}^{2}$ parameters with the early termination criterion (Sec. IV). An IPython notebook detailing the SPARC implementation is available at [17].

Figure 12 shows that for $L=1024$, the best value of $M$ among those considered increases from $M=2^{9}$ at lower snr values to $M=2^{13}$ at higher snr values. This is due to the effect discussed in Section III-A, where larger snr values can support larger values of $M$, before performance starts degrading due to loss of concentration.

At both $R=1$ and $R=1.5$, the SPARCs outperform the LDPC coded modulation at $E_{b}/N_{0}$ values close to the Shannon limit, but the error rate does not drop off as quickly at higher values of $E_{b}/N_{0}$. One way to enhance SPARC performance at higher snr is by treating it as a high-dimensional modulation scheme and adding an outer code. This is the focus of the next section.

VII AMP with Partial Outer Codes

Figures 12 and 13 show that for block lengths of the order of a few thousands, AMP-decoded SPARCs do not exhibit a steep waterfall in section error rate. Even at high $E_{b}/N_{0}$ values, it is still common to observe a small number of section errors. If these could be corrected, we could hope to obtain a sharp waterfall behavior similar to the LDPC codes.

In the simulations of the AMP decoder described above, when $M$ and $R_{\text{PA}}$ are chosen such that the average error rates are well-concentrated around the state evolution prediction, the number of section errors observed is similar across trials. Furthermore, we observe that the majority of sections decoded incorrectly are those in the flat region of the power allocation, i.e., those with the lowest allocated power. This suggests we could use a high-rate outer code to protect just these sections, sacrificing some rate, but less than if we naïvely protected all sections. We call the sections covered by the outer code protected sections, and conversely the earlier sections which are not covered by the outer code are unprotected. In [4], it was shown that a Reed-Solomon outer code (that covered all the sections) could be used to obtain a bound the probability of codeword error from a bound on the probability of excess section error rate.

Encoding with an outer code (e.g., LDPC or Reed-Solomon code) is straightforward: just replace the message bits corresponding to the protected sections with coded bits generated using the usual encoder for the chosen outer code. To decode, we would like to obtain bit-wise posterior probabilities for each codeword bit of the outer code, and use them as inputs to a soft-information decoder, such as a sum-product or min-sum decoder for LDPC codes. The output of the AMP decoding algorithm permits this: it yields $\beta^{T}$, which contains weighted section-wise posterior probabilities; we can directly transform these into bit-wise posterior probabilities. See Algorithm 2 for details.

Moreover, in addition to correcting AMP decoding errors in the protected sections, successfully decoding the outer code also provides a way to correct remaining errors in the unprotected sections of the SPARC codeword. Indeed, after decoding the outer code we can subtract the contribution of the protected sections from the channel output sequence $y$, and re-run the AMP decoder on just the unprotected sections. The key point is that subtracting the contribution of the later (protected) sections eliminates the interference due to these sections; then running the AMP decoder on the unprotected sections is akin to operating at a much lower rate.

Thus the decoding procedure has three stages: i) first round of AMP decoding, ii) decoding the outer code using soft outputs from the AMP, and iii) subtracting the contribution of the sections protected by the outer code, and running the AMP decoder again for the unprotected sections. We find that the final stage, i.e., running the AMP decoder again after the outer code recovers errors in the protected sections of the SPARC, provides a significant advantage over a standard application of an outer code, i.e., decoding the final codeword after the second stage.

We describe this combination of SPARCs with outer codes below, using an LDPC outer code. The resulting error rate curves exhibit sharp waterfalls in final error rates, even when the LDPC code only covers a minority of the SPARC sections.

We use a binary LDPC outer code with rate $R_{LDPC}$, block length $n_{LDPC}$ and code dimension $k_{LDPC}$, so that $k_{LDPC}/n_{LDPC}=R_{LDPC}$. For clarity of exposition we assume that both $n_{LDPC}$ and $k_{LDPC}$ are multiples of $\log M$ (and consequently that $M$ is a power of two). As each section of the SPARC corresponds to $\log M$ bits, if $\log M$ is an integer, then $n_{LDPC}$ and $k_{LDPC}$ bits represent an integer number of SPARC sections, denoted by

[TABLE]

respectively. The assumption that $k_{LDPC}$ and $n_{LDPC}$ are multiples of $\log M$ is not necessary in practice; the general case is discussed at the end of the next subsection.

We partition the $L$ sections of the SPARC codeword as shown in Fig 14. There are $L_{user}$ sections corresponding to the user (information) bits; these sections are divided into unprotected and protected sections, with only the latter being covered by the outer LDPC code. The parity bits of the LDPC codeword index the last $L_{parity}$ sections of the SPARC. For convenience, the protected sections and the parity sections together are referred to as the LDPC sections.

For a numerical example, consider the case where $L=1024$, $M=256$. There are $\log M=8$ bits per SPARC section. For a $(5120,4096)$ LDPC code ($R_{LDPC}=4/5$) we obtain the following relationships between the number of the sections of each kind:

[TABLE]

There are $L_{user}\log M=7168$ user bits, of which the final $k_{LDPC}=4096$ are encoded to a systematic $n_{LDPC}=5120$-bit LDPC codeword. The resulting $L\log M=8192$ bits (including both the user bits and the LDPC parity bits) are encoded to a SPARC codeword using the SPARC encoder and power allocation described in previous sections.

We continue to use $R$ to denote the overall user rate, and $n$ to denote the SPARC code length so that $nR=L_{user}\log M$. The underlying SPARC rate (including the overhead due to the outer code) is denoted by $R_{SPARC}$. We note that $nR_{SPARC}=L\log M$, hence $R_{SPARC}>R$. For example, with $R=1$ and $L,M$ and the outer code parameters as chosen above, $n=L_{user}(\log M)/R=7168$, so $R_{SPARC}=1.143$.

VII-A Decoding SPARCs with LDPC outer codes

At the receiver, we decode as follows:

Run the AMP decoder to obtain $\beta^{T}$. Recall that entry $j$ within section $\ell$ of $\beta^{T}$ is proportional to the posterior probability of the column $j$ being the transmitted one for section $\ell$. Thus the AMP decoder gives section-wise posterior probabilities for each section $\ell\in[L]$. 2. 2.

Convert the section-wise posterior probabilities to bit-wise posterior probabilities using Algorithm 2, for each of the $L_{LDPC}$ sections. This requires $O(L_{LDPC}M\log M)$ time complexity, of the same order as one iteration of AMP. 3. 3.

Run the LDPC decoder using the bit-wise posterior probabilities obtained in Step 2 as inputs. 4. 4.

If the LDPC decoder fails to produce a valid LDPC codeword, terminate decoding here, using $\beta^{T}$ to produce $\hat{\beta}$ by selecting the maximum value in each section (as per usual AMP decoding). 5. 5.

If the LDPC decoder succeeds in finding a valid codeword, we use it to re-run AMP decoding on the unprotected sections. For this, first convert the LDPC codeword bits to a partial $\hat{\beta}_{LDPC}$ as follows, using a method similar to the original SPARC encoding:

1. (a)

Set the first $L_{unprotected}M$ entries of $\hat{\beta}_{LDPC}$ to zero, 2. (b)

The remaining $L_{LDPC}$ sections (with $M$ entries per section) of $\hat{\beta}_{LDPC}$ will have exactly one-non zero entry per section, with the LDPC codeword determining the location of the non-zero in each section. Indeed, noting that $n_{LDPC}=L_{LDPC}\log M$, we consider the LDPC codeword as a concatenation of $L_{LDPC}$ blocks of $\log M$ bits each, so that each block of bits indexes the location of the non-zero entry in one section of $\hat{\beta}_{LDPC}$. The value of the non-zero in section $\ell$ is set to $\sqrt{nP_{\ell}}$, as per the power allocation.

Now subtract the codeword corresponding to $\hat{\beta}_{LDPC}$ from the original channel output $y$, to obtain $y^{\prime}=y-A\hat{\beta}_{LDPC}$. 6. 6.

Run the AMP decoder again, with input $y^{\prime}$, and operating only over the first $L_{unprotected}$ sections. As this operation is effectively at a much lower rate than the first decoder (since the interference contribution from all the protected sections is removed), it is more likely that the unprotected bits are decoded correctly than in the first AMP decoder.

We note that instead of generating $y^{\prime}$, one could run the AMP decoder directly on $y$, but enforcing that in each AMP iteration, each of the $L_{LDPC}$ sections has all its non-zero mass on the entry determined by $\hat{\beta}_{LDPC}$, i.e., consistent with Step 5.b). 7. 7.

Finish decoding, using the output of the final AMP decoder to find the first $L_{unprotected}M$ elements of $\hat{\beta}$, and using $\hat{\beta}_{LDPC}$ for the remaining $L_{LDPC}M$ elements.

In the case where $n_{LDPC}$ and $k_{LDPC}$ are not multiples of $\log M$, the values $L_{LDPC}=n_{LDPC}/\log M$ and $L_{protected}=k_{LDPC}/\log M$ will not be integers. Therefore one section at the boundary of $L_{unprotected}$ and $L_{protected}$ will consist of some unprotected bits and some protected bits. Encoding is not affected in this situation, as the LDPC encoding happens prior to SPARC codeword encoding. When decoding, conversion to bit-wise posterior probabilities is performed for all sections containing LDPC bits (including the intermediate section at the boundary) and only the $n_{LDPC}$ bit posteriors corresponding to the LDPC codeword are given to the LDPC decoder. When forming $\hat{\beta}_{LDPC}$, the simplest option is to treat the intermediate section as though it were unprotected and set it to zero. It is also possible to compute column posterior probabilities which correspond to the fixed LDPC bits and probabilities arising from $y$, though doing so is not covered in this paper.

VII-B Simulation results

The combined AMP and outer LDPC setup described above was simulated using the (5120, 4096) LDPC code ($R_{LDPC}=4/5$) specified in [18] with a min-sum decoder. Bit error rates were measured only over the user bits, ignoring any bit errors in the LDPC parity bits.

Figure 15 plots results at overall rate $R=\frac{4}{5}$, where the underlying LDPC code (modulated with BPSK) can be compared to the SPARC with LDPC outer code, and to a plain SPARC with rate $\frac{4}{5}$. In this case $R_{PA}=0$, giving a flat power allocation. Figure 16 plots results at overall rate $R=1.5$, where we can compare to the QAM-64 WiMAX LDPC code, and to the plain SPARC with rate 1.5 of Figure 13.

The plots show that protecting a fraction of sections with an outer code does provide a steep waterfall above a threshold value of $\frac{E_{b}}{N_{0}}$. Below this threshold, the combined SPARC + outer code has worse performance than the plain rate $R$ SPARC without the outer code. This can be explained as follows. The combined code has a higher SPARC rate $R_{SPARC}>R$, which leads to a larger section error rate for the first AMP decoder, and consequently, to worse bit-wise posteriors at the input of the LDPC decoder. For $\frac{E_{b}}{N_{0}}$ below the threshold, the noise level at the input of the LDPC decoder is beyond than the error-correcting capability of the LDPC code, so the LDPC code effectively does not correct any section errors. Therefore the overall performance is worse than the performance without the outer code.

Above the threshold, we observe that the second AMP decoder (after subtracting the contribution of the LDPC-protected sections) is successful at decoding the unprotected sections that were initially decoded incorrectly. This is especially apparent in the $R=\frac{4}{5}$ case (Figure 15), where the section errors are uniformly distributed over all sections due to the flat power allocation; errors are just as likely in the unprotected sections as in the protected sections.

VII-C Outer code design choices

In addition to the various SPARC parameters discussed in previous sections, performance with an outer code is sensitive to what fraction of sections are protected by the outer code. When more sections are protected by the outer code, the overhead of using the outer code is also higher, driving $R_{SPARC}$ higher for the same overall user rate $R$. This leads to worse performance in the initial AMP decoder, which has to operate at the higher rate $R_{SPARC}$. As discussed above, if $R_{SPARC}$ is increased too much, the bit-wise posteriors input to the LDPC decoder are degraded beyond its ability to successfully decode, giving poor overall performance.

Since the number of sections covered by the outer code depends on both $\log M$ and $n_{LDPC}$, various trade-offs are possible. For example, given $n_{LDPC}$, choosing a larger value of $\log M$ corresponds to fewer sections being covered by the outer code. This results in smaller rate overhead, but increasing $\log M$ may also affect concentration of the error rates around the SE predictions, as discussed in Section III-A. We conclude with two remarks about the choice of parameters for the SPARC and the outer code.

When using an outer code, it is highly beneficial to have good concentration of the section error rates for the initial AMP decoder. This is because a small number of errors in a single trial can usually be fully corrected by the outer code, while occasional trials with a very large number of errors cannot. 2. 2.

Due to the second AMP decoder operation, it is not necessary for all sections with low power to be protected by the outer code. For example, in Figure 15, all sections have equal power, and around 30% are not protected by the outer code. Consequently, these sections are often not decoded correctly by the first decoder. Only once the protected sections are removed is the second decoder able to correctly decode these unprotected sections. In general the aim should be to cover all or most of the sections in the flat region of the power allocation, but experimentation is necessary to determine the best trade-off.

An interesting direction for future work would be to develop an EXIT chart analysis to jointly optimize the design of the SPARC and the outer LDPC code.

Acknowledgement

The authors thank the Editor and the anonymous referees for several helpful comments which improved the paper.