The Velocity of the Decoding Wave for Spatially Coupled Codes on BMS Channels

TL;DR

This paper analyzes the wave-like propagation of decoding errors in spatially coupled LDPC codes over BMS channels, deriving a formula for the wave velocity and supporting it with numerical evidence.

Contribution

It provides a theoretical formula for the velocity of the decoding wave in spatially coupled codes and introduces practical approximations validated by numerical data.

Findings

Derived a formula for the decoding wave velocity v

Proposed simpler approximations for v

Supported findings with numerical data

Abstract

We consider the dynamics of belief propagation decoding of spatially coupled Low-Density Parity-Check codes. It has been conjectured that after a short transient phase, the profile of "error probabilities" along the spatial direction of a spatially coupled code develops a uniquely-shaped wave-like solution that propagates with constant velocity v. Under this assumption, and for transmission over general Binary Memoryless Symmetric channels, we derive a formula for v. We also propose approximations that are simpler to compute and support our findings using numerical data.

Tables

Table 1. TABLE I: Normalized velocities of the wave on the LDPC( x 3 , x 6 superscript 𝑥 3 superscript 𝑥 6 x^{3},x^{6} ) on the BEC( ϵ italic-ϵ \epsilon ) with L = 256 𝐿 256 L=256 for w = 8 𝑤 8 w=8

$ϵ$	0.455	0.465	0.475	0.485
$v_{\text{BEC}}$	0.05754	0.03741	0.02004	0.00456
$v_e$	0.05813	0.03750	0.02000	0.00468
$v_{a_2}$	0.03470	0.02623	0.01663	0.00476
$v_{\text{B}}/\alpha$	0.06108	0.03992	0.02149	0.00491

Table 2. TABLE II: Normalized velocities of the waves on the LDPC( x 3 , x 6 superscript 𝑥 3 superscript 𝑥 6 x^{3},x^{6} ) and LDPC( x 4 , x 8 superscript 𝑥 4 superscript 𝑥 8 x^{4},x^{8} ) for the BIAWGNC( σ n 2 superscript subscript 𝜎 𝑛 2 \sigma_{n}^{2} ) with L = 100 𝐿 100 L=100 , w = 3 𝑤 3 w=3 , within GA.

$2/{\sigma }_n^2$	2.33	2.35	2.38	2.40
$v_{\text{GA}}$, $(3,6)$	0.0183	0.0222	0.0283	0.0325
$v_e$, $(3,6)$	0.0183	0.0233	0.0317	0.0375
$v_{\text{GA}}$, $(4,8)$	0.0237	0.0258	0.0312	0.0381
$v_e$, $(4,8)$	0.0217	0.0250	0.0308	0.0342

Equations

$$H(\mathtt{x})=\int \mathrm{d}\mathtt{x}(\alpha ){\log }_2(1+e^{-\alpha })$$

$${\mathtt{x}}^{(t+1)}={\mathtt{c}}_{\mathtt{h}}\text{varoast}{\lambda }^{\text{varoast}}({\rho }^{\text{boxast}}({\mathtt{x}}^{(t)}))$$

$$W_s(\mathtt{x};\mathtt{c})$$

$$-H(\mathtt{x}\text{boxast}{\rho }^{\text{boxast}}(\mathtt{x}))-\frac{1}{L^{\prime }(1)}H(\mathtt{c}\text{varoast}{L}^{\text{varoast}}({\rho }^{\text{boxast}}(\mathtt{x}))).$$

$$\displaystyle\mathtt{x}_{z}^{(t+1)}=\frac{1}{w}\sum\limits_{i=0}^{w-1}\mathtt{c}_{z-i}\varoast\lambda^{\varoast}\Bigg{(}\frac{1}{w}\sum\limits_{j=0}^{w-1}\rho^{\boxast}\Big{(}\mathtt{x}_{z-i+j}^{(t)}\Big{)}\Bigg{)}.$$

$$\displaystyle U(\underline{\mathtt{x}};\mathtt{c})=\sum\limits_{z=-w+1}^{L}\Bigg{\{}\frac{1}{R^{\prime}(1)}H(R^{\boxast}(\mathtt{x}_{z}))+H(\rho^{\boxast}(\mathtt{x}_{z}))$$

$$\displaystyle-H(\mathtt{x}_{z}\boxast\rho^{\boxast}(\mathtt{x}_{z}))-\frac{1}{L^{\prime}(1)}H\Big{(}\mathtt{c}_{z}\varoast L^{\varoast}\Big{(}\frac{1}{w}\sum\limits_{i=0}^{w-1}\rho^{\boxast}(\mathtt{x}_{z+i})\Big{)}\Big{)}\Bigg{\}}.$$

$$\displaystyle\mathtt{x}(z,t+1)=\int_{0}^{1}\mathrm{d}u\,\mathtt{c}(z-u)\varoast\lambda^{\varoast}\Big{(}\int_{0}^{1}\mathrm{d}s\,\rho^{\boxast}\big{(}\mathtt{x}(z-u+s,t)\big{)}\Big{)},$$

$$\mathcal{W}$$

$$+H({\rho }^{\text{boxast}}(\mathtt{x}(z,t)))-H(\mathtt{x}(z,t)\text{boxast}{\rho }^{\text{boxast}}(\mathtt{x}(z,t)))$$

$$-H({\rho }^{\text{boxast}}({\mathtt{x}}_0(z,t)))+H({\mathtt{x}}_0(z,t)\text{boxast}{\rho }^{\text{boxast}}({\mathtt{x}}_0(z,t)))$$

$$\displaystyle\qquad-\frac{1}{L^{\prime}(1)}H\Big{(}\mathtt{c}(z)\varoast L^{\varoast}\big{(}\int_{0}^{1}\mathrm{d}s\,\rho^{\boxast}(\mathtt{x}(z+s,t))\big{)}\Big{)}$$

$$\displaystyle\qquad+\frac{1}{L^{\prime}(1)}H\Big{(}\mathtt{c}(z)\varoast L^{\varoast}\big{(}\int_{0}^{1}\mathrm{d}s\,\rho^{\boxast}(\mathtt{x}_{0}(z+s,t))\big{)}\Big{)}\Big{\}}.$$

$$\displaystyle\int_{\mathbb{R}}\mathrm{d}z\,H\Big{(}(\mathtt{x}(z,t+1)-$$

$$=\frac{\delta \mathcal{W}}{\delta \mathtt{x}}[\eta (z,t)]$$

$$\displaystyle v=\frac{\Delta E}{\int_{\mathbb{R}}\mathrm{d}z\,H\Big{(}\rho^{\prime\boxast}(\mathtt{X}(z))\boxast(\mathtt{X}^{\prime}(z))^{\boxast 2}\Big{)}},$$

$$\mathtt{X}(z)$$

$$\displaystyle v\int_{\mathbb{R}}\mathrm{d}z\,H\Big{(}\rho^{\prime\boxast}(\mathtt{X}(z))\boxast\mathtt{X}^{\prime}(z)^{\boxast 2}\Big{)}.$$

$$\frac{\delta {\mathcal{W}}_s}{\delta \mathtt{x}}[{\mathtt{X}}^{\prime }(z)]=W_s({\mathtt{x}}_{\text{BP}};\mathtt{c})-W_s({\Delta }_{\infty };\mathtt{c}),\frac{\delta {\mathcal{W}}_i}{\delta \mathtt{x}}[{\mathtt{X}}^{\prime }(z)]=0.$$

$$\displaystyle\psi(m)=\big{(}\sqrt{4\pi m}\,\big{)}^{-1}\int_{\mathbb{R}}\mathrm{d}z\,e^{-(z-m)^{2}/4m}\log_{2}(1+e^{-z}).$$

$$v_{\text{GA}}=\frac{W_s^{\text{GA}}(p_{\text{BP}};c)-W_s^{\text{ GA}}(0;c)}{-(r-1){\int }_{\mathbb{R}}\mathrm{d}z(p^{\prime }(z){)}^2\frac{{\psi }^{\prime \prime }((r-2){\psi }^{-1}(1-p(z)))}{({\psi }^{\prime }({\psi }^{-1}(1-p(z))){)}^2}},$$

$$\displaystyle W_{s}^{\text{\tiny GA}}(p;c)=\big{(}1-\frac{1}{r}\big{)}\psi\big{(}r\psi^{-1}(1-p)\big{)}-\psi\big{(}(r-1)\psi^{-1}(1-p)\big{)}$$

$$\displaystyle+\frac{1}{r}-\frac{1}{\ell}\psi\big{(}\psi^{-1}(H(\mathtt{c}))+\ell\psi^{-1}\big{(}1-\psi((r-1)\psi^{-1}(1-p))\big{)}\big{)}$$

$$\displaystyle p(z)-v_{\text{\tiny GA}}p^{\prime}(z)=\int_{0}^{1}\mathrm{d}u\,\psi\Big{(}\psi^{-1}(H(\mathtt{c}))$$

$$\displaystyle+(\ell-1)\psi^{-1}\Big{(}1-\int_{0}^{1}\mathrm{d}s\,\psi\big{(}(r-1)\psi^{-1}(1-p(z-u+s))\big{)}\Big{)}\Big{)}.$$

$$v_{\text{BEC}}=\frac{W_{\text{BEC}}(x_{\text{BP}};ϵ)-W_{\text{BEC}}(0;ϵ)}{{\int }_{\mathbb{R}}\mathrm{d}z{\rho }^{\prime }(1-x(z))(x^{\prime }(z){)}^2},$$

$$W_{\text{BEC}}(x;ϵ)=\frac{1-R(1-x)}{R^{\prime }(1)}-x\rho (1-x)-\frac{ϵL(1-\rho (1-x))}{L^{\prime }(1)}.$$

$$x(z)-v_{\text{\tiny BEC}}x^{\prime}(z)=\epsilon\int_{0}^{1}\mathrm{d}u\,\lambda\Big{(}\int_{0}^{1}\mathrm{d}s\,(1-\rho(1-x(z-u+s))\Big{)}.$$

$$v_{\text{B}}=\alpha \frac{W_{\text{BEC}}(x_{\text{BP}};ϵ)-W_{\text{BEC}}(0;ϵ)}{\sum _{z\in \mathbb{Z}}{\rho }^{\prime }(1-x_z)(x_z-x_{z-1}{)}^2},\alpha \le 2.$$

$$v_{a_i}$$

Full text

I Introduction

Spatial coupling is a construction of Low-Density Parity-Check (LDPC) codes that has been shown to be capacity-achieving on general Binary Memoryless Symmetric (BMS) channels under Belief Propagation (BP) decoding [1]. The capacity-achieving property is due to the “threshold saturation” of the BP threshold of the coupled system towards the maximum a-posteriori (MAP) threshold of the uncoupled code ensemble [1], [2].

To study the performance of codes under spatial coupling, it is useful to analyze the decoding profile along the spatial axis of coupling. For the sake of the discussion let the integer $z\in\{0,\dots,L-1\}$ denote the position along the spatial direction of the graph construction, where $L$ is the length of the coupling chain. In the general framework of BMS channels the decoding profile consists of the vector of probability distributions of the log-likelihoods of bits under BP decoding. The $z$-th component of this vector, call it $\mathtt{x}_{z}$, equals the log-likelihood distribution of the bits located at the $z$-th position. In the special case of the Binary Erasure Channel (BEC) this reduces to a vector of erasure probabilities $0<x_{z}<1$. This decoding profile satisfies a set of coupled Density Evolution (DE) iterative equations. It has been proven that under DE iterations, as long as the channel noise is below the MAP threshold, DE iterations drive $\mathtt{x}_{z}$ to the all-$\Delta_{\infty}$ vector (the Dirac mass at infinite log-likelihood, i.e. perfect knowledge of the bits) [1], [2]. In the special case of the BEC this corresponds to a vector of erasure probabilities driven to zero by DE iterations [3].

An interesting phenomenon that occurs during decoding, when the channel noise is between the BP and the MAP thresholds, is the appearance of a decoding wave or soliton. It has been observed that after a transient phase the decoding profile develops a fixed shape that seems independent of the initial condition and travels at constant velocity $v$. The soliton is depicted in Fig. 1 for the case of a $(3,6)$-regular spatially coupled code on the BEC. Note that the same phenomenon has been observed for general BMS channels.

A solitonic wave solution has been proven to exist in [4] for the BEC, but the question of the independence of the shape from initial conditions is left open. In [4] and [5], bounds on the velocity of the wave for the BEC are proposed. In [6] a formula for the wave in the context of the coupled Curie-Weiss toy model is derived and tested numerically.

In this work we derive a formula for the velocity of the wave in the continuum limit $L\gg w\gg 1$, with transmission over general BMS channels (see Equ. (7)). Our derivation rests on the assumption that the soliton indeed appears. For simplicity we limit ourselves to the case where the underlying uncoupled LDPC codes has only one nontrivial stable BP fixed point.

For the specific case of the BEC the formula greatly simplifies because density evolution reduces to a one-dimensional scalar system of equations. For general channels we also apply the Gaussian approximation [7], which reduces the problem to a one-dimensional scalar system and yields a more tractable velocity formula. We compare the numerical predictions of these velocity formulas with the empirical value of the velocity for finite $L$ and $w$.

We propose a further approximation scheme (valid close to the MAP threshold) that expresses the velocity solely in terms of degree distributions of the code (in the spirit of [4], [5]). This may be useful to provide easy-to-handle design principles that maximize the velocity.

Our formula can be applied to estimate parameters involved in the scaling law [8] of finite-size ensembles. This is briefly discussed as a further possible application.

The derivation of the velocity formula combines the use of the “potential functional” introduced and used in a series of works ([2], [3], [9], [10]) and the continuum limit $L\gg w\gg 1$ which makes the derivations analytically tractable.

In section II, we describe the setup and notation. In section III, we state our main result and show a sketch of the its derivation. The Gaussian approximation, the application to the BEC as well as further approximations, and the application to finite-size ensembles are discussed in section IV. More details for some of the derivations can be found in [11].

II Potential formulation and continuum limit

We consider (almost) the same setting and notation as in [2]. One important difference is that we will later consider the continuum limit, which is an approximation of the discrete expressions in the regime of large spatial length $L$ and window size $w$.

II-A Preliminaries

Consider a symmetric probability measure $\mathtt{x}(\alpha)$ on the extended real numbers $\bar{\mathbb{R}}$. These are measures satisfying $\mathtt{x}(\alpha)=e^{\alpha}\mathtt{x}(-\alpha)$ for all $\alpha\in\bar{\mathbb{R}}$. Here $\alpha\in\bar{\mathbb{R}}$ is interpreted as a “log-likelihood variable”. The (linear) entropy functional is defined as

[TABLE]

and will play an important role.

In the sequel we will use the Dirac masses $\Delta_{0}(\alpha)$ and $\Delta_{\infty}(\alpha)$ at zero and infinite likelihood, respectively. We will also need the standard variable-node and check-node convolution operators $\varoast$ and $\boxast$ for log-likelihood ratio (LLR) message distributions involved in DE equations (see [12]).

II-B Single system

Consider an LDPC($\lambda,\rho$) code ensemble and transmission over the BMS channel. Here $\lambda(y)=\sum_{l}\lambda_{l}y^{l-1}$ and $\rho(y)=\sum_{r}\rho_{r}y^{r-1}$ are the usual edge-perspective variable-node and check-node degree distributions. The node-perspective degree distributions $L$ and $R$ are defined by $L^{\prime}(y)=L^{\prime}(1)\lambda(y)$ and $R^{\prime}(y)=R^{\prime}(1)\rho(y)$.

We denote by $\mathtt{x}^{(t)}(\alpha)$ the variable-node output distribution at time $t\in\mathbb{N}$. We consider a family of BMS channels whose distribution $\mathtt{c}_{\mathtt{h}}(\alpha)$ in the log-likelihood domain is parametrized by the channel entropy $H(\mathtt{c}_{\mathtt{h}})=\mathtt{h}$. On the BEC, for example, we have $\mathtt{c}_{\epsilon}(\alpha)=\epsilon\Delta_{0}(\alpha)+(1-\epsilon)\Delta_{\infty}(\alpha)$, $H(\mathtt{c}_{\epsilon})=\epsilon$.

We can track the average behavior of the BP decoder by means of the DE iterative equation [12]

[TABLE]

with initial condition $\mathtt{x}^{(0)}=\Delta_{0}$. The BP threshold $\mathtt{h}_{\text{\tiny BP}}$ is the largest value of $\mathtt{h}$ for which the DE recursion converges to $\Delta_{\infty}$.

From now on we will omit the subscript $\mathtt{h}$ and the argument $\alpha$ of the distribution $\mathtt{c}_{\mathtt{h}}(\alpha)$ to alleviate notation. Later on, $\mathtt{c}_{z}$ and $\mathtt{c}(z)$ describe the channel distribution at the spatial position $z$ in discrete and continuous settings, respectively.

The potential functional $W_{s}(\mathtt{x};\mathtt{c})$ (of the “single” or uncoupled system) is

[TABLE]

The DE equation is obtained by setting to zero the functional derivative of $W_{s}(\mathtt{x};\mathtt{c})$ with respect to $\mathtt{x}$.

The BP threshold $\mathtt{h}_{\text{\tiny BP}}$ is strictly smaller than the MAP threshold $\mathtt{h}_{\text{\tiny MAP}}$. Spatial coupling, however, exhibits the attractive property of threshold saturation which makes it possible to decode perfectly up till $\mathtt{h}_{\text{\tiny MAP}}$. The definitions of the BP and MAP thresholds above extend to the spatially coupled setting.

II-C Spatially coupled system

Since the natural setting for coupling is discrete, we first describe the system in discrete space before taking the continuum limit.

The coupled LDPC($\lambda,\rho$) code ensemble is defined as follows. Consider $L+w$ “replicas” of the single system described in section II-B, on the spatial coordinates $z\in\{-w+1,...,L\}$. The system at position $z$ is coupled to other systems by means of a uniform coupling window of width $w$. We denote by $\mathtt{x}_{z}^{(t)}$ the check-node input distribution at position $z\in\{-w+1,...,L\}$ on the spatial axis, and at time $t\in\mathbb{N}$. We then write the DE equation of the coupled system in discrete space as

[TABLE]

Here $\mathtt{c}_{z}=\mathtt{c}$, for $z\in\{0,\dots,L\}$ and $\mathtt{c}_{z}=\Delta_{\infty}$ otherwise. We fix the left boundary to $\mathtt{x}_{z}^{(t)}=\Delta_{\infty}$ for $z\in\{-w+1,...,-1\}$, for all $t\in\mathbb{N}$. The initial condition on the right side is $\mathtt{x}_{z}^{(0)}=\Delta_{0}$, for $z\in\{0,...,L\}$. The initialization to perfect information at the left boundary is what allows seed propagation along the chain of coupled codes.

We denote by $\underline{\mathtt{x}}$ the profile vector. Then the expression of the discrete potential functional is

[TABLE]

The coupled DE equation (4) is obtained by setting to zero the functional derivative of this potential with respect to $\underline{\mathtt{x}}$.

II-D Continuum Limit

We now consider the coupled system in the continuum limit (see [4], [13], [14] for the case of the BEC) $L\rightarrow+\infty$ and then $w\rightarrow+\infty$. We set $\mathtt{x}(\frac{z}{w},t)\equiv\mathtt{x}_{z}^{(t)}$ and replace $\frac{z}{w}\to z$ where the new $z$ is a continuous variable on the spatial axis, $z\in\mathbb{R}$ (we slightly abuse notation here). The DE equation (4) becomes

[TABLE]

where $\mathtt{c}(z)$ is now the BMS channel distribution at the continuous spatial position $z\in\mathbb{R}$ (we again slightly abuse notation) and the boundary / initial conditions are $\mathtt{x}(z,t)=\Delta_{\infty}$, for $z<0$ and all $t\in\mathbb{R}$ / $\mathtt{x}(z,0)=\Delta_{0}$ for $z\geq 0$.

The potential functional $\mathcal{W}(\mathtt{x};\mathtt{c})$ of the coupled system in the continuum limit is obtained from (5). In order to get a finite result when $L\to+\infty$ we must normalize the potential by subtracting an “energy” associated to a fixed profile $\mathtt{x}_{0}$ that satisfies $\mathtt{x}_{0}(z,t)\to\Delta_{\infty}$ when $z\to-\infty$ and $\mathtt{x}_{0}(z,t)\to\mathtt{x}_{\text{\tiny BP}}$ when $z\to+\infty$ ($\mathtt{x}_{\text{\tiny BP}}$ is the nontrivial stable fixed point log-likelihood density of BP for the single uncoupled code ensemble), for all $t\in\mathbb{N}$. The functional is thus defined as follows,

[TABLE]

It can be shown that the integral converges under suitable assumptions on the entropy of $\mathtt{x}$ as $z\to\pm\infty$.

Once the functional derivative of $\mathcal{W}$ in the direction $\eta$ is computed, one finds that the DE equation is equivalent to

[TABLE]

This is a gradient descent equation in an infinite-dimensional space of measures.

III Main Result

III-A Velocity Formula

We restrict ourselves to code ensembles with a single nontrivial stable BP fixed point. Consider the case when the channel entropy $\mathtt{h}$ satisfies $\mathtt{h}_{\text{\tiny BP}}<\mathtt{h}<\mathtt{h}_{\text{\tiny MAP}}$. After a few iterations of DE, which we call the “transient phase”, one observes a solitonic (wavelike) behavior as depicted in Figure 1. This motivates us to make the following assumptions: (i) after a transient phase the profile develops a fixed shape $\mathtt{X}(\cdot)$; (ii) the shape is independent of the initial condition; (iii) the shape travels at constant speed $v$; (iv) the shape satisfies the boundary conditions $\mathtt{X}(z)\to\Delta_{\infty}$ for $z\to-\infty$ and $\mathtt{X}(z)\to\mathtt{x}_{\text{\tiny BP}}$ for $z\to+\infty$. We thus make the ansatz $\mathtt{x}(z,t)=\mathtt{X}(z-vt)$.

From this ansatz and equation (6) we obtain our main result. The velocity of the soliton is (primes are derivatives)

[TABLE]

where $\Delta E$ is the energy gap defined as $\Delta E=W_{s}(\mathtt{x}_{\text{\tiny BP}};\mathtt{c})-W_{s}(\Delta_{\infty};\mathtt{c})$. We recall $W_{s}(\cdot;\cdot)$ is the potential of the uncoupled system (3), and $\Delta_{\infty}$ the trivial fixed point (Dirac mass at infinity). With our normalizations $W_{s}(\Delta_{\infty};c)=0$. Fig. 2 illustrates the energy gap $\Delta E$ for the $(3,4)$-regular code ensemble on the BEC($\epsilon$).

Formula (7) involves the soliton shape $\mathtt{X}$. Using the DE equation and $x(z,t)=X(z-vt)$ we find that $\mathtt{X}(z)$ is the solution of

[TABLE]

Equs. (7)-(8) form a closed system of equations that can be solved iteratively to obtain $\mathtt{X}$ and $v$.

III-B Brief Sketch of Derivation

Consider equation (6). Under the ansatz $\mathtt{x}(z,t)=\mathtt{X}(z-vt)$ for small $v$ we get $\mathtt{x}(z,t+1)-\mathtt{x}(z,t)\approx-v\mathtt{X}^{\prime}(z-vt)$. Choosing the direction $\eta(z,t)=\mathtt{X}^{\prime}(z-vt)$ we can rewrite (after a few manipulations involving properties of $\otimes$ and $\boxplus$) the left-hand side of (6) as

[TABLE]

We now consider the right-hand side of (6), which is the functional derivative of $\mathcal{W}(\mathtt{x};\mathtt{c})$ in the direction of $\eta(z,t)=\mathtt{X}^{\prime}(z-vt)$. We first split $\mathcal{W}$ into two parts: the single system potential $\mathcal{W}_{s}(\mathtt{x};\mathtt{c})$ that remains if we ignore the coupling effect, and the rest which constitutes the “interaction potential” $\mathcal{W}_{i}(\mathtt{x};\mathtt{c})$ (see [13] for similar splittings or [11] for the exact definitions). With some care one can then show that

[TABLE]

We conclude that the “interaction” part does not contribute to the velocity and only the energy gap remains.

IV Applications

IV-A Gaussian Approximation

In order to simplify the analysis of a spatially coupled ensemble with transmission over general BMS channels, one may use the method of Gaussian approximations [7]. The idea is to assume that the densities of the LLR messages appearing in the DE equations are symmetric Gaussian densities (symmetric Gaussian densities $\mathtt{x}(\alpha)=(\sqrt{2\pi}\sigma)^{-1}\exp(-(\alpha-m)^{2}/2\sigma^{2})$ are characterized by the relation $\sigma^{2}=2m$). Furthermore the channel density $\mathtt{c}$ is replaced by a BIAWGNC($\sigma_{n}^{2}$) with the same entropy $H(\mathtt{c})$.

The system becomes scalar one-dimensional since one only tracks the evolution of the means or equivalently the entropies of the densities. For simplicity, we consider the $(\ell,r)$-regular ensemble. Density evolution is conveniently expressed in terms of the entropies $p(z,t)=H(\mathtt{x}(z,t))$. One then observes a “scalar” wave propagation much like the one of Fig. 1. The entropy of a symmetric Gaussian density of mean $m$ equals

[TABLE]

With this function, the Gaussian approximation for the velocity reads (primes are derivatives)

[TABLE]

where $p(z)$ denotes the shape the entropy profile, $p_{\text{\tiny BP}}=H(\mathtt{x}_{\text{\tiny BP}})$, and (3) now becomes

[TABLE]

(here $\psi^{-1}(H(\mathtt{c}))$ is the mean of the BIAWGNC($\sigma_{n}$) that has the same entropy as the channel $\mathtt{c}$ and equals $2/\sigma_{n}^{2}$). The shape $p(z)$ is computed from

[TABLE]

IV-B Velocity on the BEC

For the BEC we can directly simplify (7) (alternatively one can rederive the formula using directly the continuum approximation over the BEC). For this case, the channel distribution is $\mathtt{c}=\epsilon\Delta_{0}+(1-\epsilon)\Delta_{\infty}$. The fixed shape of the decoding wave is entirely characterized by the (scalar) erasure probability $x(z)$, i.e., $\mathtt{X}(z)=x(z)\Delta_{0}+(1-x(z))\Delta_{\infty}$ and $\mathtt{X}^{\prime}(z)=x^{\prime}(z)\Delta_{0}-x^{\prime}(z)\Delta_{\infty}$. Then using $\mathtt{X}(z)\varoast\Delta_{0}=\mathtt{X}(z)$, $\mathtt{X}(z)\varoast\Delta_{\infty}=\Delta_{\infty}$, $\mathtt{X}(z)\boxast\Delta_{0}=\Delta_{0}$, $\mathtt{X}(z)\boxast\Delta_{\infty}=\mathtt{X}(z)$, we find $\lambda^{\varoast}(\mathtt{X}(z))\to\lambda(x(z))$, $\rho^{\boxast}(\mathtt{X}(z))\to 1-\rho(1-x(z))$. The velocity becomes

[TABLE]

where the single potential (3) now is

[TABLE]

Again, the erasure profile has to be computed from the one-dimensional equation

[TABLE]

The formula obtained here for the BEC is obviously very similar to the upper bound proved in [5] (Theorem 1) for a discrete system

[TABLE]

In [5] it is conjectured based on numerical simulations that $\alpha=1$ would be a tight bound. Our results here are perfectly consistent with the findings of [5].

IV-C Further approximations for scalar systems

The one-dimensional formulas for the velocity might help design degree distributions. However it is costly to compute the shape of the profile (that enters the denominators) for every degree distribution. We propose a hierarchy of approximations for the velocity that involve only the degree distributions and quantities related to the single system, i.e, no profile shape needs to be computed. These are good for $\epsilon$ close to $\epsilon_{\text{\tiny MAP}}$. The first two approximations of the hierarchy are, for $i=1,2$,

[TABLE]

where $F_{i}(x)=\int_{0}^{x}\mathrm{d}y\,(1-\rho^{-1}(1-\lambda^{-1}(f_{i}(y)/\epsilon)))$ and where $f_{1}(y)=y$, $f_{2}(y)=y-v_{a_{1}}\sqrt{-12y^{2}+24F_{1}(y)}$. The derivation is not shown here due to length constraints, but is to some extent inspired from that of [6] for the coupled Curie-Weiss model introduced in [9], [10]. We note that the approximation scheme breaks down if the quantities under the square roots are negative; this depends on the code parameters.

One can also derive similar approximative formulas within the Gaussian approximation which also is a one-dimensional scalar system. This is not shown here.

IV-D Numerical Simulations

In this section we compare numerical predictions with the velocity formulas for the cases of the BEC($\epsilon$) and BIAWGNC($\sigma_{n}^{2}$), to the further approximations, and also to the empirical velocity. The empirical velocity $v_{e}$ is the velocity calculated from the decoding profiles obtained by running DE (4). In particular, it is equal to the average of $\Delta z/(w\Delta I)$, where $\Delta z$ is the spatial distance between the centers of the kinks (or fronts) of two profiles, and $\Delta I$ is the number of iterations of DE that were made to go from the first profile to the second. It serves as the reference value of the velocity with which we compare our formula.

Figure 3 shows velocities (normalized by $w$) for the BEC for a $(3,6)$ coupled code with $L=128$ and $w=3$ in a range close to the MAP threshold. The velocity decreases to zero as $\epsilon$ approaches $\epsilon_{\text{\tiny MAP}}$ and confirms that the soliton becomes “static” at the MAP threshold (we point out that at $\epsilon_{\text{\tiny MAP}}$ the existence and unicity up to translations of the static shape has been proven by displacement convexity techniques [13], [14]). We see that the theoretical velocity $v_{\text{\tiny BEC}}$ and empirical velocity $v_{e}$ nicely match over a wide range of noise. The approximation $v_{a_{2}}$ works very well for $\epsilon$ close to $\epsilon_{\text{\tiny MAP}}$, and also with respect to $v_{\text{\tiny B}}$ with the added advantage that it does not require running DE. Similar findings are also illustrated in table I for the coupled $(3,6)$-regular ensemble for $L=256$, $w=8$, and for different values of $\epsilon$.

Table II compares the theoretical and empirical velocities, $v_{\text{\tiny GA}}$ and $v_{e}$ for a BIAWGNC($\sigma_{n}^{2}$). The velocities are obtained for the $(3,6)$- and the $(4,8)$-regular ensembles for different values of $\psi^{-1}(H(c))=2/\sigma_{n}^{2}$ (twice the signal to noise ratio).

IV-E Scaling Law for Finite-Length Coupled Codes

The authors in [8] propose a scaling law to predict the error probability of a finite-length spatially coupled $(\ell,r,L)$ ensemble when transmission takes place over the BEC. The derived scaling law depends on “scaling parameters”, one of which we will relate to the velocity of the decoding wave. Note that the $(\ell,r,L)$ ensemble in [8] is slightly different than the one here but it is still of interest to discuss an application of the velocity formula to the scaling law.

Whenever a variable node is decoded, it is removed from the graph along with its edges. One way to track this peeling process is to analyze the evolution of the degree distribution of the residual graph across iterations, which serves as a sufficient statistic. This statistic can be described by a system of differential equations, whose solution determines the mean of the fraction of degree-one check nodes $\hat{r}_{1}$ and the variance (around this mean) at any time during the decoding process. As shown in [8] there exists a “steady state phase” where the mean and the variance are constant, and during which one can observe the appearance of the decoding wave. (Note that here we consider one-sided termination instead of two-sided termination in [8], so the fraction $\hat{r}_{1}$ here is equal to half the fraction called $\hat{r}_{1}(*)$ in [8]).

Consider transmission over the BEC($\epsilon$) and let $\epsilon_{(\ell,r,L)}$ denote the BP threshold of the finite-size ensemble. We write the first-order Taylor expansion of \hat{r}_{1}\big{|}_{\epsilon} around $\epsilon_{(\ell,r,L)}>\epsilon$ as \hat{r}_{1}\big{|}_{\epsilon}\approx\hat{r}_{1}\big{|}_{\epsilon_{(\ell,r,L)}}+\gamma\,\Delta\epsilon where $\Delta\epsilon=\epsilon_{(\ell,r,L)}-\epsilon$. Thus, since \hat{r}_{1}\big{|}_{\epsilon_{(\ell,r,L)}}=0 (by definition), then $\gamma\approx\hat{r}_{1}|_{\epsilon}/\Delta\epsilon$. This parameter $\gamma$ enters in the scaling law and is determined experimentally. Obviously it would be desirable to have a theoretical handle on $\gamma$. It is argued in [8] that $\gamma\approx\bar{\gamma}$ where $\bar{\gamma}=x_{\text{\tiny BP}}\,v/\Delta\epsilon$ and where $v$ is the velocity of the decoding wave. We find that for the $(3,6)$ ensemble, $\gamma=2.155$, $\bar{\gamma}=1.960$; for the $(5,10)$ ensemble, $\gamma=2.095$, $\bar{\gamma}=1.733$; for the $(4,12)$ ensemble, $\gamma=2.140$, $\bar{\gamma}=1.778$, to list a few examples. The differences might mostly be related to the difference in ensembles considered here and in [8], and also to the relatively large value of $\Delta\epsilon=0.04$ (chosen in [8] due to stability issues in numerical simulations).

Acknowledgment

R. E. thanks Andrei Giurgiu for finding the bug in her code. We also thank Rüdiger Urbanke for discussions.