The Velocity of the Propagating Wave for General Coupled Scalar Systems

TL;DR

This paper derives a formula for the propagation velocity of wave-like profiles in spatially coupled scalar systems, with applications to coding and compressive sensing, enhancing understanding of message-passing algorithms.

Contribution

It introduces a novel formula for wave velocity in coupled scalar systems, applicable to various algorithms like LDPC decoding and compressive sensing.

Findings

Derived a general velocity formula for solitonic wave profiles

Validated the formula with applications to LDPC codes and compressive sensing

Provided insights into the dynamics of message-passing algorithms

Abstract

We consider spatially coupled systems governed by a set of scalar density evolution equations. Such equations track the behavior of message-passing algorithms used, for example, in coding, sparse sensing, or constraint-satisfaction problems. Assuming that the "profile" describing the average state of the algorithm exhibits a solitonic wave-like behavior after initial transient iterations, we derive a formula for the propagation velocity of the wave. We illustrate the formula with two applications, namely Generalized LDPC codes and compressive sensing.

Equations

$$x^{(t+1)}=f(g(x^{(t)};ϵ);ϵ),$$

$${ϵ}_s\triangleq \sup \{ϵ|x^{(\infty )}=x_{\text{good}}\}.$$

$$U_s(x;ϵ)=xg(x;ϵ)-G(x;ϵ)-F(g(x;ϵ);ϵ),$$

$$\displaystyle x_{i}^{(t+1)}=\sum\limits_{j=-L}^{L}A_{j,i}\,f\Big{(}\sum\limits_{k=-L}^{L}A_{j,k}\,g(x_{k}^{(t)};\epsilon);\epsilon\Big{)},$$

$$U_c(\mathbf{x};ϵ)=\sum _{i=-L+W}^{L-W}$$

$$\displaystyle-\sum_{i=-L+W}^{L-W}F\Big{(}\sum_{j=-L}^{L}A_{i,j}g(x_{j};\epsilon);\epsilon\Big{)},$$

$$\displaystyle\Delta\mathcal{W}[x(\cdot,\cdot);\epsilon]\triangleq\int_{\mathbb{R}}\mathrm{d}z\,\Big{\{}x(z,t)g(x(z,t);\epsilon)-x_{0}(z)g(x_{0}(z);\epsilon)$$

$$\displaystyle\qquad-G(x(z,t);\epsilon)-F\Big{(}\int_{0}^{1}\mathrm{d}u\,w(u)\,g(x(z-u,t);\epsilon);\epsilon\Big{)}$$

$$\displaystyle\qquad+G(x_{0}(z);\epsilon)+F\Big{(}\int_{0}^{1}\mathrm{d}u\,w(u)\,g(x_{0}(z-u);\epsilon);\epsilon\Big{)}\Big{\}}.$$

$$\displaystyle\mathcal{W}_{s}[x(\cdot,\cdot);\epsilon]\triangleq\int_{\mathbb{R}}\mathrm{d}z\,\Big{\{}$$

$$-G(x(z,t);ϵ)-F(g(x(z,t);ϵ);ϵ)$$

$$\displaystyle+G(x_{0}(z);\epsilon)-F(g(x_{0}(z);\epsilon);\epsilon)\Big{\}},$$

$$\displaystyle\mathcal{W}_{i}[x(\cdot,\cdot);\epsilon]\triangleq\int_{\mathbb{R}}\mathrm{d}z\,\Big{\{}$$

$$\displaystyle-F\Big{(}\int_{0}^{1}\mathrm{d}u\,w(u)g(x(z-u,t);\epsilon);\epsilon\Big{)}$$

$$\displaystyle+F\Big{(}\int_{0}^{1}\mathrm{d}u\,w(u)\,g(x_{0}(z-u);\epsilon);\epsilon\Big{)}\Big{\}}.$$

$$v=\frac{U_s(x_{\text{bad}};ϵ)-U_s(x_{\text{good}};ϵ)}{{\int }_{\mathbb{R}}\mathrm{d}z{g}^{\prime }(X(z);ϵ)(X^{\prime }(z){)}^2},$$

$$\frac{\delta \Delta \mathcal{W}[x(\cdot ,\cdot );ϵ]}{\delta x(\cdot ,\cdot )}[\eta (z,t)]$$

$$\displaystyle=\lim_{\gamma\to 0}\frac{1}{\gamma}\Big{\{}\Delta\mathcal{W}[x(\cdot,\cdot)+\gamma\eta(\cdot,\cdot);\epsilon]-\Delta\mathcal{W}[x(\cdot,\cdot);\epsilon]\Big{\}}$$

$$\displaystyle=\int_{\mathbb{R}}\mathrm{d}z\,\eta(z,t)g^{\prime}(x(z,t);\epsilon)\Big{\{}x(z,t)$$

$$\displaystyle\qquad\quad-\int_{0}^{1}{d}u\,w(u)f(\int_{0}^{1}{d}s\,w(s)g(x(z-u+s,t);\epsilon);\epsilon)\Big{\}}$$

$$\displaystyle=-\int_{\mathbb{R}}\mathrm{d}z\,\eta(z,t)g^{\prime}(x(z,t);\epsilon)\big{(}x(z,t+1)-x(z,t)\big{)}$$

$$\frac{\delta \Delta \mathcal{W}[x(\cdot ,\cdot );ϵ]}{\delta x(\cdot ,\cdot )}$$

$$\frac{\delta {\mathcal{W}}_s[X(\cdot );ϵ]}{\delta X(\cdot )}[X^{\prime }(z)]+\frac{\delta {\mathcal{W}}_i[X(\cdot );ϵ]}{\delta X(\cdot )}[X^{\prime }(z)]$$

$$\frac{\delta {\mathcal{W}}_s[X(\cdot );ϵ]}{\delta X(\cdot )}[X^{\prime }(z)]$$

$$\displaystyle=\int_{\mathbb{R}}\mathrm{d}z\,X^{\prime}(z)\big{(}X(z)g^{\prime}(X(z);\epsilon)-g^{\prime}(X(z);\epsilon)f(g(X(z);\epsilon);\epsilon)\big{)}$$

$$\displaystyle=\int_{\mathbb{R}}\mathrm{d}z\,\frac{\mathrm{d}}{\mathrm{d}z}\big{\{}X(z)g(X(z);\epsilon)-G(X(z);\epsilon)-F(g(X(z);\epsilon);\epsilon)\big{\}}$$

$$\displaystyle=\Bigl{[}U_{s}(X(z);\epsilon)\Bigr{]}_{-\infty}^{+\infty}=U_{s}(x_{\text{\tiny bad}};\epsilon)-U_{s}(x_{\text{\tiny good}};\epsilon).$$

$$\frac{\delta {\mathcal{W}}_i[X(\cdot );ϵ]}{\delta X(\cdot )}[X^{\prime }(z)]$$

$$\displaystyle=\int_{\mathbb{R}}\mathrm{d}z\frac{\mathrm{d}}{\mathrm{d}z}\Big{\{}F(g(X(z);\epsilon);\epsilon)$$

$$\displaystyle\qquad\qquad\qquad-F\big{(}\int_{0}^{1}\mathrm{d}u\,w(u)g(X(z-u);\epsilon);\epsilon\big{)}\Big{\}}$$

$$\displaystyle=\Bigl{[}F(g(X(z);\epsilon);\epsilon)-F\big{(}\int_{0}^{1}\mathrm{d}uw(u)g(X(z-u);\epsilon);\epsilon\big{)}\Bigr{]}_{-\infty}^{+\infty}=0$$

$$f(x;ϵ)$$

$$g(x,e,n;ϵ)$$

$$U_{\text{GLDPC}}(x,e,n;ϵ)=\frac{e}{n}g(x,e,n;ϵ)-$$

$$-\frac{ϵ}{2}{g}^2(x,e,n;ϵ).$$

$$f(x,\mathtt{snr};\delta )$$

$$g(x,\mathtt{snr};\delta )$$

$$U_{\text{SC}}(x;\delta )=-$$

$$\displaystyle-2I\Big{(}S;\sqrt{\mathtt{snr}}S+Z\Big{)}+2I\Big{(}S;\sqrt{\frac{1}{\frac{1}{\mathtt{snr}}+\frac{x}{\delta}}}S+Z\Big{)}.$$

$$q_0(y)=(1-\rho )\delta (y)+\rho {\Phi }_0(y),$$

Full text

I Introduction

Spatial coupling is a graph construction that was first introduced for coding on Low-Density Parity-Check (LDPC) codes by Felstrom and Zigangirov [1]. Spatially coupled systems have been shown to exhibit excellent performance under low complexity message-passing algorithms. Due to this attractive property, they have been extensively studied in different frameworks, such as coding [2], [3] (a review with applications in the broader context of communications is found in [3]), compressive sensing [4], [5], [6], statistical physics and constraint-satisfaction [7], [8], [9], [10].

To assess the performance of a message-passing algorithm one analyzes the evolution of the messages exchanged during the algorithm as the number of iterations increases. This can be expressed as a set of scalar recursive equations (see Equ. (1) for uncoupled systems and Equ. (3) for coupled systems). In coding these are the density evolution equations, and in compressive sensing these are the state evolution equations.

A spatially coupled system is obtained from the underlying “single” (uncoupled) one by taking $2L+1$ copies of it and connecting every $W$ consecutive single systems by means of a “coupling window”. For scalar systems, the average behavior of the system at position $i\in\{-L,\dots,L\}$ of the coupling axis and at iteration $t\in\mathbb{N}$ of the message-passing algorithm is described by a single scalar $x_{i}^{(t)}$. Therefore, the evolution of the coupled system can be analyzed by tracking the vector $\mathbf{x}^{(t)}=\{x_{-L}^{(t)},\dots,x_{L}^{(t)}\}$, which we call the “profile”, as $t$ increases.

Under certain initial conditions and after an initial number of iterations, the profile demonstrates a solitonic behavior. That is, after a transient phase, it appears to develop a fixed shape that is independent of the initial condition and travels at a constant velocity as $t$ increases. In fact, it has been proved in [11] that a solitonic wave solution exists in the context of coding when transmission takes place over the Binary Erasure Channel (BEC). However, the question of the independence of the shape from the initial conditions is left open. The soliton is illustrated in Fig. 1 for a Generalized LDPC (GLDPC) code (see Sec. IV-A for details).

In [11] and [12], bounds on the velocity of the wave for coding on the BEC are proposed. In [13] a formula for the velocity of the wave in the context of the coupled Curie-Weiss toy model is derived and tested numerically. In [14], a formula in the context of coding on general binary input memoryless symmetric channels is derived.

In this work we derive a formula for the velocity of the wave in the continuum limit $L\gg w\gg 1$ for general scalar systems. By means of numerical simulations, we find that our formula is a very good estimate for the empirical velocity. We limit ourselves to the cases where the scalar recursive DE equation of the underlying uncoupled system has exactly two stable and one unstable fixed points. Equivalently the potential function (of the uncoupled system) has two minima and one local maximum. Fig. 1 illustrates this setting for the GLDPC example.

II Preliminaries

II-A Density Evolution and Potential Functions

We adopt the framework and notations of [15]. Let $\mathcal{E}=[0,\epsilon_{\text{\tiny max}}]$, where $\epsilon_{\text{\tiny max}}\in(0,\infty)$, denote the space of parameters, and let $\mathcal{X}=[0,x_{\text{\tiny max}}(\epsilon)]$ and $\mathcal{Y}=[0,y_{\text{\tiny max}}(\epsilon)]$, such that $x_{\text{\tiny max}}(\epsilon)$, $y_{\text{\tiny max}}(\epsilon)\in(0,\infty)$ and $y_{\text{\tiny max}}(\epsilon)=g(x_{\text{\tiny max}}(\epsilon);\epsilon)$. Consider bounded and smooth functions $f:\mathcal{Y}\times\mathcal{E}\to\mathcal{X}$ and $g:\mathcal{X}\times\mathcal{E}\to\mathcal{Y}$, increasing in both arguments. We consider the following (uncoupled) recursion,

[TABLE]

where $t\in\mathbb{N}$ denotes the iteration number. The recursion is initialized with $x^{(0)}=x_{\text{\tiny max}}$. Since $f(g(\mathcal{X}))\subset\mathcal{X}$, the initialization of the recursion (1) implies that $x^{(1)}\leq x^{(0)}=x_{\text{\tiny max}}$. Moreover, since the functions $f$ and $g$ are monotonic and bounded, the recursion will converge to a limiting value $x^{(\infty)}$ when the number of iterations is large, and this limit is a fixed point since $f$ and $g$ are continuous.

The typical picture in the context of applications such as coding or compressive sensing is as follows. It may help to think as $\epsilon$ as the level of noise for coding, as the inverse fraction of measurements in compressed sensing, or as the density of constraints in constraint-satisfaction problems.

We define $x_{\text{\tiny good}}$ as the fixed point of (1) obtained by the initialization $x^{(0)}=0$. We furthermore define the algorithmic threshold $\epsilon_{s}$ as

[TABLE]

Since $f$ and $g$ are monotonic, we can see that for $\epsilon<\epsilon_{s}$, the recursion (1) will converge to $x_{\text{\tiny good}}$ for any initialization of $x^{(0)}$ in $[0,x_{\text{\tiny max}}(\epsilon)]$. For $\epsilon>\epsilon_{s}$ new fixed points appear. We will limit ourselves to systems with one extra stable fixed point that we call $x_{\text{\tiny bad}}$ such that $x_{\text{\tiny bad}}>x_{\text{\tiny good}}$. For $\epsilon>\epsilon_{s}$ the iterations initialized at $x_{\text{\tiny max}}$ converge to $x_{\text{\tiny bad}}$. (It is easy to see that there also must exist an extra unstable fixed point $x_{\text{\tiny unst}}$ between these two, i.e., $x_{\text{\tiny good}}<x_{\text{\tiny unst}}<x_{\text{\tiny bad}}$.)

One can equivalently describe the (uncoupled) system by a potential function $U_{s}$ defined as

[TABLE]

where $F(x;\epsilon)=\int_{0}^{x}\mathrm{d}s\,f(s;\epsilon)$ and $G(x;\epsilon)=\int_{0}^{x}\mathrm{d}s\,g(s;\epsilon)$. The fixed points of (1) can be obtained by setting the derivative with respect to $x$ of the potential $U_{s}(x;\epsilon)$ to zero. The stable fixed points correspond to minima of $U_{s}$ and the unstable one to a local maximum of $U_{s}$. An example of the potential for the GLDPC code (see Sec. IV-A) is shown in Fig. 2. There, $\epsilon>\epsilon_{s}$ and there are two minima corresponding to $x_{\text{\tiny good}}=0$ and $x_{\text{\tiny bad}}>0$ with one local maximum corresponding to $x_{\text{\tiny unst}}$. If we run density evolution, the iterations will get stuck at $x_{\text{\tiny bad}}>0$. In general, to check whether the analysis in this work applies to a certain application, one can plot its corresponding potential function and check that it indeed has 2 stable fixed points (minima) and 1 unstable fixed point (maximum).

To obtain the spatially coupled system associated to the uncoupled one described above, we start by defining the “coupling window” function that satisfies $w(z)>0$, when $0\leq z<1$, $w(z)=0$ otherwise, and $\int_{\mathbb{R}}\mathrm{d}z\,w(z)=1$. Then, we define the normalized function w_{W}(z)=w(z)/\Big{(}\frac{1}{W}\sum_{j=0}^{W-1}w(\frac{j}{W})\Big{)}. Remark that $\frac{1}{W}\sum_{j=0}^{W-1}w_{W}(\frac{j}{W})=1$ and that as $W\to\infty$, $w_{W}(z)\to w(z)/\int_{\mathbb{R}}\mathrm{d}z\,w(z)=w(z)$.

The coupled system is then obtained by taking $2L+1$ copies of the single system on the positions $i=\{-L,\dots,L\}$ and connecting them using the “coupling matrix” $A_{j,k}=\frac{1}{W}w_{W}(\frac{k-j}{W})$. The discrete “profile” $\mathbf{x}^{(t)}=\{x_{-L}^{(t)},\dots,x_{L}^{(t)}\}$ is then fixed at the boundaries as follows: $x_{i}^{(t)}=x_{\text{\tiny good}}$, for $i=\{-L,\dots,-L+W-1\}$ and all $t\in\mathbb{N}$, $x_{i}^{(t)}=x_{\text{\tiny bad}}$, for $i=\{L-W+1,\dots,L\}$ and all $t\in\mathbb{N}$. We run density evolution on the remainder of the chain. More specifically, for $i\in\{-L+W,\dots,L-W\}$, the coupled scalar recursion is

[TABLE]

The boundary condition here is well adapted to study the propagation of the wave after the transient phase is over. Indeed, simulations and heuristic arguments indicate that an initial profile that increases from a seeding value smaller or equal to $x_{\text{\tiny good}}$ (at the left boundary) to $x_{\text{\tiny max}}$ (on the right boundary) is attracted towards the class of profiles defined above.

The spatially coupled system can be described by a potential functional $U_{c}$ defined as

[TABLE]

where $\mathbf{x}=\{x_{-L},\dots,x_{L}\}$. The fixed point form of Equ. (3) can be obtained by setting the derivative with respect to $\mathbf{x}$ of the potential $U_{c}(\mathbf{x};\epsilon)$ to zero.

A highly attractive property of spatially coupled systems is that they exhibit the so-called threshold saturation phenomenon. That is, for all $\epsilon<\epsilon_{c}$ where $\epsilon_{c}>\epsilon_{s}$, the coupled recursions (3) drive the profile $\underline{x}^{(t)}=[x_{-w+1}^{(t)},\dots,x_{L}^{(t)}]$ to the desirable fixed point $\underline{x}^{(\infty)}=[x_{\text{\tiny good}},\dots,x_{\text{\tiny good}}]$. Here $\epsilon_{c}$ is a threshold defined by $U(x_{\text{\tiny good}};\epsilon_{c})=U(x_{\text{\tiny bad}};\epsilon_{c})$ and often called the potential threshold (note that in this equation $x_{\text{\tiny good}}$ and $x_{\text{\tiny bad}}$ themselves depend on $\epsilon_{c}$).

In the sequel we consider the range $\epsilon\in[\epsilon_{s},\epsilon_{c}]$. It is for these values of the parameter $\epsilon$ that a soliton is observed. Let us repeat our basic assumption here: the recursion (1) has exactly two stable fixed points $x_{\text{\tiny good}}$ and $x_{\text{\tiny bad}}$. With more than two stable fixed points the propagating wave has a more complicated structure and our formulas would have to be adapted accordingly (see [12] for a nice discussion of this issue in coding).

II-B Continuum Limit

We consider the system in the continuum limit, which is obtained by first taking $L\to\infty$ and then $W\to\infty$ [11], [16], [17]. We set $\mathtt{x}(\frac{i}{W},t)\equiv\mathtt{x}_{i}^{(t)}$ and replace $\frac{i}{W}\to z$, $\frac{j}{W}\to u$, $\frac{k}{W}\to s$, where $z$, $u$, $s\in\mathbb{R}$ are continuous spatial variables. The coupled recursion, in the continuum limit, can then be written as

[TABLE]

The boundary conditions on the continuous profile $x(\cdot,\cdot)$ become $x(z,t)\to x_{\text{\tiny good}}$ when $z\to-\infty$ and $x(z,t)\to x_{\text{\tiny bad}}$ when $z\to+\infty$. Again, this boundary condition captures the profiles obtained after the transient phase has passed, and is well adapted to the study of the wave propagation.

Let $x_{0}(z)$ be a static (time independent) profile that satisfies the boundary conditions $x_{0}(z)\to x_{\text{\tiny good}}$ when $z\to-\infty$ and $x_{0}(z)\to x_{\text{\tiny bad}}$ when $z\to+\infty$. This profile can be thought as an initial condition for the recursions. For us however it serves as a reference profile in order to define the potential functional in the continuum limit. We look at the continuous version of $U(\mathbf{x};\epsilon)$ which we call $\mathcal{W}[x(\cdot,\cdot);\epsilon]$ and subtract from it $\mathcal{W}[x_{0}(\cdot);\epsilon]$ so that the integrals converge. The potential functional $\Delta\mathcal{W}[x(\cdot,\cdot);\epsilon]$ in the continuum limit is thus defined as

[TABLE]

As long as $x_{0}(z)$ converges to its limiting values fast enough the integrals over the spatial axis are well defined. We can further split the continuous potential functional into two parts: the single potential $\mathcal{W}_{s}[x(\cdot,\cdot);\epsilon]$ (that we obtain setting $w(z)\to 0$) and the interaction potential $\mathcal{W}_{i}[x(\cdot,\cdot);\epsilon]$ (that is caused only by coupling), defined as follows

[TABLE]

III Velocity for General Scalar Systems

III-A Statement of main result

We assume that after a number of iterations, which we call the transient phase, the density profile $x(\cdot,\cdot)$ develops a fixed shape which we call $X(\cdot)$ that moves with constant velocity $v$. That is, we make the ansatz $x(z,t)=X(z-vt)$.

Then, we find the following formula for the velocity $v$

[TABLE]

where $g^{\prime}=\partial_{x}g$ is the derivative of $g$ with respect to its first argument, and $X^{\prime}$ is the derivative of the profile.

III-B Derivation of main result

Evaluating the functional derivative of $\Delta\mathcal{W}[x(\cdot,\cdot);\epsilon]$ in an arbitrary direction $\eta(\cdot,\cdot)$, and then using (5) we obtain

[TABLE]

Using the ansatz $x(z,t)=X(z-vt)$ and the approximation $x(z,t+1)-x(z,t)\approx-vX^{\prime}(z)$, the functional derivative of $\Delta\mathcal{W}[x(\cdot,\cdot);\epsilon]$ in the special direction $\eta(z,t)=X^{\prime}(z)$ becomes

[TABLE]

To obtain our formula (6) we separate the left-hand side of the equation above into two contributions

[TABLE]

and calculate each term separately. For the first (uncoupled) part we find

[TABLE]

For the second (interaction) part we find

[TABLE]

IV Applications

The general formula for the velocity of the soliton for scalar systems (6) can be applied on several examples. In particular we recover the results of [14] for standard LDPC codes over the BEC, as well as on general Binary Memoryless Symmetric (BMS) channels within the scalar Gaussian approximation. In this section we provide two more scalar applications, namely GLDPC codes and compressive sensing.

The predictions of our formula are compared with the observed, empirical velocity $v_{e}$ that is obtained by running the scalar recursions. To obtain $v_{e}$, we plot the discrete profile $\mathbf{x}$ at different iterations of the scalar recursions and find the average of $\Delta z/(W\Delta I)$, where $\Delta z$ is the spatial difference between the kinks of the profiles, $\Delta I$ is the difference in the number of iterations, and $W$ is the size of the coupling window.

IV-A Generalized LDPC Codes

We consider a GLDPC code described as follows: The variable (bit) nodes have degree 2 and the check nodes have degree $n$. The rules of the check nodes are given by a primitive BCH code of blocklength $n$ (unlike LDPC codes where they are parity checks). An attractive property of BCH codes is that they can be designed to correct a chosen number of errors. For instance, one can design a BCH code so that it corrects all patterns of at most $e$ erasures on the BEC, and all error patterns of weight at most $e$ on the Binary Symmetric Channel (BSC).

We consider transmission on the BEC or BSC and denote by $\epsilon$ the channel parameter. The density evolution recursions have been derived for both channels, based on a bounded distance decoder for the BCH code [18]. We have $\epsilon_{\text{\tiny max}}=x_{\text{\tiny max}}=y_{\text{\tiny max}}=1$, and for $n$ and $e$ fixed [15],

[TABLE]

The formula for the velocity of the soliton appearing in the case of the GLDPC codes is found from (6), where the single potential of the system $U_{\text{\tiny GLDPC}}(\cdot)$ is given by

[TABLE]

Figure 3 shows the velocities (normalized by $W$) for the spatially coupled GLPDC code with $n=15$ and $e=3$, when the coupling parameters satisfy $L=250$ and $W=3$ and we use the uniform coupling window. We plot the velocities for $\epsilon\in[\epsilon_{s},\epsilon_{c}]=[0.348,0.394]$. We observe that the formula for the velocity provides a very good estimation of the empirical velocity $v_{e}$.

IV-B Compressive Sensing

Let $\mathbf{s}$ be a length-$n$ signal vector (where the components are i.i.d. copies of a random variable $S$) which is acquired through $m$ linear measurements. We assume that the measurement matrix has i.i.d Gaussian elements $\mathcal{(}0,1/\sqrt{n})$. We call $\delta=m/n$ the fixed measurement ratio when $n\to\infty$. The relation between $\delta$ and the generic $\epsilon$ used in this paper is $\epsilon=\delta^{-1}$. We assume that $\mathbb{E}[S^{2}]=1$ and that each component of $\mathbf{s}$ is corrupted by independent Gaussian noise of variance $\sigma^{2}=1/\mathtt{snr}$. To recover $\mathbf{s}$ one implements the so-called approximate message-passing (AMP) algorithm. It has been shown that the analysis of this algorithm is given by state evolution [4].

We consider the approach described in [15]. Consider the minimum mean-squared error (mmse) function given by $\mathtt{mmse}(\mathtt{snr})=\mathbb{E}_{S,Y}[(S-\hat{S}(Y,\mathtt{snr}))^{2}]$ where $\hat{S}(Y,\mathtt{snr})=\mathbb{E}_{S|Y}[S|Y]$ and $Y=\sqrt{\mathtt{snr}}S+Z$, $Z\sim\mathcal{N}(0,1)$. The state evolution equations, which track the mean squared error of the AMP estimate, then correspond to the recursion defined by

[TABLE]

where $\delta_{\text{\tiny max}}=1$, $x_{\text{\tiny max}}=\mathtt{mmse}(0)$, $y_{\text{\tiny max}}=g(x_{\text{\tiny max}})$.

The formula for the velocity $v_{\text{\tiny CS}}$ of the soliton in the case of compressive sensing can be obtained from (6) where the single potential $U_{\text{\tiny SC}}(\cdot)$ of the system is given by

[TABLE]

To check that this is indeed the correct potential we differentiate $U_{\text{\tiny SC}}(x;\epsilon)$ with respect to $x$ and use the relation [19] $\frac{1}{2}\mathtt{mmse}(\mathtt{snr})=\frac{\mathrm{d}}{\mathrm{d}\mathtt{snr}}I(S;\sqrt{\mathtt{snr}}S+Z)$, where $I(A;B)$ denotes the mutual information between the random variables $A$ and $B$ and is measured in nats.

For the compressive sensing scheme, we assume that the prior distribution on $S$ is the Bernoulli-Gaussian described by

[TABLE]

where $\Phi_{0}(y)=(1/\sqrt{2\pi})e^{-y^{2}/2}$. Figure 5 shows the velocities (normalized by $W$) for the spatially coupled compressive sensing scheme with $\mathtt{snr}=10^{5}$ and $\rho=0.1$, when the coupling parameters satisfy $L=250$ and $W=4$ and we use the uniform coupling window. We remark that for this application, the potential threshold $\delta_{c}$ is smaller than $\delta_{s}$ because the smaller the value of $\delta$, the less measurements of the signal we make, which induces more uncertainty. We plot on Fig. 5 the velocities for $\delta\in[\delta_{c},\delta_{s}]=[0.157,0.208]$. Similarly, we observe that the formula for the velocity provides a good estimation of the empirical velocity $v_{e}$.

Acknowledgments

R. E. thanks Tongxin Li for fruitful discussions and Jean Barbier for giving her part of his code on compressive sensing and for patiently explaining it to her. Part of this work was done during the authors’ stay at the Institut Henri Poincaré - Centre Émile Borel. The authors thank this institution for its hospitality and support.