The rank of sparse random matrices

TL;DR

This paper determines the rank of sparse random matrices over any field with fixed row and column sparsity, providing a formula for low-density parity check codes and confirming Lelarge's conjecture.

Contribution

It introduces a new method involving random perturbation to analyze the rank of sparse matrices, applicable across various fields.

Findings

Derived a formula for the rank of sparse random matrices

Confirmed Lelarge's conjecture on LDPC code rates

Developed a novel perturbation technique for matrix analysis

Abstract

We determine the rank of a random matrix over an arbitrary field with prescribed numbers of non-zero entries in each row and column. As an application we obtain a formula for the rate of low-density parity check codes. This formula vindicates a conjecture of Lelarge (2013). The proofs are based on coupling arguments and a novel random perturbation, applicable to any matrix, that diminishes the number of short linear relations.

Figures

Equations

$$\sum _{i=1}^n{\mathbf{d}}_i=\sum _{i=1}^{\mathbf{m}}{\mathbf{k}}_i,$$

$${\mathbf{A}}_{ij}$$

$$\Phi :[0,1]$$

$$\underset{n\to \infty }{\lim }\frac{\mathrm{rk}(\mathbf{A})}{n}$$

$$\frac{1}{n}\mathrm{nul}\mathbf{A}\to \underset{\alpha \in [0,1]}{\max }\Phi (\alpha )$$

$$\varphi (\alpha )=1-\alpha -D^{\prime }\left(1-K^{\prime }(\alpha )/k\right)/d.$$

$$\rho =\max \{x\in [0,1]:\varphi (x)=0\}.$$

$$\underset{n\to \infty }{\lim }\frac{{\mathbf{n}}^{\ast }}{n}$$

$$\mathrm{rk}(\mathbf{A})/n\le 1-\max \{\Phi (0),\Phi (\rho )\}+o(1)\text{mbox}a.a.s.$$

$$\frac{\mathrm{rk}(\mathbf{A})}{n}\le 1-D\left(1-\frac{K^{\prime }(\rho )}{k}\right)+\frac{d}{k}(1-K(\rho ))-\frac{K^{\prime }(\rho )}{k}{D}^{\prime }\left(1-\frac{K^{\prime }(\rho )}{k}\right).$$

$$\mathrm{rk}(\mathbf{A})/n$$

$$\underset{n\to \infty }{\lim }\mathrm{rk}(\mathbf{A})/n=1-\max \{\Phi (0),\Phi (\rho )\}\text{mbox}inprobability.$$

$$\underset{n\to \infty }{\lim }\frac{\mathrm{rk}(A(\mathbb{G}(n,n,p)))}{n}$$

$$\underset{n\to \infty }{\lim }\frac{\mathrm{rk}\mathbf{A}}{n}$$

$$\Phi (x)=$$

$$-\frac{99}{5}{x}^6+\frac{187}{25}{x}^4-\frac{154}{75}{x}^3-\frac{2}{75}.$$

$$\mathbb{P}\left[\hat{\mathbf{d}}=\ell \right]$$

$$f(\epsilon ,n)$$

$$\left(\begin{array}{cccc}1& 0& 1& 1\\ 1& 1& 0& 1\\ 1& 1& 1& 0\end{array}\right).$$

$$0=(111)\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)=(111)\left[\left(\begin{array}{cccc}1& 0& 1& 1\\ 1& 1& 0& 1\\ 1& 1& 1& 0\end{array}\right)\xi \right]=\left[(111)\left(\begin{array}{cccc}1& 0& 1& 1\\ 1& 1& 0& 1\\ 1& 1& 1& 0\end{array}\right)\right]\xi =(100)\xi ={\xi }_1.$$

$$\displaystyle{\mathbb{P}}\left[{\mbox{$A[\boldsymbol{\theta}]$ is $(\delta,\ell)$-free}}\right]>1-\delta.$$

$$\mathrm{nul}\left(\begin{array}{cc}A& 0\\ B& C\end{array}\right)-\mathrm{nul}A=n^{\prime }-\mathrm{rk}(B_{\ast }C).$$

$$\underset{n\to \infty }{\mathrm{limsup}}\frac{1}{n}\mathbb{E}[\mathrm{nul}({\mathbf{A}}_n)]$$

$$\underset{N\to \infty }{\mathrm{limsup}}\frac{1}{N}\sum _{n=1}^{N-1}\mathbb{E}[\mathrm{nul}({\mathbf{A}}_{n+1})]-\mathbb{E}[\mathrm{nul}({\mathbf{A}}_n)]$$

$$\underset{n\to \infty }{\mathrm{limsup}}\frac{1}{n}\mathbb{E}[\mathrm{nul}({\mathbf{A}}_n)]\le \underset{n\to \infty }{\mathrm{limsup}}\mathbb{E}[\mathrm{nul}({\mathbf{A}}_{n+1})]-\mathbb{E}[\mathrm{nul}({\mathbf{A}}_n)].$$

$$\mathbb{E}[\mathrm{nul}({\mathbf{A}}_{n+1})]-\mathbb{E}[\mathrm{nul}({\mathbf{A}}_n)]=\mathbb{E}\left[\mathrm{nul}({\mathbf{A}}_{n+1})-\mathrm{nul}({\mathbf{A}}_n)\right].$$

$${\mathbf{m}}_{\epsilon ,n}$$

$$\bigcup _{i=1}^{{\mathbf{m}}_{\epsilon ,n}}\left\{a_i\right\}\times [{\mathbf{k}}_i]\text{mbox}and\bigcup _{j=1}^n\left\{x_j\right\}\times [{\mathbf{d}}_j].$$

$$({\mathbf{A}}_{\epsilon ,n}{)}_{p_i,x_j}$$

Full text

Coja-Oghlan supported by DFG CO 646/3 and 646/4. Ergür’s research was partially supported by Einstein Foundation, Berlin and NSF CCF 2110075. Gao’s research is supported by ARC DE170100716 and ARC DP160100835. This submission combines the preprints arXiv:1810.07390 and arXiv:1906.05757. An extended abstract of this work appeared in the proceedings of the 31st ACM-SIAM Symposium on Discrete Algorithms (2020) 579–591.

1. Introduction

1.1. Background and motivation

The theory of random matrices, which commenced with the nuclear physics-inspired work of Wigner in the 1950s [88], has been one of the great success stories at the junction of probability, mathematical physics and combinatorics. Nevertheless, quite a few basic questions remain open to this day. For instance, while dense random matrices such as the Gaussian Orthogonal Ensemble are reasonably well understood (e.g., [72]), far less is known about sparse random matrices where the expected number of non-zero entries per row or column is bounded. Yet over the last two or three decades such sparse random matrices, with entries from finite or infinite fields, have emerged to play a pivotal role in several exciting applications. Modern error-correcting codes are a case in point. For instance, the codebook of a low-density parity check code (‘ldpc code’), a class of codes that has been at the centre of tremendous recent developments in coding theory [43, 62, 83], comprises the kernel of a sparse random matrix over a finite field drawn from a carefully tailored distribution. In addition, sparse random matrices occur in randomised constructions of Ramanujan graphs [6, 17, 39], statistical inference [61], the analysis of algorithms [32] and the theory of random constraint satisfaction problems [1, 47].

Among the fundamental questions about such random matrices that have remained open, perhaps the most conspicuous one concerns the rank. Although this parameter was already studied in early contributions [8, 9, 59], there has been no comprehensive rank formula for sparse random matrices. The present paper furnishes one. To be precise, we will determine the asymptotic rank of a sparse random matrix with prescribed numbers of non-zero entries in the rows and columns. Among other applications, important classes of ldpc codes are based on precisely such random matrices as a diligent choice of the degrees greatly boosts the code’s performance [83]. Moreover, the rank is linearly related to the rate of the code, arguably the code’s most basic parameter.

Lelarge [63] noticed that an upper bound on the rank of a sparse random matrix can be derived from the matching number of random bipartite graphs, which was determined by Bordenave, Lelarge and Salez [16]. Lelarge went on to conjecture that this bound be tight for sparse random matrices over the binary field $\mathbb{F}_{2}$. We prove this conjecture. In fact, we prove a much stronger result. Namely, we show that Lelarge’s conjectured formula holds for sparse random matrices over any field, finite or infinite, regardless the distribution of the non-zero matrix entries. Thus, the rank is governed by the location of the non-zero entries rather than the distribution of the matrix entries.

The proof of the rank formula evinces an interesting connection to statistical physics. Indeed, Lelarge already observed that a sophisticated but mathematically non-rigorous physics approach called the ‘cavity method’ renders a wrong prediction as to the rank for certain degree distributions.111The derivation of this erroneous prediction was posed as an exercise in [73, Chapter 19]. This discrepancy merits attention because the cavity method has been brought to bear on a panoply of theoretical as well as real-world problems, ranging from spin glasses to machine learning [90]. We manage to shed light on the issue. Specifically, the ‘replica symmetric’ version of the cavity method predicts that the rank of a random matrix over a finite field can be expressed analytically as the maximum of a variational problem. A priori, this variational problem asks to optimise a functional called the Bethe free entropy over an infinite-dimensional space of probability measures. Such optimisation problems have been tackled in the physics literature numerically by means of a heuristic called population dynamics. For the rank problem this was carried out by Alamino and Saad [3]. But thanks to the algebraic nature of the problem we can show that the rank actually comes out as the solution to a variational problem on a restricted domain. We are thus left with a dramatically simplified variational problem, which ultimately boils down to a humble one-dimensional optimisation task. We will see that the optimal solution to this one-dimensional problem does indeed yield the rank (over any field). Furthermore, the solution can be lifted to a solution to the original infinite-dimensional problem. As an aside, we do not know if the original infinite-dimensional variational problem may possess spurious maximisers that boost its value beyond the optimal value of the restricted version, thereby spoiling the accuracy of the original physics formula. We will return to this question, and to the physics slant on the problem, in Section 2.3.3. In any case, for certain degree distributions the maximum values that we obtain by way of the restricted variational problem actually exceed those that surfaced in the experiments from [3] or the heuristic derivations from [73] for the unrestricted formula; hence the discrepancy between the physics predictions and mathematical reality.

Apart from remedying the discrepancy, we prove the rank formula by effectively turning the physicists’ cavity calculations into a rigorous mathematical argument. The crucial tool that makes this possible is a novel perturbation, applicable to any matrix, that diminishes the number of short linear relations (see Proposition 2.4 below). We expect that this perturbation will find future applications. Let us proceed to introduce the random matrix model and state the main results. A discussion of related work and a detailed comparison with the physics work follow in Section 2, once we have the necessary notation in place.

1.2. The rank formula

Let $\mathbb{F}$ be a field equipped with a $\sigma$-algebra that turns $\mathbb{F}$ into a standard Borel space and let $\chi:[0,1]^{2}\to\mathbb{F}^{*}=\mathbb{F}\setminus\left\{{0}\right\}$ be a measurable map. Let $(\boldsymbol{\zeta}_{i},\boldsymbol{\xi}_{i})_{i\geq 1}$ be mutually independent uniformly distributed $[0,1]$-valued random variables. Moreover, let $\boldsymbol{d},\boldsymbol{k}\geq 0$ be integer-valued random variables such that $0<\mathbb{E}[\boldsymbol{d}^{r}]+\mathbb{E}[\boldsymbol{k}^{r}]<\infty$ for a real $r>2$ and set $d=\mathbb{E}[\boldsymbol{d}]$, $k=\mathbb{E}[\boldsymbol{k}]$. Let $n>0$ be an integer divisible by the greatest common divisor of the support of $\boldsymbol{k}$ and let ${\boldsymbol{m}}\,\sim\,{\rm Po}(dn/k)$ be independent of the $\boldsymbol{\zeta}_{i},\boldsymbol{\xi}_{i}$. Further, let $(\boldsymbol{d}_{i},\boldsymbol{k}_{i})_{i\geq 1}$ be copies of $\boldsymbol{d},\boldsymbol{k}$, mutually independent and independent of ${\boldsymbol{m}},\boldsymbol{\zeta}_{i},\boldsymbol{\xi}_{i}$. Given

[TABLE]

draw a simple bipartite graph $\boldsymbol{G}$ comprising a set $\{a_{1},\ldots,a_{{\boldsymbol{m}}}\}$ of check nodes and a set $\{x_{1},\ldots,x_{n}\}$ of variable nodes such that the degree of $a_{i}$ equals $\boldsymbol{k}_{i}$ and the degree of $x_{j}$ equals $\boldsymbol{d}_{j}$ for all $i,j$ uniformly at random. Then let $\boldsymbol{A}$ be the ${\boldsymbol{m}}\times n$-matrix with entries

[TABLE]

Thus, the $i$-th row of $\boldsymbol{A}$ features precisely $\boldsymbol{k}_{i}$ non-zero entries and the $j$-th column contains precisely $\boldsymbol{d}_{j}$ non-zero entries. Moreover, the non-zero entries of $\boldsymbol{A}$ are drawn in the vein of an exchangeable array by evaluating the function $\chi$ at a random poisition $(\boldsymbol{\zeta}_{i},\boldsymbol{\xi}_{j})$. Routine arguments show that $\boldsymbol{A}$ is well-defined for large enough $n$, i.e., (1.1) is satisfied and there exists a simple $\boldsymbol{G}$ with the desired degrees with positive probability; see Proposition 1.10 below. We call $\boldsymbol{G}$ the Tanner graph of $\boldsymbol{A}$. Also recall that the rank $\operatorname{rk}A$ of the matrix $A$ is defined as the maximal number of linear independent rows (or columns). In addition, $\operatorname{nul}A$ is the dimension of the kernel of $A$ and the sum $\operatorname{rk}A+\operatorname{nul}A$ equals the number of columns of $A$.

The following theorem, the main result of the paper, provides an asymptotic formula for the rank of $\boldsymbol{A}$. Let $D(x)$ and $K(x)$ denote the probability generating functions of $\boldsymbol{d}$ and $\boldsymbol{k}$, respectively. Since $\mathbb{E}[\boldsymbol{d}^{2}]+\mathbb{E}[\boldsymbol{k}^{2}]<\infty$, the functions $D(x),K(x)$ are continuously differentiable on the unit interval. Therefore, the function

[TABLE]

is continuous.

Theorem 1.1**.**

For any $\boldsymbol{d},\boldsymbol{k}$ we have, uniformly for all $\chi$,

[TABLE]

Perhaps surprisingly, the r.h.s. of (1.3) depends only on the degree distributions $\boldsymbol{d},\boldsymbol{k}$ but not in any way on the field $\mathbb{F}$ or the choice of non-zero entries (within the aforementioned model). Furthermore, let us emphasise that the function $\Phi$, being continuous on the unit interval, is guaranteed to attain a maximum. However, this maximum need not be unique, and non-uniqueness of the maximiser may have interesting combinatorial repercussions [23].

A second point that may seem surprising at first glance is that the rank converges to any non-random value at all, as provided by (1.3). A heuristic explanation can be given on grounds of physics reasoning. Indeed, the nullity of $\boldsymbol{A}$ (dimension of the kernel) corresponds to the logarithm of the partition function of a natural Boltzmann distribution, namely the uniform distribution on the kernel of $\boldsymbol{A}$. Commonly the normalised logarithm of such a partition functions (known as the “free entropy” in physics jargon) converges to a constant for random systems that are “self-averaging”. Here “self-averaging” means that a small perturbation to the system, i.e., the matrix in our case, cannot cause disproportionate tremors in logarithm of the partition function. In the random matrix model that we consider here the self-averaging condition is clearly satisfied because changing a single matrix entry can at most alter the nullity by one. Therefore, the Azuma–Hoeffding inequality easily implies that $\operatorname{nul}\boldsymbol{A}$ concentrates about its mean. That said, there is no general theorem that guarantees convergence to a deterministic value in self-averaging systems, so even this aspect of Theorem 1.1 is not in any way a triviality.

Theorem 1.1 establishes a generalised version of Lelarge’s rank conjecture [63] with a tighter conditions on the moments of $\boldsymbol{d},\boldsymbol{k}$. Specifically, Lelarge only considered matrices over the field $\mathbb{F}_{2}$, while here we consider general fields and allow for a very general choice of non-zero entries. That said, while here we assume that $\mathbb{E}[\boldsymbol{d}^{r}],\mathbb{E}[\boldsymbol{k}^{r}]<\infty$ for a real $r>2$, Lelarge considered degree distributions with $\mathbb{E}[\boldsymbol{d}^{2}],\mathbb{E}[\boldsymbol{k}^{2}]<\infty$. We did not undertake a serious attempt to weaken the moment condition to $r=2$, but this may conceivably introduce significant new techical difficulties.

The theorem covers a very general class of sparse random matrices. Indeed, since $\boldsymbol{d},\boldsymbol{k}$ have finite means the matrix $\boldsymbol{A}$ is sparse, i.e., the expected number of non-zero entries is $O(n)$ as $n\to\infty$. Yet because the degree distributions are subject only to the condition $\mathbb{E}[\boldsymbol{d}^{r}]+\mathbb{E}[\boldsymbol{k}^{r}]<\infty$, the typical maximum number of non-zero entries per row or column may approach $\sqrt{n}$. Furthermore, the choice of the non-zero entries of the matrix by way of the measurable map $\chi$, reminiscent of an exchangeable array, allows for rather general choices of non-zero matrix entries. To elaborate, recall that an exchangeable array is an infinite matrix $(\boldsymbol{\chi}_{ij})_{i,j\geq 1}$ of $\mathbb{F}^{*}$-valued random variables such that the distribution of any finite top-left sub-matrix is invariant under row and column permutations [53]. The Aldous–Hoover representation theorem shows that any such array can be described by a function $\mathcal{X}:[0,1]^{4}\to\mathbb{F}^{*}$ [4, 46]. Specifically, any finite sub-matrix of $\boldsymbol{\chi}_{ij}$ can be obtained by substituting suitable independent random variables that are uniformly distributed on the unit interval $[0,1]$ into $\mathcal{X}$. Theorem 1.1 therefore implies the rank formula for a Hadamard product of the biadjacency matrix of the random bipartite graph $\boldsymbol{G}$ and the commensurately dimensioned top-left bit of the exchangeable array $(\boldsymbol{\chi}_{ij})_{i,j}$. Of course, an immediate special case is the random matrix whose non-zero entries are drawn mutually independently from an arbitrary distribution on $\mathbb{F}^{*}$.222To see this, assume that ${\boldsymbol{\chi}}$ is an $\mathbb{F}^{*}$-valued random variable. Then given $n$ pick a large integer $N\gg n^{2}$. Let $\chi:[0,1]^{2}\to\mathbb{F}^{*}$ be a step function obtained by chopping $[0,1]$ into $N$ sub-intervals of size $1/N$ and assigning a value drawn from ${\boldsymbol{\chi}}$ independently to each of the $N^{2}$ resulting rectangles. Because Theorem 1.1 provides uniform convergence in $\chi$, we obtain the rank of a matrix with non-zero entries drawn from ${\boldsymbol{\chi}}$.

The lower bound on the rank constitutes the principal contribution of Theorem 1.1. Indeed, the upper bound $\operatorname{rk}(\boldsymbol{A})/n\leq 1-\max_{\alpha\in[0,1]}\Phi(\alpha)+o(1)$ as $n\to\infty$ a.a.s. was already derived in [63] from the Leibniz determinant formula and the formula for the matching number of a random bipartite graph from [16].333While [63] only dealt with matrices over $\mathbb{F}_{2}$, the argument extends to other fields without further ado. Nonetheless, in the appendix we give an independent proof of the upper bound, which is shorter than the combination [16, 63].

Theorem 1.1 implies a formula for the rate of a common class of ldpc codes. Such codes are based on random matrices $\boldsymbol{A}$ over finite fields $\mathbb{F}_{q}$ with suitable degree distributions $\boldsymbol{d},\boldsymbol{k}$. Specifically, a common construction of ldpc codes involves an optimisation over the degree distributions $\boldsymbol{d},\boldsymbol{k}$ of the variables/checks so as to maximise the probability that the Belief Propagation message passing algorithm (or a variant thereof) recovers the original codeword from the received, noisy data [83]. The codebook consists of the kernel of the random matrix $\boldsymbol{A}$. Hence, the rate of the code equals $\operatorname{nul}\boldsymbol{A}/n$. Since Theorem 1.1 implies that

[TABLE]

we thus obtain the rate.

1.3. The 2-core bound

There is a simple graph-theoretic upper bound on the rank, and Theorem 1.1 puts us in a position to investigate if and when this bound is tight. To state this bound, we recall that the 2-core of $\boldsymbol{G}$ is the subgraph $\boldsymbol{G}_{*}$ obtained by repeating the following operation.

While there is a variable node $x_{i}$ of degree one or less, remove that variable node along with the adjacent check node (if any). 444Strictly speaking, what we describe here is the 2-core of the hypergraph whose vertices are the variable nodes and whose edges are the neighbourhoods of the check nodes.

Of course, the 2-core may be empty, i.e. with no variable or check nodes. In the case that ${\mathbb{P}}(\boldsymbol{k}=0)>0$ it is possible to have a 2-core without any variable node but with a non-empty set of check nodes whose degrees are all zero. Extending prior results that dealt with the degrees of all check nodes coinciding [26, 76], we compute the likely number of variable and check nodes in the 2-core. Let

[TABLE]

Note that $\Phi^{\prime}(\alpha)=dK^{\prime\prime}(\alpha)\phi(\alpha)/k$. Since $\boldsymbol{d},\boldsymbol{k}$ have finite second moments and $\phi(0)\geq 0$ while $\phi(1)\leq 0$, we can define

[TABLE]

Theorem 1.2**.**

Assume that $\phi^{\prime}(\rho)<0$ and let ${\boldsymbol{n}}^{*}$ and ${\boldsymbol{m}}^{*}$ be the number of variable and check nodes in the 2-core, respectively. Then

[TABLE]

Remark 1.3**.**

1. (a)

If  ${\mathbb{P}}(\boldsymbol{k}=1)=0$ then $1-D\left(1-\frac{K^{\prime}(0)}{k}\right)-\frac{K^{\prime}(0)}{k}D^{\prime}\left(1-\frac{K^{\prime}(0)}{k}\right)$ evaluates to zero, and a.a.s. $dK(0)/k$ is the number of check nodes with degree zero in $\boldsymbol{G}$ divided by $n$, up to an $o(1)$ error. 2. (b)

If ${\boldsymbol{d}}\leq 1$ then we observe that $\phi(\alpha)=-\alpha$ and thus $\rho=0$. In this case, $1-D\left(1-\frac{K^{\prime}(0)}{k}\right)-\frac{K^{\prime}(0)}{k}D^{\prime}\left(1-\frac{K^{\prime}(0)}{k}\right)$ evaluates to zero. This agrees with the trivial fact that ${\boldsymbol{n}}^{*}=0$, and $\frac{{\boldsymbol{m}}^{*}}{n}\to dK(0)/k$ a.a.s. in this case. 3. (c)

If  ${\mathbb{P}}(\boldsymbol{k}=1)>0$ and ${\mathbb{P}}({\boldsymbol{d}}\geq 2)>0$ then $\phi(0)>0$, which implies that $\rho>0$. Thus the right hand sides of (1.6) are both positive.

Theorem 1.2 yields an elementary upper bound on the rank of $\boldsymbol{A}$, as follows, which we refer to as the 2-core bound:

[TABLE]

To see that $\operatorname{rk}(\boldsymbol{A})/n\leq 1-\Phi(0)+o(1)$ a.a.s., let $\boldsymbol{A}^{\prime}$ be the matrix comprising the rows of $\boldsymbol{A}$ that contain at most one non-zero entry and let ${\boldsymbol{m}}^{\prime}$ be the number of such rows. Then $\operatorname{rk}(\boldsymbol{A})\leq{\boldsymbol{m}}-{\boldsymbol{m}}^{\prime}+\operatorname{rk}(\boldsymbol{A}^{\prime})$. Moreover, routine arguments reveal that $({\boldsymbol{m}}-{\boldsymbol{m}}^{\prime})/n\sim d(1-K(0)-K^{\prime}(0))/k$ and $\operatorname{rk}(\boldsymbol{A}^{\prime})/n\sim 1-D(1-K^{\prime}(0)/k)$ a.a.s. (see Appendix D for a proof), deducing the desired upper bound for $\operatorname{rk}(\boldsymbol{A})$.

The other upper bound in (1.7) can be deduced by considering the 2-core and lower bounding the nullity. Counting only solutions to $\boldsymbol{A}x=0$ where $x_{i}=0$ for all variables that belong to the 2-core $\boldsymbol{G}_{*}$, we obtain $\operatorname{nul}(\boldsymbol{A})\geq n-{\boldsymbol{n}}^{*}-({\boldsymbol{m}}-{\boldsymbol{m}}^{*})$. Invoking Theorem 1.2, we thus find that as $n\to\infty$,

[TABLE]

Now $\phi(\rho)=0$ implies $D^{\prime}\left(1-\frac{K^{\prime}(\rho)}{k}\right)=d(1-\rho)$. Substituting this into the inequality above yields

[TABLE]

The following theorem shows that the 2-core bound is tight in several cases of interest.

Theorem 1.4**.**

Assume that

1. (i)

either $\mathrm{Var}(\boldsymbol{d})=0$ or $\boldsymbol{d}\,\sim\,{\rm Po}_{\geq\ell}(\lambda)$ for an integer $\ell\geq 0$ and $\lambda>0$, and 2. (ii)

either $\mathrm{Var}(\boldsymbol{k})=0$ or $\boldsymbol{k}\,\sim\,{\rm Po}_{\geq\ell^{\prime}}(\lambda^{\prime})$ for an integer $\ell^{\prime}\geq 0$ and $\lambda^{\prime}>0$.

Then

[TABLE]

Remark 1.5**.**

Under either condition of Theorem 1.4 (i) or (ii), the condition $\phi^{\prime}(\rho)<0$ of Theorem 1.2 is satisfied, unless ${\mathbb{P}}({\boldsymbol{d}}=1)=0$ and $2(k-1){\mathbb{P}}({\boldsymbol{d}}=2)\geq d$. We will prove this in the proof of Theorem 1.4.

On the basis of a canny but non-rigorous statistical physics approach called the cavity method several authors predicted that (over finite fields) the 2-core bound (1.7) is universally tight for all $\boldsymbol{d},\boldsymbol{k}$. Alamino and Saad reached this conclusion by way of numerical experiments [3], while Mézard and Montanari [73] posed a non-rigorous but analytical derivation as an exercise. However, the prediction turns out to be erroneous. Indeed, Lelarge [63] produced an example of $\boldsymbol{d},\boldsymbol{k}$ whose function $\Phi(\alpha)$ attains its unique maximum at a value $0<\alpha<\rho$. We will see another counterexample momentarily. On the positive side, Theorem 1.4 verifies that the 2-core bound actually is tight in all the cases for which Alamino and Saad [3] conducted numerical experiments.

1.4. Examples

Let us conclude this section by investigating a few examples of degree distributions $\boldsymbol{d},\boldsymbol{k}$ and their resulting rank formulas.

Example 1.6** (the identity matrix).**

As was brought to our attention by an anonymous reviewer, in the case $\boldsymbol{d}=\boldsymbol{k}=1$ deterministically the matrix $\boldsymbol{A}$ is just a permutation matrix, which clearly has full rank. Accordingly, we find $D(x)=K(x)=x$ and $\Phi(x)=0$. Hence, (1.3) boils down to the trivial fact $\operatorname{rk}\boldsymbol{A}\sim n$.

Example 1.7** (the adjacency matrix of random bipartite graphs).**

Let $\mathbb{G}=\mathbb{G}(n,n,p)$ be a random bipartite graph on vertices $v_{1},\ldots,v_{n},v_{1}^{\prime},\ldots,v_{n}^{\prime}$ such that for any $i,j\in[n]$ the edge $\{v_{i},v_{j}^{\prime}\}$ is present with probability $p$ independently. With $p=\Delta/n$ for a fixed $\Delta>0$ for large $n$ the vertex degrees asymptotically have distribution ${\rm Po}(\Delta)$. Indeed, with the choice $\boldsymbol{d}\,\sim\,{\rm Po}(\Delta)$ and $\boldsymbol{k}\,\sim\,{\rm Po}(\Delta)$ the adjacency matrix $A(\mathbb{G}(n,n,p))$ and the random matrix $\boldsymbol{A}$ can be coupled such that $\operatorname{rk}{A(\mathbb{G}(n,n,p))}=\operatorname{rk}\left({\boldsymbol{A}}\right)+o(n)$ a.a.s. Hence, Theorem 1.1 shows that over any field $\mathbb{F}$,

[TABLE]

in probability. Theorem 1.4 implies that the 2-core bound is tight in this example.

Example 1.8** (fixed row sums).**

Motivated by the minimum spanning tree problem on weighted random graphs, Cooper, Frieze and Pegden [28] studied the rank of the random matrix with degree distributions $\boldsymbol{k}=k\geq 3$ fixed and $\boldsymbol{d}\,\sim\,{\rm Po}(d)$ over the field $\mathbb{F}_{2}$. The same rank formula was obtained independently in [7] for arbitrary finite fields. Extending both these results, Theorem 1.1 shows that the rank of the random matrix with these degrees over any field $\mathbb{F}$ with any choice $\chi$ of non-zero entries is given by

[TABLE]

Once more Theorem 1.4 shows that the 2-core bound is tight.

Example 1.9** (non-exact 2-core bound).**

There are plenty of choices of $\boldsymbol{d},\boldsymbol{k}$ where the 2-core bound fails to be tight. Degree distributions that render graphs $\boldsymbol{G}$ with an unstable 2-core furnish particularly egregious offenders. In such graphs the removal of a small number of randomly chosen checks $a_{i}$ likely causes the 2-core to collapse. Analytically, the instability manifests itself in $\rho$ from (1.5) being a local minimum of $\Phi(x)$. For instance, letting $\boldsymbol{d},\boldsymbol{k}$ be the distributions with $D(x)=(22x^{2}+3x^{11})/25$ and $K(x)=x^{3}$, we obtain

[TABLE]

Hence, $\rho=1$ and $\Phi^{\prime\prime}(1)>0$, while the global maximum is attained at $\alpha\approx 0.75$.

1.5. Preliminaries

Throughout the paper we consistently keep the assumptions on the distributions $\boldsymbol{d},\boldsymbol{k}$ listed in Section 1. In particular, $\mathbb{E}[\boldsymbol{d}^{r}]+\mathbb{E}[\boldsymbol{k}^{r}]<\infty$ for some real $r>2$. Because all-zero rows and columns do not add to the rank, we may assume that $\boldsymbol{d}\geq 1,\boldsymbol{k}\geq 1$. We write $\gcd(\boldsymbol{k})$ and $\gcd(\boldsymbol{d})$ for the greatest common divisor of the support of $\boldsymbol{d}$ and $\boldsymbol{k}$, respectively. When working with $\boldsymbol{A}$ we tacitly assume that $\gcd(\boldsymbol{k})$ divides $n$. In order to highlight the number of columns we write $\boldsymbol{A}_{n}=\boldsymbol{A}$ and $\boldsymbol{G}_{n}=\boldsymbol{G}$ for the corresponding Tanner graph. The following proposition, whose proof can be found in Section 4.2, shows that $\boldsymbol{A}_{n}$ is well-defined.

Proposition 1.10**.**

With probability $\Omega(n^{-1/2})$ over the choice of ${\boldsymbol{m}}$, $(\boldsymbol{d}_{i})_{i\geq 1}$, $(\boldsymbol{k}_{i})_{i\geq 1}$ the condition (1.1) is satisfied and there exists a simple Tanner graph $\boldsymbol{G}$ with variable degrees $\boldsymbol{d}_{1},\ldots,\boldsymbol{d}_{n}$ and check degrees $\boldsymbol{k}_{1},\ldots,\boldsymbol{k}_{{\boldsymbol{m}}}$.

We introduce the size-biased random variables

[TABLE]

Throughout the paper we let $(\boldsymbol{k}_{i},\boldsymbol{d}_{i},\hat{\boldsymbol{k}}_{i},\hat{\boldsymbol{d}}_{i})_{i\geq 1}$ denote mutually independent copies of $\boldsymbol{k},\boldsymbol{d},\hat{\boldsymbol{k}},\hat{\boldsymbol{d}}$. Unless specified otherwise, all these random variables are assumed to be independent of any other sources of randomness.

We use common notation for graphs and multi-graphs. For instance, for a vertex $v$ of a multi-graph $G$ we denote by $\partial_{G}v$ the set of neighbours of $v$. More generally, for an integer $\ell\geq 1$ we let $\partial_{G}^{\ell}v$ be the set of vertices at distance precisely $\ell$ from $v$. We omit the reference to $G$ where possible.

The proofs of the main results rely on taking a double limit where we first take the number $n$ of columns to infinity and subsequently send an error parameter $\varepsilon$ to zero. We use the asymptotic symbols with an index $n$ such as $O_{n}(\,\cdot\,)$, $o_{n}(\,\cdot\,)$ to refer to the inner limit $n\to\infty$ only. Thus, for functions $f(\varepsilon,n),g(\varepsilon,n)$ we write

[TABLE]

For example, $1/(\varepsilon n)=o_{n}(1)$. Additionally, we will use the symbols $O_{\varepsilon,n}$, $o_{\varepsilon,n}$, etc. to refer to the double limit $\varepsilon\to 0$ after $n\to\infty$. Thus,

[TABLE]

For instance, $\varepsilon+1/(\varepsilon n)=o_{\varepsilon,n}(1)$.

Finally, we need the following basic lemma on sums of independent random variables.

Lemma 1.11**.**

Let $r>2$, $\delta>0$ and suppose that $(\boldsymbol{\lambda}_{i})_{i\geq 1}$ are independent copies of a random variable $\boldsymbol{\lambda}\geq 0$ with $\mathbb{E}[\boldsymbol{\lambda}^{r}]<\infty$. Further, let $s=\Theta_{n}(n)$. Then ${\mathbb{P}}\left[{\left|{\sum_{i=1}^{s}(\boldsymbol{\lambda}_{i}-\mathbb{E}[\boldsymbol{\lambda}])}\right|>\delta n}\right]=o_{n}(1/n).$

For the sake of completeness the proof of Lemma 1.11 is included in the appendix.

2. Overview

We survey the proof of Theorem 1.1 and subsequently compare these techniques with those employed in prior work. The main contribution of the paper is the ‘$\geq$’-part of (1.3), i.e., the lower bound on the rank. We prove this lower bound via a technique inspired by the physicists’ cavity method. The scaffolding of the proof is provided by a coupling argument reminiscent of a proof strategy known in mathematical physics jargon under the name ‘Aizenman-Sims-Starr scheme’ [2] or ‘cavity ansatz’ [73]:

To calculate the mean of a random variable $X_{n}$ on a random system of size $n$ in the limit $n\to\infty$, calculate the difference $\mathbb{E}[X_{n+1}]-\mathbb{E}[X_{n}]$ upon going to a system of size $n+1$. Perform this calculation by coupling the systems of sizes $n$ and $n+1$ such that the latter results from the former by adding only a bounded number of elements.

We will apply this approach to $X_{n}=\operatorname{nul}\boldsymbol{A}_{n}$. The coupling will be such that $X_{n+1}$ is the nullity of a random matrix obtained from $\boldsymbol{A}_{n}$ obtained by adding a few rows and columns. Thus, we need to calculate the ensuing change in nullity upon adding to a matrix several rows/columns whose number is random and bounded in expectation.

In general, such a calculation hardly seems possible. To carry it out we would need to understand the linear dependencies among the coordinates where the new rows sport non-zero entries, an exceedingly complicated task. Two facts deliver us from this complexity. First, the positions of the non-zero entries of the new rows are (somewhat) random. Second, we develop a random perturbation, applicable to any matrix, that diminishes the number of short linear relations (Proposition 2.4 below). To be precise, we will conclude that by applying the perturbation, for any fixed $\ell$ the probability that a set of $\ell$ coordinates forms a proper relation in the sense of Definition 2.1 below can be made negligibly small without substantially altering the nullity. In effect, the probability that there will be linear dependencies among the positions of the non-zero entries of the new rows will turn out to be negligible. Since this perturbation argument is the linchpin of the entire proof, this is what we shall begin with. Subsequently we will explain how this general perturbation renders the desired lower bound on the rank.

2.1. Short linear relations

Define the support of a vector $\xi\in\mathbb{F}^{U}$ as ${\text{supp}}(\xi)=\left\{{i\in U:\xi_{i}\neq 0}\right\}$.

Definition 2.1**.**

Let $A$ be an $m\times n$-matrix over a field $\mathbb{F}$.

• •

A set $\emptyset\neq I\subseteq[n]$ is a relation of $A$ if there exists a row vector $y\in\mathbb{F}^{1\times m}$ such that $\emptyset\neq{\text{supp}}(yA)\subseteq I$.

• •

If $I=\left\{{i}\right\}$ is a relation of $A$, then we call $i$ frozen in $A$. Let $\mathfrak{F}(A)$ be the set of all frozen $i\in[n]$.

• •

A set $I\subseteq[n]$ is a proper relation of $A$ if $I\setminus\mathfrak{F}(A)$ is a relation of $A$.

• •

For $\delta>0$, $\ell\geq 1$ we say that $A$ is $(\delta,\ell)$-free if there are no more than $\delta n^{\ell}$ proper relations $I\subseteq[n]$ of size $|I|=\ell$.

Thus, if $I\subseteq[n]$ is a relation of $A$, then by adding up suitable multiples of the rows of the homogeneous linear system $Ax=0$ we can infer a non-trivial linear relation involving the variables $(x_{i})_{i\in I}$ only. In the simplest case the set $I=\{i\}$ may be a singleton. Then the equation $x_{i}=0$ is implicit in $Ax=0$ and we call coordinate $i$ frozen. In particular, $i$ is frozen if $A$ contains a row whose only non-zero entry appears in column $i$. However, this is not the only possibility. For instance, in the following $\mathbb{F}_{2}$-matrix variable $x_{1}$ is frozen because the sum of all three rows equals $(1\ 0\ 0)$:

[TABLE]

In effect, for any vector $\xi$ in the kernel of (2.1) we have

[TABLE]

Generally, a linear number $\Omega(n)$ of rows may have to collude to cause freezing. Moreover, although the proof is just a bit of routine linear algebra, it is worthwhile including the following characterisation of frozen coordinates.

Fact 2.2**.**

A coordinate $i$ is frozen in the matrix $A$ iff $\xi_{i}=0$ for all $\xi\in\ker A$.

Proof.

Let $A$ be an $m\times n$ matrix over an arbitrary field. The calculation from (2.2) readily generalises to arbitrary matrices and implies that $\xi_{i}=0$ for any frozen coordinate $i\in[n]$ and any $\xi\in\ker A$.

Conversely, assume that for coordinate $i\in[n]$ we have $\xi_{i}=0$ for all $\xi\in\ker A$. Let $e^{(i)}\in\mathbb{F}^{1\times n}$ be the vector whose $i$-th coordinate equals one and whose other coordinates are equal to zero. Moreover, obtain $A^{+}$ from $A$ by adding $e^{(i)}$ as an extra row. Because $\xi_{i}=0$ for all $\xi\in\ker A$ we have $\ker A^{+}=\ker A$. Therefore, $\operatorname{rk}A=\operatorname{rk}A^{+}$ and thus $e^{(i)}$ is a linear combination of the rows of $A$. Hence, $i\in\mathfrak{F}(A)$. ∎

Furthermore, excluding frozen coordinates, a proper relation $I$ of $A$ renders a non-trivial linear relation amongst at least two of the variables $(x_{i})_{i\in I}$. Finally, $A$ is $(\delta,\ell)$-free if only few $\ell$-subsets $I\subseteq[n]$ are proper relations.

We proceed to put forward a small random perturbation that will mostly rid a given matrix of short proper relations, an observation that we expect to be of independent interest.

Definition 2.3**.**

Let $A$ be an $m\times n$ matrix and let $\theta\geq 0$ be an integer. Let $\boldsymbol{i}_{1},\boldsymbol{i}_{2},\ldots,\boldsymbol{i}_{\theta}\in[n]$ be uniformly random and mutually independent column indices. Then the matrix $A[\theta]$ is obtained by adding $\theta$ new rows to $A$ such that for each $j\in[\theta]$ the $j$-th new row has precisely one non-zero entry, namely a one in the $\boldsymbol{i}_{j}$-th column.

In other words, in $A[\theta]$ we expressly peg $\theta$ randomly chosen variables $x_{\boldsymbol{i}_{1}},\ldots,x_{\boldsymbol{i}_{\theta}}$ of the linear system $Ax=0$ to zero. The proof of the following proposition is based on a blend of algebraic and probabilistic ideas.

Proposition 2.4**.**

For any $\delta>0$, $\ell>0$ there exists $\mathcal{T}=\mathcal{T}(\delta,\ell)>0$ such that for any matrix $A$ over any field $\mathbb{F}$ the following is true. With $\boldsymbol{\theta}\in[\mathcal{T}]$ chosen uniformly at random we have

[TABLE]

The key feature of Proposition 2.4 is that the maximum number $\mathcal{T}$ of variables that get pegged to zero does not depend on the matrix $A$ or its size but on $\delta$ and $\ell$ only. Moreover, since adding a single row can change the nullity by at most one, we obtain $|\operatorname{nul}(A)-\operatorname{nul}A[\boldsymbol{\theta}]|\leq\mathcal{T}$. Hence, while eliminating short proper relations, the perturbation does not shift the nullity significantly. Proposition 2.4 is a sweeping generalisation of a probabilistic result from [7], where the perturbation from Definition 2.3 was applied to matrices over finite fields to diminish stochastic dependencies amongst entries of randomly chosen vectors in the kernel. That argument, in turn, was inspired by ideas from information theory [24, 77, 82]. We will come back to this in Section 2.3.

We will incorporate the perturbation from Proposition 2.4 into the Aizenman-Sims-Starr coupling argument, which reduces the rank calculation to studying the impact of a few additional rows and columns on the rank. The following lemma, whose proof consists of a few lines of linear algebra, shows how the impact of such operations can be tracked in the absence of proper relations. Specifically, the lemma shows that all we need to know about the matrix $A$ to which we add rows/columns is the set $\mathfrak{F}(A)$ of frozen variables.

Lemma 2.5**.**

Let $A,B,C$ be matrices of size $m\times n$, $m^{\prime}\times n$ and $m^{\prime}\times n^{\prime}$, respectively, and let $I\subseteq[n]$ be the set of all indices of non-zero columns of $B$. Moreover, obtain $B_{*}$ from $B$ by replacing for each $i\in I\cap\mathfrak{F}(A)$ the $i$-th column of $B$ by zero. Unless $I$ is a proper relation of $A$ we have

[TABLE]

Observe that the quantity on the l.h.s. of (2.4) (and thus the one on the r.h.s. as well) may be either positive or negative, depending on $A,B,C$.

To put Proposition 2.4 and Lemma 2.5 to work, we need to explain the construction of the telescoping series of random variables upon which the Aizenman-Sims-Starr argument is based. That is our next step.

2.2. The Aizenman-Sims-Starr scheme

In order to derive the desired lower bound on the rank we need to bound the nullity of $\boldsymbol{A}_{n}$ from above. In line with the Aizenman-Sims-Starr scheme [2, 79], a first stab at this problem might be to write a telescoping sum

[TABLE]

Providing that $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{n+1})]-\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{n})]$ is bounded, the lim sup of the sequence of summands exists. In this case, due to the normalising factor $1/N$ on the r.h.s. of (2.5), we obtain

[TABLE]

Hence, combinig (2.5) and (2.6), we obtain the bound

[TABLE]

To obtain an explicit estimate, we should thus attempt to couple $\boldsymbol{A}_{n+1}$ and $\boldsymbol{A}_{n}$ so that we can write a single expectation

[TABLE]

Ideally, to bring the tools from Section 2.1 to bear, under this coupling $\boldsymbol{A}_{n+1}$ should be obtained from $\boldsymbol{A}_{n}$ by adding one column and a few rows.

Unfortunately, this direct approach flounders for obvious reasons. For instance, depending on the distributions $\boldsymbol{d},\boldsymbol{k}$, due to divisibility issues $\boldsymbol{A}_{n+1}$ may not even be defined for all $n$.555For instance, suppose that $\boldsymbol{d}=3$ and $\boldsymbol{k}=4$ deterministically. Then (1.1) boils down to $4{\boldsymbol{m}}=3n$, and thus $\boldsymbol{A}_{n}$ is well-defined only if $n$ is divisible by four. To deal with this issue we introduce a more malleable version of the random matrix model, without significantly altering the rank. Specifically, we introduce a parameter $\varepsilon>0$, for which we choose a large enough $\mathcal{T}=\mathcal{T}(\varepsilon)>0$. Then for integers $n\geq\mathcal{T}$ we construct a random matrix $\boldsymbol{A}_{\varepsilon,n}$ as follows. Like in Section 1.2 let $\chi:[0,1]^{2}\to\mathbb{F}^{*}$ be a measurable map and let $(\boldsymbol{\zeta}_{i},\boldsymbol{\xi}_{i})_{i\geq 1}$ be uniformly distributed $[0,1]$-valued random variables. Further, let

[TABLE]

Additionally, choose $\boldsymbol{\theta}\in[\mathcal{T}]$ uniformly at random and, as before, let $(\boldsymbol{d}_{i})_{i\geq 1}$, $(\boldsymbol{k}_{i})_{i\geq 1}$ be copies of $\boldsymbol{d}$, $\boldsymbol{k}$. All of these random variables are mutually independent. Further, let $\boldsymbol{\Gamma}_{\varepsilon,n}$ be a uniformly random maximal matching of the complete bipartite graph with vertex classes

[TABLE]

As in the well known configuration model of random graphs, we think of $\left\{{a_{i}}\right\}\times[\boldsymbol{k}_{i}]$ as a set of clones of $a_{i}$ and of $\{x_{j}\}\times[\boldsymbol{d}_{j}]$ as a set of clones of $x_{j}$. We obtain a random Tanner graph $\boldsymbol{G}_{\varepsilon,n}$ with variable nodes $x_{1},\ldots,x_{n}$ and check nodes $a_{1},\ldots,a_{{\boldsymbol{m}}_{\varepsilon,n}},p_{1},\ldots,p_{\boldsymbol{\theta}}$ by inserting an edge between $a_{i}$ and $x_{j}$ for each matching edge that joins the sets $\left\{{a_{i}}\right\}\times[\boldsymbol{k}_{i}]$ and $\{x_{j}\}\times[\boldsymbol{d}_{j}]$. Additionally, check node $p_{i}$ is adjacent to $x_{i}$ for each $i\in[\boldsymbol{\theta}]$. To be clear, we do not need to set aside any unmatched variable clones as partners of the $p_{i}$. We simply add the $x_{i}$-$p_{i}$-edges on top of the configuration model. Since ultimately $\mathcal{T}$ will be chosen to be of order $o(n)$, the number of these additional edges is relatively small.

Since there may be several edges joining clones of the same variable and check node, $\boldsymbol{G}_{\varepsilon,n}$ may be a multigraph. Finally, we construct a random matrix $\boldsymbol{A}_{\varepsilon,n}$ whose rows are indexed by the check nodes $a_{1},\ldots,a_{{\boldsymbol{m}}_{\varepsilon,n}}$ and whose columns are indexed by $x_{1},\ldots,x_{n}$ such that the non-zero entries of $\boldsymbol{A}_{\varepsilon,n}$ represent the edges of the matching $\boldsymbol{\Gamma}_{\varepsilon,n}$. Specifically, the matrix entries read

[TABLE]

Morally, $\boldsymbol{A}_{\varepsilon,n}$ mimics the matrix obtained from the original model $\boldsymbol{A}_{n}$ by deleting every row with probability $\varepsilon$ independently (which, of course, would be unworkable because still the model is not generally defined for all $n$). Furthermore, the purpose of the check nodes $p_{1},\ldots,p_{\boldsymbol{\theta}}$ is to ensure that $\boldsymbol{A}_{\varepsilon,n}$ is $(\delta,\ell)$-free for a small enough $\delta=\delta(\varepsilon)$ and a large enough $\ell=\ell(\varepsilon)$. Indeed, while Proposition 2.4 requires that a random set of $\boldsymbol{\theta}$ variables be pegged, the checks $p_{1},\ldots,p_{\boldsymbol{\theta}}$ just freeze the first $\boldsymbol{\theta}$ variables. But since the distribution of the Tanner graph $\boldsymbol{G}_{\varepsilon,n}-\{p_{1},\ldots,p_{\boldsymbol{\theta}}\}$ is invariant under permutations of the variable nodes, both constructions are equivalent. The following proposition shows that going to $\boldsymbol{A}_{\varepsilon,n}$ does not shift the rank significantly.

Proposition 2.6**.**

For any any $0<C<C^{\prime}$ and any function $\mathcal{T}=\mathcal{T}(\varepsilon)\geq 0$ the following is true. If

[TABLE]

Analogously, if

[TABLE]

By construction, the degrees of the checks $a_{i}$ and the variables $x_{j}$ in $\boldsymbol{G}_{\varepsilon,n}-\{p_{1},\ldots,p_{\boldsymbol{\theta}}\}$ are upper-bounded by $\boldsymbol{k}_{i}$ and $\boldsymbol{d}_{j}$, respectively. We thus refer to $\boldsymbol{k}_{i}$ and $\boldsymbol{d}_{j}$ as the target degrees of $a_{i}$ and $x_{j}$. Indeed, since $\boldsymbol{G}_{\varepsilon,n}$ will turn out to feature few if any multi-edges and ${\boldsymbol{m}}_{\varepsilon,n}$ is significantly smaller than $dn/k$ and thus

[TABLE]

most check nodes $a_{i}$ have degree precisely $\boldsymbol{k}_{i}$ a.a.s. But we expect that about $\varepsilon dn$ variable nodes $x_{i}$ will have degree less than $\boldsymbol{d}_{i}$. In fact, a.a.s. $\boldsymbol{\Gamma}_{\varepsilon,n}$ fails to cover about $\varepsilon dn$ ‘clones’ from the set $\bigcup_{i=1}^{n}\{x_{i}\}\times[\boldsymbol{d}_{i}]$. Let us call such unmatched clones cavities.

The cavities provide the wiggle room that we need to couple $\boldsymbol{A}_{\varepsilon,n}$ and $\boldsymbol{A}_{\varepsilon,n+1}$. An instant idea might be to couple $\boldsymbol{G}_{\varepsilon,n+1}$ and $\boldsymbol{G}_{\varepsilon,n}$ such that the former is obtained by adding one variable node $x_{n+1}$ along with $\boldsymbol{d}_{n+1}$ new adjacent check nodes. Additionally, the new checks get connected with some random cavities of $\boldsymbol{G}_{\varepsilon,n}$. In effect, the coupling takes the form

[TABLE]

where $\boldsymbol{B}$ has $n$ columns and $\boldsymbol{d}_{n+1}$ rows and $\boldsymbol{C}$ is a column vector of size $\boldsymbol{d}_{n+1}$ a.a.s. But this direct attempt has a subtle flaw. Indeed, going from $\boldsymbol{A}_{\varepsilon,n}$ to $\boldsymbol{A}_{\varepsilon,n+1}$, (2.9) adds $\mathbb{E}[\boldsymbol{d}_{n+1}]=d$ rows on the average. Yet actually we should be adding merely $\mathbb{E}[{\boldsymbol{m}}_{\varepsilon,n+1}-{\boldsymbol{m}}_{\varepsilon,n}]=(1-\varepsilon)d/k$ rows. To remedy this problem we borrow a trick from prior applications of the Aizenman-Sims-Starr scheme in combinatorics [7, 24, 25]. Namely, we set up a coupling under which both $\boldsymbol{A}_{\varepsilon,n},\boldsymbol{A}_{\varepsilon,n+1}$ are obtained by adding a few rows/columns to a common ‘base matrix’ $\boldsymbol{A}^{\prime}$. Thus, instead of (2.9) we obtain

[TABLE]

To be precise, $\boldsymbol{C}^{\prime}$ above is a column vector with an expected $(1-\varepsilon)d$ non-zero entries and $\boldsymbol{B},\boldsymbol{B}^{\prime}$ are matrices whose numbers of non-zero entries are bounded in expectation. Furthermore, the base matrix $\boldsymbol{A}^{\prime}$ itself is quite similar to $\boldsymbol{A}_{\varepsilon,n}$, except that $\boldsymbol{A}^{\prime}$ has a slightly smaller number of rows. In Section 5 we will present the construction in full detail and apply Proposition 2.4 and Lemma 2.5 to prove the following upper bound on the change in nullity. Recall the function $\Phi$ from (1.2) and recall that $\boldsymbol{\theta}\in[\mathcal{T}]$ with $\mathcal{T}=\mathcal{T}(\varepsilon)$ dependent on $\varepsilon$ only is the number of pinned variables in the construction of $\boldsymbol{A}_{\varepsilon,n}$.

Proposition 2.7**.**

There exists a function $\mathcal{T}=\mathcal{T}(\varepsilon)>0$ such that

[TABLE]

As an immediate consequence of Proposition 2.7 we obtain the desired upper bound on the nullity.

Corollary 2.8**.**

We have

[TABLE]

Proof.

Proposition 2.7 yields

[TABLE]

as claimed. ∎

Proof of Theorem 1.1.

The desired lower bound on the rank of $\boldsymbol{A}_{n}$ is an immediate consequence of Proposition 2.6 and Corollary 2.8. ∎

2.3. Discussion

Before delving into the technical details of the proofs of the various propositions, we compare the proof strategy and the results with previous work. We begin with a discussion of related work on the rank problem. Roughly speaking, prior work on the rank of random matrices relies on separate strands of techniques, depending on whether the average number of non-zero entries per row/column is bounded or unbounded. Subsequently we discuss the physicists’ (non-rigorous) cavity method and explain how it led to an erroneous prediction.

2.3.1. Dense matrices

The difficulty of the rank problem for dense random matrices strongly depends on the distribution of the matrix entries. For instance, a square matrix with independent Gaussian entries in each row has full rank with probability one simply because the submanifold of singular matrices has Lebesgue measure zero. By contrast, the case of matrices with independent uniform $\pm 1$ entries is more subtle. Komlós [58] proved that such matrices are regular a.a.s. Vu [87] subsequently presented a simpler proof, based on collision probabilities and Erdős’ Littlewood-Offord inequality. An intriguing conjecture, which has inspired a distinguished line of research [48, 67, 52, 85, 86], asserts that the dominant reason for a random $\pm 1$-matrix being singular is the existence of a pair of identical rows or columns.

Interesting enough, the limiting probability that a dense square matrix with entries drawn uniformly from a finite field is singular lies strictly between zero and one. Kovalenko and Levitskaya [59, 60, 65, 66] obtained a precise formula for the distribution of the rank of dense random matrices with independent entries over finite fields via the method of moments. For more recent improvements see [40, 70] and the references therein.

A further line of work deals with random $m\times n$ matrices in which the number of non-zero entries per row diverges in the limit of large $n$ but is of order $o_{n}(n)$. Relating the permanent and the determinant, Balakin [9] and, using delicate moment calculations, Blömer, Karp and Welzl [12] dealt with the rank of such matrices over finite fields. Moreover, using expansion arguments, Costello and Vu [29, 30] studied the real rank of random symmetric matrices of a similar density. They find that such matrices essentially have full rank a.a.s., apart from a small defect based on local phenomena. In the words of [30], “dependency [comes] from small configurations”.

2.3.2. Sparse matrices

Matters are quite different in the sparse case where the average number of non-zero entries per row is bounded. In fact, as we will discover in due course the formula from Theorem 1.1 is driven by “dependency coming from large configurations”, i.e., by minimally linearly dependent sets of unbounded size.

The first major contribution dedicated to sparse matrices was a paper by Dubois and Mandler [33] on the random $3$-XORSAT problem. Translated into random matrices, this problem asks for what ratios $m/n$ a random $m\times n$-matrix over $\mathbb{F}_{2}$ with precisely three one-entries in each row has full rank (i.e., equal to $m\wedge n$) a.a.s. Thus, the random matrix model is just the one from Example 1.8 with $k=3$. Dubois and Mandler pinpointed the precise full row rank threshold $m/n\approx 2.75$. The proof relies on the first moment method applied to $|\ker\boldsymbol{A}|$, which boils down to a one-dimensional calculus problem. Matters get more complicated when one considers a greater number $k>3$ of non-zero entries per row. This more general problem, known as random $k$-XORSAT, was solved independently by Dietzfelbinger et al. [32] and by Pittel and Sorkin [81] via technically demanding moment calculations. Unfortunately, considering fields $\mathbb{F}_{q}$ with $q>2$ complicates the moment calculation even further. Yet undertaking a computer-assisted tour-de-force Falke and Goerdt [44] managed to extend the method to $\mathbb{F}_{3}$. However, extending this strategy to infinite fields is a non-starter as $|\ker\boldsymbol{A}|$ may be infinite.

In a previous paper Ayre, Coja-Oghlan, Gao and Müller [7] applied the Aizenman-Sims-Starr scheme to the study of sparse random matrices with precisely $k$ non-zero entries per row as in Example 1.8, over finite fields. The present paper goes beyond that earlier contribution in two crucial ways. First, we develop a far more delicate coupling scheme that accommodates general degree distributions $\boldsymbol{d},\boldsymbol{k}$ rather than just the Poisson-constant degrees from Example 1.8, including degree sequences for which the 2-core bound fails to be tight (in contrast to Example 1.8). Apart from rendering a proof of Lelarge’s conjecture, we expect that this more general coupling scheme will find further uses in the theory of random factor graphs; for instance, it seems applicable to generalisations of the models from [24].

Second, the rank calculation in [7] is based on a probabilistic view that does not extend to infinite fields. Indeed, the proof there is based on a close study of a uniformly random element $\boldsymbol{\sigma}$ of the kernel of the random matrix $\boldsymbol{A}$. Specifically, [7, Lemma 3.1] analyses the impact of the perturbation from Definition 2.3 on a matrix $A\in\mathbb{F}^{m\times n}$ for a finite field $\mathbb{F}$. With $\boldsymbol{\sigma}=(\boldsymbol{\sigma}_{1},\ldots,\boldsymbol{\sigma}_{n})\in\ker(A)$ a uniformly random element of the kernel, the lemma shows that for a large enough $\mathcal{T}=\mathcal{T}(\delta,\mathbb{F})>0$ and a uniformly random $0\leq\boldsymbol{\theta}\leq\mathcal{T}$,

[TABLE]

As in Proposition 2.4, the necessary value of $\mathcal{T}$ is independent of $n,m$ and $A$. Thus, the random perturbation renders the vector entries $(\boldsymbol{\sigma}_{i},\boldsymbol{\sigma}_{j})$ nearly stochastically independent, for most $i,j$. Thanks to general results from [10], (2.11) extends from pairwise independence to $\ell$-wise independence, albeit with a weaker error bound $\delta$. The result [7, Lemma 3.1] was inspired by general statements about probability measures on discrete cubes from [24, 77, 82].

Inherently, this stochastic approach does not generalise to infinite fields, where, for starters, it does not even make sense to speak of a uniformly random element of the kernel. That is why here we replace the stochastic approach from the earlier paper by the more versatile algebraic approach summarised in Proposition 2.4, which are applicable to any field—say, the reals, the field $\mathbb{Q}_{p}$ of $p$-adic numbers, the algebraic closure of a finite field or a structure as complex as a function field. Instead of showing stochastic independence, Proposition 2.4 renders linear independence amongst most bounded-size subsets of coordinates. Apart from being more general, this algebraic viewpoint allows for a cleaner, more direct proof of the rank formula. Additionally, on finite fields the stochastic independence (2.11) follows from the linear independence provided by Proposition 2.4, with a significantly improved bound on $\mathcal{T}(\delta)$. We work this out in detail in Appendix B.

The single prior contribution on the rational rank of sparse random matrices is due to Bordenave, Lelarge and Salez [15], who computed the rational rank of the (symmetric) adjacency matrix of a random graph with a given vertex degree distribution. The proof is based on local weak convergence and the ‘objective method’ [5]. An intriguing question for future research is to extend the techniques from the present paper to symmetric random matrices.

2.3.3. The cavity method (and its caveats)

On the basis of the cavity method, an analytic but non-rigorous technique inspired by the statistical mechanics of disordered systems, it had been predicted erroneously that over finite fields the 2-core bound (1.7) on the rank of $\boldsymbol{A}$ is universally tight for general degree distributions $\boldsymbol{d},\boldsymbol{k}$ [3, 73]. Where did the cavity method go astray?

The method comes in two instalments, the simpler replica symmetric ansatz and the more elaborate one-step replica symmetry breaking ansatz (‘1RSB’). The former predicts that the rank of $\boldsymbol{A}$ over a finite field $\mathbb{F}_{q}$ converges in probability to the solution of an optimisation problem on an infinite-dimensional space of probability measures. To be precise, let $\mathcal{P}(\mathbb{F}_{q})$ be the space of probability measures on $\mathbb{F}_{q}$. Identify this space with the standard simplex in $\mathbb{R}^{q}$. Further, let $\mathcal{P}^{2}(\mathbb{F}_{q})$ be the space of all probability measures on $\mathcal{P}(\mathbb{F}_{q})$. Given $\pi\in\mathcal{P}^{2}(\mathbb{F}_{q})$ let $(\boldsymbol{\mu}_{i,j}^{\pi})_{i,j\geq 1}$ be a sequence of independent samples from $\pi$. Recalling (1.10), the Bethe free entropy is defined by

[TABLE]

The replica symmetric ansatz predicts that

[TABLE]

For a detailed (heuristic) derivation of the Bethe free entropy and the prediction (2.12) we refer to [3]. But let us briefly comment on the intended semantics of $\pi$. Consider the Tanner graph $\boldsymbol{G}$ representing $\boldsymbol{A}$. Suppose that variable node $x_{i}$ and check node $a_{j}$ are adjacent. Then for $\sigma\in\mathbb{F}_{q}$ we define the Belief Propagation message $\mu_{\boldsymbol{A},x_{j}\to a_{i}}(\sigma)$ from $x_{j}$ to $a_{i}$ as follows. Obtain $\boldsymbol{A}_{x_{j}\to a_{i}}$ from $\boldsymbol{A}$ by changing the $ij$-th matrix entry to zero; this corresponds to deleting the $x_{j}$-$a_{i}$-edge from the Tanner graph. Then $\mu_{\boldsymbol{A},x_{j}\to a_{i}}(\sigma)$ is the probability that in a uniformly random vector $\boldsymbol{\sigma}\in\ker\boldsymbol{A}_{x_{j}\to a_{i}}$ we have $\boldsymbol{\sigma}_{j}=\sigma$. Further, define $\pi_{\boldsymbol{A}}$ as the empirical distribution of the $\mu_{\boldsymbol{A},x_{j}\to a_{i}}$ over the edges of the Tanner graph:

[TABLE]

Then the replica symmetric ansatz predicts that $\pi_{\boldsymbol{A}}$ is asymptotically a maximiser of the Bethe free energy, i.e., $\sup_{\pi\in\mathcal{P}^{2}(\mathbb{F}_{q})}\mathcal{B}(\pi)=\mathcal{B}(\pi_{\boldsymbol{A}})+o_{n}(1)$ a.a.s. Thus, the maximiser $\pi$ in (2.12) is deemed to encode the Belief Propagation messages on the edges of the Tanner graph of $\boldsymbol{A}$.

A bit of linear algebra that seems to have gone unnoticed in the physics literature reveals that the messages actually have a very special form [7, Lemma 2.3]. Namely, any message $\mu_{\boldsymbol{A},x_{j}\to a_{i}}$ is either the uniform distribution $q^{-1}\boldsymbol{1}$ on $\mathbb{F}_{q}$ or the atom $\delta_{0}$ on [math]. In effect, the rank should come out as the Bethe free entropy $\mathcal{B}(\pi_{\alpha})$ of a convex combination

[TABLE]

In fact, a simple calculation yields $\Phi(\alpha)=\mathcal{B}(\pi_{\alpha})$ for all $\alpha\in[0,1]$. Thus, Theorem 1.1 shows that

[TABLE]

vindicating the cavity method to an extent. However, we do not know whether the Bethe free entropy possesses other spurious maximisers $\pi\in\mathcal{P}^{2}(\mathbb{F}_{q})$ with $\mathcal{B}(\pi)>\sup_{\alpha\in[0,1]}\mathcal{B}(\pi_{\alpha})$.

Alamino and Saad [3] tackled the optimisation problem (2.12) by means of a numerical heuristic called population dynamics, without noticing the restriction to $(\pi_{\alpha})_{\alpha\in[0,1]}$. In all the examples that they studied they found that $\pi\in\{\pi_{0},\pi_{\rho}\}$, with $\rho$ from (1.5); in fact, all their examples fall within the purview of Theorem 1.4.666Strictly speaking, Alamino and Saad, who worked numerically with $n$ in the hundreds, reported $\pi\in\{\pi_{0},\pi_{1}\}$. Indeed, $\rho\in\{0,1\}$ in the first class of examples that they studied, but not in the other two. For instance, in their example (3) the actual value of $\rho$ is either [math] or a number strictly smaller than one, although $\rho>0.97$ whenever $\Phi(\rho)>\Phi(0)$. This led Alamino and Saad to conjecture that the maximiser $\pi$ is generally of this form, although they cautioned that further evidence seems necessary. Example 1.9 and [63] provide counterexamples. The more sophisticated 1RSB cavity method is presented in [73, Chapter 19], where an exercise asks the reader to verify that the 2-core bound is tight (over finite fields). While Theorem 1.4 gives sufficient conditions for this to be correct, the aforementioned counterexamples apply.

2.4. Organisation

We proceed to prove Proposition 2.4, the ‘key lemma’ upon which the proof of Theorem 1.1 rests, in Section 3. Subsequently in Section 4 we use concentration inequalities and the local limit theorem for sums of independent random variables to prove Proposition 2.6. Additionally, Section 4 contains Proposition 1.10, which shows that the random matrix model (1.1) is well defined, a standard argument that we include for the sake of completeness. Dealing with the full details of the coupling scheme, Section 5 contains the proof of Proposition 2.7. Further, Section 6 deals with the proof of Theorem 1.2 and in Section 7 we prove Theorem 1.4. For the sake of completeness a proof of Lemma 1.11 is included in Appendix A. Moreover, in Appendix B we elaborate on the relation between the algebraic perturbation from Proposition 2.4 and the stochastic version from [7]. Finally, Appendix C contains a self-contained proof of the upper bound on the rank for Theorem 1.1 via the interpolation method from mathematical physics.

3. Linear relations: proof of Proposition 2.4

In this section we prove Proposition 2.4 and Lemma 2.5. The somewhat delicate proof of the former is based on a blend of probabilistic and algebraic arguments. The proof of the latter is purely algebraic and fairly elementary.

3.1. Proof of Proposition 2.4

Observe that Proposition 2.4 is not an asymptotic statement to the extent that we need to exhibit a function $\mathcal{T}=\mathcal{T}(\delta,\ell)$ such that (2.3) holds for all matrices $A$ (ultimately in (3.12) we will see that $\mathcal{T}(\delta,\ell)$ scaling as $\ell^{3}/\delta^{4}$ does the trick). Nevertheless, letting $n$ denote the number of columns of $A$, we may safely assume that $n>n_{0}=n_{0}(\delta,\ell)$ for any specific $n_{0}$ that depends on $\delta,\ell$ only. Indeed, to deal with $n\leq n_{0}$ for any fixed value $n_{0}$ we could just pick $\mathcal{T}\geq\mathcal{T}_{0}(\delta,\ell)$ for a large enough $\mathcal{T}_{0}(\delta,\ell)$ so that with probability at least $1-\varepsilon$ we have $\{\boldsymbol{i}_{1},\ldots,\boldsymbol{i}_{\boldsymbol{\theta}}\}=[n]$. Note that we do not need to worry about the possibility that $\mathcal{T}>n$ because the $\boldsymbol{i}_{j}$ are drawn with replacement. Further, if $\{\boldsymbol{i}_{1},\ldots,\boldsymbol{i}_{\boldsymbol{\theta}}\}=[n]$, then a glimpse at Definition 2.1 shows that all coordinates are frozen. Therefore, $A[\boldsymbol{\theta}]$ is $(\delta,\ell)$-free. Hence, from now on we assume that $n\geq n_{0}=n_{0}(\delta,\ell)$ for a sufficiently large $n_{0}$.

Given any matrix $M$ we define a minimal $h$-relation of $M$ as a relation $I$ of $M$ of size $|I|=h$ that does not contain a proper subset that is a relation of $M$. Let ${\mathcal{R}}_{h}(M)$ be the set of all minimal $h$-relations of $M$ and set $R_{h}(M)=|{\mathcal{R}}_{h}(M)|$. Thus, $R_{1}(M)=|\mathfrak{F}(M)|$ is just the number of frozen variables of $M$. Additionally, let ${\mathcal{R}}_{\leq h}(M)=\bigcup_{1\leq i\leq h}{\mathcal{R}}_{i}(M)$ and $R_{\leq h}(M)=|{\mathcal{R}}_{\leq h}(M)|$. Let $\boldsymbol{i}_{1},\boldsymbol{i}_{2},\boldsymbol{i}_{3},\ldots\in[n]$ be uniformly distributed independent random variables.

The proof of Proposition 2.4 is based on a potential function argument. To get started we observe that

[TABLE]

This inequality implies that the random variable

[TABLE]

is non-negative. The random variable $\Delta_{t}$ gauges the increase in frozen variables upon addition of $\ell$ more rows that expressly freeze specific variables. Thus, ‘big’ values of $\Delta_{t}$, say $\Delta_{t}=\Omega_{n}(1)$, witness a kind of instability as pegging a few variables to zero entails that another $\Omega_{n}(n)$ variables get frozen to zero due to implicit linear relations. We will exploit the observation that, since $\Delta_{t}\in[0,1]$ and $\mathbb{E}[\Delta_{t}]$ is monotonically increasing in $t$, such instabilities cannot occur for many $t$. Thus, the expectation $\mathbb{E}[\Delta_{\boldsymbol{\theta}}]$ will serve as our potential. A similar potential was used in [7] to study stochastic dependencies in the case of finite fields $\mathbb{F}$. But in the present more general context the analysis of the potential is significantly more subtle. The following lemma puts a lid on the potential.

Lemma 3.1**.**

We have $\mathbb{E}[\Delta_{\boldsymbol{\theta}}]\leq\ell/\mathcal{T}$.

Proof.

For any $r\in\{0,1,\ldots,\ell-1\}$ we have

[TABLE]

Observe that there is no problem here taking the limit $j\to\infty$ as the coordinates $\boldsymbol{i}_{j}$ from Definition 2.3 are chosen independently with replacement. In the case that $j\gg n$ the likely outcome is thus that all coordinates of $A[r+j\ell]$ are frozen, which is why $\lim_{j\to\infty}\mathbb{E}[R_{1}(A[r+j\ell])]=n$. Hence, (3.2) yields

[TABLE]

Summing (3.3) on $r$, we obtain

[TABLE]

Since $\boldsymbol{\theta}\in[\mathcal{T}]$ is chosen uniformly and independently of everything else, dividing (3.4) by $\mathcal{T}$ yields

[TABLE]

as desired. ∎

Remark 3.2**.**

Lemma 3.1 provides a bound on the mean of $\mathbb{E}[\Delta_{\boldsymbol{\theta}}]$ for a random $\boldsymbol{\theta}$. The requirement that $\boldsymbol{\theta}$ be random stems from the fact that the proof is based on an averaging argument. It is an open question whether this random value could be replaced by a deterministic value, and whether the choice of such a deterministic value would have to depend on $A$.

The following lemma shows that unless $A[t]$ is $(\delta,\ell)$-free, there exist many minimal $h$-relations for some $2\leq h\leq\ell$.

Lemma 3.3**.**

If $A[t]$ fails to be $(\delta,\ell)$-free then there exists $2\leq h\leq\ell$ such that $R_{h}(A[t])\geq\delta n^{h}/\ell$.

Proof.

Assume that

[TABLE]

Since every proper relation $I$ of size $|I|=\ell$ contains a minimal $h$-relation $J\subseteq I$ for some $2\leq h\leq\ell$, (3.6) implies that $A[t]$ possesses fewer than $\delta n^{\ell}$ proper relations of size $\ell$ in total. Hence, if (3.6) holds, then $A[t]$ is $(\delta,\ell)$-free. ∎

As a next step we show that $\Delta_{t}$ is large if $A[t]$ possesses many minimal $h$-relations for some $2\leq h\leq\ell$.

Lemma 3.4**.**

If $R_{h}(A[t])\geq\delta n^{h}/\ell$ for some $2\leq h\leq\ell$, then $\Delta_{t}\geq\delta^{2}/\ell^{2}$.

Proof.

Let ${\mathcal{R}}_{v,h}(A[t])$ be the set of all relations $I\in{\mathcal{R}}_{h}(A[t])$ that contain $v\in[n]$ and set $r_{v,t,h}=|{\mathcal{R}}_{v,h}(A[t])|$. Moreover, let $\mathcal{V}_{t,h}$ be the set of all $v\in[n]$ with $r_{v,t,h}\geq\delta hn^{h-1}/(2\ell)$. We assumed $|R_{h}(A[t])|\geq\delta n^{h}/\ell$, and every $h$-relation is affiliated with an $h$-element subset of $[n]$. Consequently,

[TABLE]

whence

[TABLE]

Consider $v\in\mathcal{V}_{t,h}$ along with a minimal $h$-relation $I\in{\mathcal{R}}_{v,h}(A[t])$. If $I=\{v,\boldsymbol{i}_{t+1},\ldots,\boldsymbol{i}_{t+h-1}\}$, i.e., $I$ comprises $v$ and the next $h-1$ indices that get pegged, then $v\in\mathfrak{F}(A[t+h-1])$. Indeed, since $I$ is a minimal $h$-relation of $A[t]$ there is a row vector $y$ such that ${\text{supp}}(yA[t])=I$. Hence, if $I\setminus\left\{{v}\right\}=\left\{{\boldsymbol{i}_{t+1},\ldots,\boldsymbol{i}_{t+h-1}}\right\}$, then we can extend $y$ to a row vector $y^{\prime}$ such that ${\text{supp}}(y^{\prime}A[t+\ell])=\left\{{v}\right\}$, and thus $v\in\mathfrak{F}(A[t+h-1])$. Furthermore, since $(\boldsymbol{i}_{t+1},\ldots,\boldsymbol{i}_{t+h-1})\in[n]^{h-1}$ is uniformly random, we conclude that

[TABLE]

Now, because every $v\in\mathcal{V}_{t,h}$ satisfies $r_{v,t,h}\geq\delta hn^{h-1}/(2\ell)$, (3.8) implies that

[TABLE]

We also notice that $\mathcal{V}_{t,h}\cap\mathfrak{F}(A[t])=\emptyset$ because no minimal $h$-relation contains a frozen variable. Therefore, combining (3.1), (3.7) and (3.9) and using linearity of expectation, we obtain

[TABLE]

as desired. ∎

Combining Lemmas 3.3 and 3.4, we immediately obtain the following.

Corollary 3.5**.**

If $A[t]$ fails to be $(\delta,\ell)$-free then $\Delta_{t}\geq\delta^{2}/\ell^{2}$.

We have all the ingredients in place to complete the proof of Proposition 2.4.

Proof of Proposition 2.4..

We define T=\left\{{t\in[\mathcal{T}]:{\mathbb{P}}\left[{A[t]\mbox{ fails to be (\delta,\ell)-free}}\right]\geq\delta/2}\right\} so that

[TABLE]

Hence, we are left to estimate ${\mathbb{P}}\left[{\boldsymbol{\theta}\in T}\right]$. Applying Corollary 3.5, we obtain for every $t\in T$,

[TABLE]

Moreover, averaging (3.11) on $t\in[\mathcal{T}]$ and applying Lemma 3.1, we obtain

[TABLE]

Consequently, choosing

[TABLE]

ensures ${\mathbb{P}}\left[{\boldsymbol{\theta}\in T}\right]\leq\delta/2$. Thus, the assertion follows from (3.10). ∎

Remark 3.6**.**

The proof presented in this section actually renders a slightly stronger statement than Proposition 2.4. Specifically, let $A$ be an $m\times N$-matrix and let $n\leq N$. Obtain $A[\theta,n]$ by pegging $\theta$ random variables from among the first $n$ variables $x_{1},\ldots,x_{n}$ of the linear system $Ax=0$ to zero. Then with $\boldsymbol{\theta}=\boldsymbol{\theta}(\delta,\ell)$ chosen as in Proposition 2.4 we find that with probability at least $1-\delta$, there are no more than $\delta n^{\ell}$ proper relations $I\subseteq[n]$. Thus, in order to rid a subset of the columns of short linear relations, it suffices to peg $\boldsymbol{\theta}$ random variables from that subset to zero. The proof of this stronger statement proceeds as above, except that we confine ourselves to minimal relations among the first $n$ columns.

3.2. Proof of Lemma 2.5

We are going to derive Lemma 2.5 from the following simpler, deterministic and non-asymptotic statement.

Lemma 3.7**.**

Let $m,n,m^{\prime},n^{\prime}\geq 1$ be integers. Let $A$ be an $m\times n$ matrix, let $B$ be an $m^{\prime}\times n$ matrix and let $C$ be an $m^{\prime}\times n^{\prime}$ matrix. Let $I\subseteq[n]$ be the set of all indices of non-zero columns of $B$. Unless $I$ is a relation of $A$ we have

[TABLE]

Proof.

Suppose that $I$ is not a relation of $A$. We begin by showing that

[TABLE]

Writing $B_{1},\ldots,B_{m^{\prime}}$ for the rows of $B$ and $r=\operatorname{rk}(B)$ for the rank and applying a row permutation if necessary, we may assume that $B_{1},\ldots,B_{r}$ are linearly independent. Hence, to establish (3.13) it suffices to prove that for all $0\leq\ell<r$,

[TABLE]

In other words, we need to show that $B_{\ell+1}$ does not belong to the space spanned by $B_{1},\ldots,B_{\ell}$ and the rows $A_{1},\ldots,A_{m}$ of $A$. Indeed, assume that $B_{\ell+1}=\sum_{i=1}^{\ell}x_{i}B_{i}+\sum_{i=1}^{m}y_{i}A_{i}$. Then $0\neq B_{\ell+1}-\sum_{i=1}^{\ell}x_{i}B_{i}=\sum_{i=1}^{m}y_{i}A_{i}$ and thus $\emptyset\neq{\text{supp}}\sum_{i=m}^{\ell}y_{i}A_{i}\subseteq I$, in contradiction to the assumption that $I$ is no relation of $A$. Hence, we obtain (3.14) and thus (3.13). Finally, to complete the proof of (2.4) we apply (3.13) to the matrices $(A\ 0)$ and $(B\ C)$, obtaining

[TABLE]

as desired. ∎

Proof of Lemma 2.5.

Recall tha $A$ has size $m\times n$. By Definition 2.1 a coordinate $i$ is frozen iff the vector $e^{(i)}\in\mathbb{F}^{1\times n}$ whose $i$-th entry equals one and whose other entries equal zero can be written as a linear combination of the rows of $A$. For every $i\in\mathfrak{F}(A)$ we can therefore apply elementary row operations (like in Gaussian elimination) to zero out the entire $i$-column of $B$. Since elementary row operations do not alter the nullity of a matrix, we therefore obtain the idenitity

[TABLE]

The assertion thus follows from Lemma 3.7. ∎

4. Concentration

The principal aim of this section is to prove Proposition 2.6, i.e., to argue that the rank of the actual matrix $\boldsymbol{A}_{n}$ that does not have any cavities and whose Tanner graph is simple is close to the expected rank of $\boldsymbol{A}_{\varepsilon,n}$ a.a.s. In other words, we need to show that the rank of a random matrix is sufficiently concentrated that conditioning on

[TABLE]

and on the event $\mathcal{S}$ that the Tanner graph is simple is inconsequential. The main tool will be the following local limit theorem for sums of independent random variables, which we use in Section 4.1 to calculate the probability of $\mathcal{D}$.

Theorem 4.1** ([34, p. 130]).**

Suppose that $(\boldsymbol{X}_{i})_{i\geq 1}$ is a sequence of i.i.d. variables that take values in $\mathbb{Z}$ such that the greatest common divisor of the support of $\boldsymbol{X}_{1}$ is one. Also assume that $\mathrm{Var}[\boldsymbol{X}_{1}]=\sigma^{2}\in(0,\infty)$. Then

[TABLE]

Subsequently, in Section 4.2 we calculate the probability of the event $\mathcal{S}$, proving Proposition 1.10 along the way. Finally, in Section 4.3 we complete the proof of Proposition 2.6.

4.1. The event $\mathcal{D}$

Because $\mathbb{E}[\boldsymbol{d}^{r}]+\mathbb{E}[\boldsymbol{k}^{r}]<\infty$ for an $r>2$, the event

[TABLE]

As an application of Theorem 4.1 we obtain the following estimate.

Lemma 4.2**.**

If $\gcd(\boldsymbol{k})$ divides $n$, then ${\mathbb{P}}\left[{\mathcal{D}}\right]=\Theta_{n}(n^{-1/2})$ and ${\mathbb{P}}\left[{\mathcal{D}\mid\mathcal{M}}\right]=\Theta_{n}(n^{-1/2})$.

Proof.

For ${\mathbb{P}}\left[{\mathcal{D}\mid\mathcal{M}}\right]$ there are several cases to consider. First, that $\mathrm{Var}(\boldsymbol{d})=\mathrm{Var}(\boldsymbol{k})=0$, i.e., $\boldsymbol{d},\boldsymbol{k}$ are both atoms. Since ${\boldsymbol{m}}$ is a Poisson variable with mean $dn/k$ we find ${\mathbb{P}}\left[{\mathcal{D}\mid\mathcal{M}}\right]={\mathbb{P}}\left[{{\boldsymbol{m}}=dn/k}\right]=\Theta_{n}(n^{-1/2})$.

Second, suppose that $\mathrm{Var}(\boldsymbol{d})>0$ but $\mathrm{Var}(\boldsymbol{k})=0$. Then Theorem 4.1 and (4.1) show that

[TABLE]

Further, given $\left|{dn-\sum_{i=1}^{n}\boldsymbol{d}_{i}}\right|\leq\sqrt{n}$ and given $k$ divides $\sum_{i=1}^{n}\boldsymbol{d}_{i}$, the event $k{\boldsymbol{m}}=\sum_{i=1}^{n}\boldsymbol{d}_{i}$ has probability $\Theta_{n}(n^{-1/2})$ by the local limit theorem for the Poisson distribution.

The case that $\mathrm{Var}(\boldsymbol{d})=0$ but $\mathrm{Var}(\boldsymbol{k})>0$ can be dealt with similarly. Indeed, pick a large enough number $L>0$ and let $I=\left\{{i\in[{\boldsymbol{m}}]:\boldsymbol{k}_{i}>L}\right\}$, ${\boldsymbol{m}}^{\prime}=|I|$, ${\boldsymbol{m}}^{\prime\prime}={\boldsymbol{m}}-|I|$, $S^{\prime}=\sum_{i\in I}\boldsymbol{k}_{i}$ and $S^{\prime\prime}=\sum_{i\in[{\boldsymbol{m}}]\setminus I}\boldsymbol{k}_{i}$. Then ${\boldsymbol{m}}^{\prime},{\boldsymbol{m}}^{\prime\prime}$ are stochastically independent, as are $S^{\prime},S^{\prime\prime}$. Moreover, since $S^{\prime}$ satisfies the central limit theorem we have

[TABLE]

Further, Theorem 4.1 applies to $S^{\prime\prime}$, which is distributed as $\sum_{i=1}^{{\boldsymbol{m}}}\boldsymbol{k}_{i}\boldsymbol{1}\{\boldsymbol{k}_{i}\leq L\}$. Hence, as $n$ is divisible by $\gcd(\boldsymbol{k})$, for large enough $L$ we have

[TABLE]

Thus, (4.3) and (4.4) show that ${\mathbb{P}}\left[{\mathcal{D}\mid\mathcal{M}}\right]=\Omega_{n}(n^{-1/2})$. The upper bound ${\mathbb{P}}\left[{\mathcal{D}\mid\mathcal{M}}\right]=O_{n}(n^{-1/2})$ follows from the uniform upper bound from Theorem 4.1.

A similar argument applies in the final case $\mathrm{Var}(\boldsymbol{d})>0$, $\mathrm{Var}(\boldsymbol{k})>0$. Indeed, Theorem 4.1 and (4.1) yield

[TABLE]

Moreover, (4.3) remains valid regardless the variance of $\boldsymbol{d}$. Hence, applying Theorem 4.1 to $S^{\prime\prime}$, we obtain

[TABLE]

Combining (4.5) and (4.6), we see that ${\mathbb{P}}\left[{\mathcal{D}\mid\mathcal{M}}\right]=\Omega_{n}(n^{-1/2})$. The matching upper bound ${\mathbb{P}}\left[{\mathcal{D}\mid\mathcal{M}}\right]=O_{n}(n^{-1/2})$ follows from the universal upper bound from Theorem 4.1 once more. The treatment of the unconditional ${\mathbb{P}}\left[{\mathcal{D}}\right]$ is similar but slightly simpler. ∎

4.2. The event $\mathcal{S}$

The random matrix $\boldsymbol{A}_{n}$ for Theorem 1.1 is identical in distribution to the random matrix $\boldsymbol{A}_{0,n}$ with $\varepsilon=0$ conditioned on the event $\mathcal{D}$ and on the event $\mathcal{S}$ that the Tanner graph $\boldsymbol{G}_{0,n}$ does not contain any multi-edges. Therefore, Proposition 1.10 is going to be a consequence of Lemma 4.2 and the following statement.

Lemma 4.3**.**

We have ${\mathbb{P}}\left[{\boldsymbol{A}_{0,n}\in\mathcal{S}|\mathcal{D}}\right]=\Omega_{n}(1)$.

We proceed to prove Lemma 4.3. Recall the event $\mathcal{M}$ from (4.1). The proof of Lemma 4.3 is essentially based on the routine approach of showing by way of a moment calculation that the number of multi-edges of $\boldsymbol{G}_{0,n}$ is asymptotically Poisson with a finite mean. This argument has been carried out illustratively for the case of random regular graphs in [50, Chapter 9]. But since here we work with very general degree distributions, technical complications arise. For instance, as a first step we need to estimate the empirical variance of the degree sequences.

Claim 4.4**.**

On the event $\mathcal{D}\cap\mathcal{M}$ we have $\frac{1}{n}\sum_{i=1}^{n}\boldsymbol{d}_{i}^{2}\to\mathbb{E}[\boldsymbol{d}^{2}]$, $\frac{1}{n}\sum_{i=1}^{{\boldsymbol{m}}}\boldsymbol{k}_{i}^{2}\to d\mathbb{E}[\boldsymbol{k}^{2}]/k$ in probability.

Proof.

We will only prove the statement about the $\boldsymbol{k}_{i}$; the same (actually slightly simplified) argument applies to the $\boldsymbol{d}_{i}$. Thanks to Bennett’s tail bound for the Poisson distribution we may condition on $\{{\boldsymbol{m}}=m\}$ for some integer $m$ with $|m-dn/k|\leq\sqrt{n}\ln n$. Fix a small $\delta>0$ and a large enough $L=L(\delta)>0$. Given ${\boldsymbol{m}}=m$ the variables $Q_{j}=\sum_{i\in[{\boldsymbol{m}}]}\boldsymbol{1}\{\boldsymbol{k}_{i}=j\}$ have a binomial distribution. Therefore, the Chernoff bound yields

[TABLE]

Hence, (4.1) and Lemma 4.2 yield

[TABLE]

Further, let

[TABLE]

and let ${\mathcal{H}}$ be the set of all integers $h\geq 1$ with $(1+\delta)^{h-1}L\leq\sqrt{n}/\ln^{9}n$. Then the Chernoff bound implies that

[TABLE]

Finally, if $|Q_{j}-dn{\mathbb{P}}\left[{\boldsymbol{k}=j}\right]/k|\leq\sqrt{n}\ln n$ for all $j\leq L$ and $\left|{R_{h}-\bar{R}_{h}}\right|\leq\delta\bar{R}_{h}+\ln^{2}n$ for all $h\in{\mathcal{H}}$, then

[TABLE]

Since this holds true for any fixed $\delta>0$, the assertion follows from (4.7) and (4.8). ∎

Claim 4.5**.**

Let $Y$ be the number of multi-edges of the Tanner graph $\boldsymbol{G}_{0,n}$ and let $\ell\geq 1$. There is $\lambda>0$ such that on

[TABLE]

we have

[TABLE]

Proof.

To estimate the $\ell$-th factorial moments of $Y$ for $\ell\geq 1$, we split the random variable into a sum of indicator variables. Specifically, let $U_{\ell}$ be the set of all families $(u_{i},v_{i},w_{i})_{i\in\ell}$ with $u_{i}\in[{\boldsymbol{m}}]$, $v_{i}\in[n]$ and $2\leq w_{i}\leq\boldsymbol{k}_{u_{i}}\wedge\boldsymbol{d}_{v_{i}}\leq\sqrt{n}/\log^{9}n$ such that the pairs $(u_{1},v_{1}),\ldots,(u_{\ell},v_{\ell})$ are pairwise distinct. Moreover, let $Y[(u_{i},v_{i},w_{i})_{i\in[\ell]}]$ be the number of ordered $\ell$-tuples of multi-edges comprising precisely $w_{i}$ edges between check $a_{u_{i}}$ and variable $x_{v_{i}}$ for each $i$. Then

[TABLE]

Moreover, letting $w=\sum_{i}w_{i}$, we claim that

[TABLE]

Indeed, the factors ${\binom{\boldsymbol{d}_{v_{i}}}{w_{i}}}{\binom{\boldsymbol{k}_{u_{i}}}{w_{i}}}w_{i}!$ count the number of possible matchings between $w_{i}$ clones of the variable node $x_{v_{i}}$, whose degree equals $\boldsymbol{d}_{v_{i}}$, and of the check node $a_{u_{i}}$ of degree $\boldsymbol{k}_{u_{i}}$. Further, since $\ell$ is bounded, the probability that all these matchings are realised in $\boldsymbol{G}_{0,n}$ is asymptotically equal to $(dn)^{-w}$.

Now, for a sequence $\boldsymbol{w}=(w_{1},\ldots,w_{\ell})$ let $Y_{\boldsymbol{w}}=\sum_{(u_{i},v_{i},w_{i})_{i\in[\ell]}\in U_{\ell}}Y[(u_{i},v_{i},w_{i})_{i\in[\ell]}]$. Then (4.9) yields

[TABLE]

the last bound follows from our conditioning on $\mathcal{M}$. As a consequence,

[TABLE]

Further, invoking (4.9), we obtain

[TABLE]

Combining (4.10) and (4.11), we conclude that on $\mathcal{D}\cap\mathcal{M}\cap\{{\boldsymbol{m}}=dn/k+o_{n}(n)\}$,

[TABLE]

Finally, on $\left\{{\sum_{i=1}^{n}\boldsymbol{d}_{i}=dn+o_{n}(n),\,\sum_{i=1}^{n}\boldsymbol{d}_{i}^{2}=n\mathbb{E}[\boldsymbol{d}^{2}]+o_{n}(n)}\right\}\cap\left\{{\sum_{i=1}^{{\boldsymbol{m}}}\boldsymbol{k}_{i}^{2}=dn\mathbb{E}[\boldsymbol{k}^{2}]/k+o_{n}(n)}\right\}$ we have

[TABLE]

and the assertion follows from (4.12)–(4.13). ∎

Claim 4.6**.**

We have ${\mathbb{P}}\left[{\mathcal{S}\mid\mathcal{D}\cap\mathcal{M}}\right]=\Omega_{n}(1)$.

Proof.

Claims 4.4 and 4.5 show together with inclusion/exclusion (e.g., [13, Theorem 1.22]) that a.a.s. on $\mathcal{M}\cap\mathcal{D}$,

[TABLE]

Since $\mathcal{S}=\{Y=0\}$, the assertion follows. ∎

Proof of Lemma 4.3.

The assertion follows immediately from (4.1), Corollary 4.2 and Corollary 4.6. ∎

Proof of Proposition 1.10.

The proposition is immediate from Lemmas 4.2 and 4.3. ∎

4.3. Proof of Proposition 2.6

The random matrix $\boldsymbol{A}_{n}$ has $n$ columns and ${\boldsymbol{m}}\,\sim\,{\rm Po}(dn/k)$ rows, with the column and row degrees drawn from the distributions $\boldsymbol{d}$ and $\boldsymbol{k}$. By comparison, $\boldsymbol{A}_{\varepsilon,n}$ has slightly fewer, namely ${\boldsymbol{m}}_{\varepsilon,n}\,\sim\,{\rm Po}((1-\varepsilon)dn/k)$ rows. One might therefore think that the proof of Proposition 2.6 is straightforward, as it appears that $\boldsymbol{A}_{n}$ is obtained from $\boldsymbol{A}_{\varepsilon,n}$ by simply adding another random ${\rm Po}(\varepsilon dn/k)$ rows. Since adding $O_{\varepsilon,n}(\varepsilon n)$ rows cannot reduce the nullity by more than $O_{\varepsilon,n}(\varepsilon n)$, the bound on $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{n})]-\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{\varepsilon,n})]$ appears to be immediate. But there is a catch. Namely, in constructing $\boldsymbol{A}_{n}$ we condition on the event $\mathcal{D}=\{\sum_{i=1}^{n}\boldsymbol{d}_{i}=\sum_{i=1}^{{\boldsymbol{m}}}\boldsymbol{k}_{i}\}$. Thus, $\boldsymbol{A}_{\varepsilon,n}$ does not have the same distribution as the top ${\rm Bin}({\boldsymbol{m}},1-\varepsilon)$ rows of $\boldsymbol{A}_{n}$ since the conditioning might distort the degree distribution. We need to show that this distortion is insignificant. To this end, recall that ${\boldsymbol{m}}_{\varepsilon,n}\,\sim\,{\rm Po}((1-\varepsilon)dn/k)$.

Lemma 4.7**.**

A.a.s. we have

[TABLE]

Proof.

Lemma 1.11 shows that $\sum_{i=1}^{n}\boldsymbol{d}_{i},\sum_{i=1}^{{\boldsymbol{m}}_{\varepsilon,n}}\boldsymbol{k}_{i}=O_{\varepsilon,n}(n)$ and $\sum_{i=1}^{{\boldsymbol{m}}_{\varepsilon,n}}\boldsymbol{k}_{i}\leq\sum_{i=1}^{n}\boldsymbol{d}_{i}$ with probability $1-o_{n}(n^{-1})$. Assuming that this is so, consider a filtration $(\mathfrak{A}_{t})_{t\leq\sum_{i=1}^{{\boldsymbol{m}}_{\varepsilon,n}}\boldsymbol{k}_{i}}$ that reveals the random matching $\boldsymbol{\Gamma}_{\varepsilon,n}$ one edge at a time. Then

[TABLE]

for all $t$. Therefore, the assertion follows from Azuma’s inequality. ∎

Let $\boldsymbol{A}_{n,\mathcal{D}}$ be the conditional version of the random matrix $\boldsymbol{A}_{0,n}$ given $\mathcal{D}$. Thus, given $\sum_{i=1}^{n}\boldsymbol{d}_{i}=\sum_{i=1}^{{\boldsymbol{m}}}\boldsymbol{k}_{i}$, we construct a random Tanner multi-graph with variable degrees $\boldsymbol{d}_{1},\ldots,\boldsymbol{d}_{n}$ and check degrees $\boldsymbol{k}_{1},\ldots,\boldsymbol{k}_{{\boldsymbol{m}}}$. Hence, the difference between $\boldsymbol{A}_{n}$ and $\boldsymbol{A}_{n,\mathcal{D}}$ is merely that in the case of $\boldsymbol{A}_{n}$ we also condition on the event $\mathcal{S}$ that the Tanner graph is simple.

Lemma 4.8**.**

There exists a coupling of $\boldsymbol{A}_{n,\mathcal{D}}$ and $\boldsymbol{A}_{\varepsilon,n}$ such that with probability at least $1-\varepsilon$ the two matrices agree in all but $O_{\varepsilon,n}(\varepsilon n)$ rows.

Proof.

Let $\boldsymbol{G}_{n,\mathcal{D}}$ and $\boldsymbol{G}_{\varepsilon,n}$ denote the Tanner graphs corresponding to $\boldsymbol{A}_{n,\mathcal{D}}$ and $\boldsymbol{A}_{\varepsilon,n}$, respectively. It suffices to construct a coupling of $\boldsymbol{G}_{n,\mathcal{D}}$ and $\boldsymbol{G}_{\varepsilon,n}$ such that these graphs differ in edges incident with at most $O_{\varepsilon,n}(\varepsilon n)$ check nodes. To construct the coupling we first generate the following parameters for $\boldsymbol{A}_{\varepsilon,n}$. Parameter $\mathcal{T}=\mathcal{T}(\varepsilon)$ is given. Generate ${\boldsymbol{\theta}}\in[\mathcal{T}]$ uniformly at random. Then generate ${\boldsymbol{m}}\sim{\rm Po}(dn/k)$ and ${\boldsymbol{m}}_{\varepsilon,n}={\rm Bin}({\boldsymbol{m}},1-\varepsilon)$ and then check nodes $a_{1},\ldots,a_{{\boldsymbol{m}}_{\varepsilon,n}}$. Each check node $a_{i}$ is associated with an integer $\boldsymbol{k}_{i}$ which is an independent copy of $\boldsymbol{k}$. To distinguish $\boldsymbol{G}_{\varepsilon,n}$ from $\boldsymbol{G}_{n,\mathcal{D}}$, we colour these check nodes red. Add ${\boldsymbol{\theta}}$ check nodes $p_{1},\ldots,p_{{\boldsymbol{\theta}}}$ to both $\boldsymbol{G}_{\varepsilon,n}$ and $\boldsymbol{G}_{n,\mathcal{D}}$.

Next generate $n$ variable nodes where variable node $x_{i}$ is associated with ${\boldsymbol{d}}_{i}$, which is an independent copy of ${\boldsymbol{d}}$. Further, let $\boldsymbol{r}_{j}=\sum_{h=1}^{{\boldsymbol{m}}}\boldsymbol{1}\left\{{\boldsymbol{k}_{h}=j}\right\}$ denote the prospective number of checks of $\boldsymbol{G}_{n,\mathcal{D}}$ of degree $j$. Applying Azuma’s inequality and (4.1), we see that for any $\varepsilon>0$ there exists $L=L(\varepsilon)>0$ such that

[TABLE]

Hence, Lemma 4.2 implies that

[TABLE]

Now condition on the event

[TABLE]

Uncolour all (red) check nodes $a_{i}$ with $\boldsymbol{k}_{i}\leq L$. Moreover, for each $j\leq L$, generate $\boldsymbol{r}_{j}-\sum_{i=1}^{{\boldsymbol{m}}_{\varepsilon,n}}\boldsymbol{1}\{\boldsymbol{k}_{i}=j\}$ additional check nodes of degree $j$ and colour them blue. Finally, for each $j>L$, generate $\boldsymbol{r}_{j}$ blue check nodes of degree $j$.

Now $\boldsymbol{G}_{\varepsilon,n}$ is generated by taking a random maximal matching from the clones of all uncoloured and red check nodes $\{a_{i}\}\times[\boldsymbol{k}_{i}]$ (excluding check nodes $p_{1},\ldots,p_{{\boldsymbol{\theta}}}$) to the set of variable clones

[TABLE]

and then adding an edge between $p_{i}$ and $x_{i}$ for $1\leq i\leq{\boldsymbol{\theta}}$. The Tanner graph $\boldsymbol{G}_{n,\mathcal{D}}$ is generated by removing all matching edges from the clones of the red check nodes, and removing edges between $p_{i}$ and $a_{i}$ for $1\leq i\leq{\boldsymbol{\theta}}$, and then matching all clones of the blue check nodes to the remaining clones of the variable nodes. Finally, (4.14) ensures that with probability at least $1-\varepsilon$, the two Tanner graphs differ in no more than $O_{\varepsilon,n}(\varepsilon n)$ check nodes. ∎

Proof of Proposition 2.6.

Assume that (2.8) is satisfied for $C>0$ and fix $C^{\prime}>C$ and a small enough $\delta>0$. Then we find a small $0<\varepsilon=\varepsilon(\delta)<\delta$ such that

[TABLE]

Hence, combining Lemmas 4.7 and 4.8 and taking into account that changing a single row can alter the nullity by at most one, we conclude that

[TABLE]

Finally, combining (4.15) and Lemma 4.3, we conclude that

[TABLE]

provided that $\varepsilon=\varepsilon(\delta)$ is small enough. Since $\boldsymbol{A}_{n,\mathcal{D}}$ given $\mathcal{S}$ is identical to $\boldsymbol{A}_{n}$, the desired upper bound on the nullity of $\boldsymbol{A}_{n}$ follows from (4.16). The same argument renders the lower bound. ∎

5. The Aizenman-Sims-Starr scheme: proof of Proposition 2.7

In this section we prove Proposition 2.7. As set out in Section 2.2, we are going to bound the difference of the nullities of $\boldsymbol{A}_{\varepsilon,n+1}$ and $\boldsymbol{A}_{\varepsilon,n}$ via Proposition 2.4 and Lemma 2.5. This requires a coupling of the random variables $\operatorname{nul}(\boldsymbol{A}_{\varepsilon,n+1})$ and $\operatorname{nul}(\boldsymbol{A}_{\varepsilon,n})$.

5.1. The coupling

We begin by introducing a more fine-grained description of the random matrices $\boldsymbol{A}_{\varepsilon,n}$ and $\boldsymbol{A}_{\varepsilon,n+1}$ to facilitate the construction of the coupling. To this end, let $\boldsymbol{M}=(\boldsymbol{M}_{j})_{j\geq 1}$ and ${\boldsymbol{\Delta}}=({\boldsymbol{\Delta}}_{j})_{j\geq 1}$ be sequences of Poisson variables with means

[TABLE]

All of these random variables are mutually independent and independent of $\boldsymbol{\theta}$ and the $(\boldsymbol{d}_{i})_{i\geq 1}$. Further, let

[TABLE]

Since $\sum_{j\geq 1}\boldsymbol{M}_{j}\,\sim\,{\rm Po}((1-\varepsilon)dn/k)$, (5.2) is consistent with the earlier convention that ${\boldsymbol{m}}_{\varepsilon,n}\,\sim\,{\rm Po}((1-\varepsilon)dn/k)$.

The random vectors $(\boldsymbol{d}_{1},\ldots\boldsymbol{d}_{n}),\boldsymbol{M}$ naturally define a random Tanner (multi-)graph $\boldsymbol{G}_{n,\boldsymbol{M}}$ with variable nodes $x_{1},\ldots,x_{n}$ and check nodes $p_{1},\ldots,p_{\boldsymbol{\theta}}$ and $a_{i,j}$, $i\geq 1$, $j\in[\boldsymbol{M}_{i}]$. Its edges are induced by a random maximal matching $\boldsymbol{\Gamma}_{n,\boldsymbol{M}}$ of the complete bipartite graph with vertex classes

[TABLE]

Each matching edge $(x_{h},s,a_{i,j},t)\in\boldsymbol{\Gamma}_{n,\boldsymbol{M}}$ induces an edge between $x_{h}$ and $a_{i,j}$ in the Tanner graph. In addition, there is an edge between $p_{i}$ and $x_{i}$ for every $i\in[\boldsymbol{\theta}]$.

To define the random matrix $\boldsymbol{A}_{n,\boldsymbol{M}}$ to go with $\boldsymbol{G}_{n,\boldsymbol{M}}$, let $\chi:[0,1]^{2}\to\mathbb{F}^{*}$ be a measurable map and let $(\boldsymbol{\zeta}_{i,j},\boldsymbol{\xi}_{i})_{i,j\geq 1}$ be uniformly distributed on $[0,1]$, mutually independent and independent of all other randomness.777Unfortunately at this point there does not seem to be an ideal notation for the matrix and its entries. Because the random vector $\boldsymbol{M}$ depends on $n$ and to preserve the analogy with common random graphs notation, we denote the random Tanner graph by $\boldsymbol{G}_{n,\boldsymbol{M}}$ and its associated random matrix by $\boldsymbol{A}_{n,\boldsymbol{M}}$. At the same time, in line with linear algebra conventions, when indexing matrix entries we let the first index refer to the row of the matrix and the second index to the column. Since the variable $n$ nodes correspond to the columns, a degree of incoherence seems unavoidable. With the rows of $\boldsymbol{A}_{n,\boldsymbol{M}}$ indexed by the check nodes of $\boldsymbol{G}_{n,\boldsymbol{M}}$ and the columns indexed by the variable nodes, we define the matrix entries by letting

[TABLE]

The Tanner graph $\boldsymbol{G}_{n+1,\boldsymbol{M}^{+}}$ and its associated random matrix $\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}}$ are defined analogously.

Lemma 5.1**.**

For any $\theta>0$ we have $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{\varepsilon,n})]=\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{n,\boldsymbol{M}})]$, $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{\varepsilon,n+1})]=\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})]$.

Proof.

We defined $\boldsymbol{A}_{\varepsilon,n}$ as the ${\boldsymbol{m}}_{\varepsilon,n}\times n$-matrix with target column and row degrees drawn from $\boldsymbol{d}$ and $\boldsymbol{k}$ independently with a $\boldsymbol{\theta}\times\boldsymbol{\theta}$ identity matrix affixed at top. In effect, because ${\boldsymbol{m}}_{\varepsilon,n}$ is a Poisson variable, the number of rows of with target degree $i$ is distributed as $\boldsymbol{M}_{i}$, and these numbers are mutually independent. Hence, $\operatorname{nul}\boldsymbol{A}_{\varepsilon,n}$ and $\operatorname{nul}\boldsymbol{A}_{n,\boldsymbol{M}}$ are identically distributed. The same argument applies to $\boldsymbol{A}_{\varepsilon,n+1}$. ∎

Up to this point we merely introduced a new description of $\boldsymbol{A}_{\varepsilon,n}$ and $\boldsymbol{A}_{\varepsilon,n+1}$. To actually couple them we introduce a third random matrix whose nullity we can easily compare to $\operatorname{nul}(\boldsymbol{A}_{n,\boldsymbol{M}})$ and $\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})$. Specifically, let ${\boldsymbol{\gamma}}_{i}\geq 0$ be the number of checks $a_{i,j}$, $j\in[\boldsymbol{M}_{i}^{+}]$, adjacent to the last variable node $x_{n+1}$ in $\boldsymbol{G}_{n+1,\boldsymbol{M}^{+}}$. Also let ${\boldsymbol{\gamma}}=({\boldsymbol{\gamma}}_{i})_{i\geq 1}$ and set

[TABLE]

In (5.3) the max is necessary because potentially $\boldsymbol{{\boldsymbol{\gamma}}}_{i}$ might exceed $\boldsymbol{M}_{i}$ as $\boldsymbol{{\boldsymbol{\gamma}}}_{i}$ might include some of the “extra” $\boldsymbol{\Delta}_{i}$ checks included in $\boldsymbol{G}_{n+1,\boldsymbol{M}^{+}}$. Consider the random Tanner graph $\boldsymbol{G}^{\prime}=\boldsymbol{G}_{n,\boldsymbol{M}^{-}}$ induced by a random maximal matching $\boldsymbol{\Gamma}^{\prime}$ of the complete bipartite graph with vertex classes

[TABLE]

For each variable $x_{i}$, $i=1,\ldots,n$, let ${\mathcal{C}}$ be the set of clones from $\bigcup_{i\in[n]}\{x_{i}\}\times[\boldsymbol{d}_{i}]$ that $\boldsymbol{\Gamma}_{n,\boldsymbol{M}^{-}}$ leaves unmatched. We call the elements of ${\mathcal{C}}$ cavities.

Now, obtain the Tanner graph $\boldsymbol{G}^{\prime\prime}$ from $\boldsymbol{G}^{\prime}$ by adding new check nodes

[TABLE]

The new checks are joined by a random maximal matching $\boldsymbol{\Gamma}^{\prime\prime}$ of the complete bipartite graph with vertex classes

[TABLE]

i.e., for each matching edge we insert a corresponding variable-check edge.

Analogously, obtain $\boldsymbol{G}^{\prime\prime\prime}$ by adding one variable node $x_{n+1}$ as well as check nodes $a_{i,j}^{\prime\prime\prime}$, $i\geq 1$, $j\in[{\boldsymbol{\gamma}}_{i}]$ and $b_{i,j}^{\prime\prime\prime}$, $i\geq 1$, $j\in[\boldsymbol{M}_{i}^{+}-\boldsymbol{M}_{i}^{-}-{\boldsymbol{\gamma}}_{i}]$ to $\boldsymbol{G}^{\prime}$. The new checks are connected to $\boldsymbol{G}^{\prime}$ via a random maximal matching $\boldsymbol{\Gamma}^{\prime\prime\prime}$ of the complete bipartite graph with vertex classes

[TABLE]

For each matching edge we insert the corresponding variable-check edge and in addition each of the check nodes $a_{i,j}^{\prime\prime\prime}$ gets connected to $x_{n+1}$ by exactly one edge.

Finally, we introduce the random matrices $\boldsymbol{A}^{\prime},\boldsymbol{A}^{\prime\prime},\boldsymbol{A}^{\prime\prime\prime}$ whose non-zero entries represent the edges of $\boldsymbol{G}^{\prime},\boldsymbol{G}^{\prime\prime},\boldsymbol{G}^{\prime\prime\prime}$. Recalling that $(\boldsymbol{\zeta}_{i,j},\boldsymbol{\xi}_{i})_{i,j\geq 1}$ are uniform on $[0,1]$ and independent of everything else, we additionally introduce independent random variables $(\boldsymbol{\zeta}_{i,j}^{\prime},\boldsymbol{\zeta}_{i,j}^{\prime\prime})_{i,j\geq 1}$, also uniform on $[0,1]$. With the rows and columns indexed by check and variable nodes, respectively, we define

[TABLE]

In line with the strategy outlined in Section 2, this construction ensures that $\boldsymbol{A}^{\prime\prime}$ and $\boldsymbol{A}^{\prime\prime\prime}$ are obtained from $\boldsymbol{A}^{\prime}$ by adding a bounded expected number of rows and, in the case of $\boldsymbol{A}^{\prime\prime\prime}$, one column. The following lemma links $\boldsymbol{A}^{\prime\prime},\boldsymbol{A}^{\prime\prime\prime}$ to the random matrices $\boldsymbol{A}_{n,\boldsymbol{M}}$, $\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}}$ from the beginning of the section.

Lemma 5.2**.**

We have $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}^{\prime\prime})]=\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{n,\boldsymbol{M}})]+o_{n}(1)$ and $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})]=\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})]+o_{n}(1).$

The proof of Lemma 5.2, deferred to Section 5.5, is tedious but relatively straightforward.

As a next step we are going to calculate the differences $\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime})$ and $\operatorname{nul}(\boldsymbol{A}^{\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime})$. We obtain expressions of one parameter of $\boldsymbol{A}^{\prime}$, namely the fraction of cavities ‘frozen’ to zero. To be precise, a cavity $(x_{i},h)\in{\mathcal{C}}$ is frozen if $x_{i}\in\mathfrak{F}(\boldsymbol{A}^{\prime})$. Let $\mathcal{F}\subseteq{\mathcal{C}}$ be the set of all frozen cavities and define $\boldsymbol{\alpha}=|\mathcal{F}|/|{\mathcal{C}}|$; in the unlikely event that ${\mathcal{C}}=\emptyset$, we agree that $\boldsymbol{\alpha}=0$. In Sections 5.3 and 5.4 we are going to establish the following two estimates.

Lemma 5.3**.**

We have $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime})]=\mathbb{E}[D(1-K^{\prime}(\boldsymbol{\alpha})/k)+d(K^{\prime}(\boldsymbol{\alpha})+K(\boldsymbol{\alpha})-1)/k]-d+o_{\varepsilon}(1).$

Lemma 5.4**.**

We have $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}^{\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime})]=d\mathbb{E}[\boldsymbol{\alpha}K^{\prime}(\boldsymbol{\alpha})]/k-d+o_{\varepsilon}(1).$

We emphasise that the r.h.s. expressions in Lemmas 5.3 and 5.4 involve expectations on the random variable $\boldsymbol{\alpha}$. A key feature of the present argument is that we manage to avoid an analysis of $\boldsymbol{\alpha}$ altogether. This is because, as the following proof of Proposition 2.7 shows, we can just replace the difference of the expectations by the largest conceivable value.

Proof of Proposition 2.7.

Combining Lemmas 5.1 and 5.2, we see that

[TABLE]

Further, combining (5.5) with Lemmas 5.3 and 5.4, we obtain

[TABLE]

The proposition is an immediate consequence of (5.6). ∎

While proving Lemmas 5.3 and 5.4 in full detail requires a fair bit of work because we are dealing with very general degree distributions $\boldsymbol{d},\boldsymbol{k}$, it is not at all difficult to fathom where the right hand side expressions come from. They actually arise naturally from Lemma 2.5 and the scarcity of short proper relations supplied by Proposition 2.4. Indeed, we can write the matrices $\boldsymbol{A}^{\prime\prime}$, $\boldsymbol{A}^{\prime\prime\prime}$ in the form

[TABLE]

with $\boldsymbol{B},(\boldsymbol{B}^{\prime}\ \boldsymbol{C}^{\prime})$ representing the new rows and, in the case of $\boldsymbol{A}^{\prime\prime\prime}$, the additional column. To calculate $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}^{\prime\prime}[\boldsymbol{\theta}])-\operatorname{nul}(\boldsymbol{A}^{\prime}[\boldsymbol{\theta}])]$ we basically need to assess the impact of adding a few more checks $a_{i,j}^{\prime\prime}$ to the Tanner graph $\boldsymbol{G}^{\prime}$ of $\boldsymbol{A}^{\prime}$. The new checks connect to randomly chosen cavities of $\boldsymbol{A}^{\prime}$. Let $k_{1},\ldots,k_{L}$ denote the degrees of the new checks. Since the distribution $\boldsymbol{k}$ of the check degrees has a finite second moment, the total degree $k_{1}+\cdots+k_{L}$ is bounded a.a.s. The random matrix $\boldsymbol{B}$ therefore encodes the non-zero entries corresponding to the edges that connect the $a_{i,j}^{\prime}$ with the cavities of $\boldsymbol{A}^{\prime}$ where the new checks attach. Furthermore, the construction of $\boldsymbol{A}^{\prime}$ ensures that a.a.s. the number of cavities is as large as $(1+o_{n}(1))\varepsilon dn$, and the $a_{i,j}^{\prime}$ hatch on to randomly chosen cavities. Therefore, Proposition 2.4, applied with $\mathcal{T}=\mathcal{T}(\varepsilon)$ large enough, implies that the probability that the set $I$ of non-zero columns of $\boldsymbol{B}$ forms a proper relation of $\boldsymbol{A}^{\prime}$ is $o_{\varepsilon}(1)$. Consequently, Lemma 2.5 yields

[TABLE]

where $\boldsymbol{B}_{*}$ is obtained from $\boldsymbol{B}$ by zeroing out all columns indexed by $\mathfrak{F}(\boldsymbol{A}^{\prime})$. Further, since the number of cavities of $\boldsymbol{A}^{\prime}$ is as large as $\Omega_{n}(n)$ while $k_{1}+\cdots+k_{L}=o_{n}(\sqrt{n})$ a.a.s., the matrix $\boldsymbol{B}$ has the following form a.a.s.: there are $L$ rows containing $k_{1},\ldots,k_{L}$ non-zero entries, respectively, and every column of $\boldsymbol{B}$ contains at most one non-zero entry. Consequently, once more because there are as many as $\Omega_{n}(n)$ cavities out of which an $\boldsymbol{\alpha}$ fraction are frozen to zero, $\boldsymbol{B}_{*}$ is close in total variation to the matrix obtained from $\boldsymbol{B}$ by zeroing out every column with probability $\boldsymbol{\alpha}$ independently. In effect, the probability that the $i$-th row of $\boldsymbol{B}_{*}$ gets zeroed out entirely is $\boldsymbol{\alpha}^{k_{i}}+o_{n}(1)$. Thus, a.a.s. we have

[TABLE]

Substituting (5.9) into (5.8) and the correct distribution of $k_{1},\ldots,k_{L}$ supplied by the coupling into (5.9), we obtain the expression displayed in Lemma 5.4. To be explicit, the correct degrees $k_{1},\ldots,k_{L}$ are provided by (5.4), i.e., there are $\boldsymbol{M}_{i}-\boldsymbol{M}_{i}^{-}$ checks of degree $i$ for every $i$. Hence, to obtain the expression in Lemma 5.3 we need to analyse the random variables $\boldsymbol{\gamma}_{i}$ from (5.3). This analysis will be conducted in Lemma 5.8 below, which shows that the $\boldsymbol{\gamma}_{i}$ are well approximated by the $\hat{\boldsymbol{\gamma}}_{i}$ from (5.15), which in turn come in terms of the reweighted check degree distribution from (1.10).

A similar but slightly more complicated calculation explains the expression in Lemma 5.3. We proceed to prove Lemmas 5.2–5.4 formally. This requires a bit of groundwork.

5.2. Groundwork

Let $P=P_{\boldsymbol{G}^{\prime}}$ be the distribution on the set $V_{n}=\left\{{x_{1},\ldots,x_{n}}\right\}$ of variables induced by choosing a cavity uniformly at random, i.e.,

[TABLE]

in the (unlikely) event that ${\mathcal{C}}=\emptyset$, we use the convention $P(x_{1})=1$. Let $\boldsymbol{x}_{1},\boldsymbol{x}_{2},\ldots\in V_{n}$ be independent samples drawn from $P$. The following lemma shows that $|{\mathcal{C}}|$ is linear in $n$ a.a.s.

Lemma 5.5**.**

A.a.s. we have $|{\mathcal{C}}|\geq\varepsilon dn/2$.

Proof.

The choice (5.1) of $\boldsymbol{M}$ ensures that $\mathbb{E}\sum_{j\geq 1}j\boldsymbol{M}_{j}=(1-\varepsilon)dn$. Moreover, because the $\boldsymbol{M}_{j}$ are mutually independent Poissons,

[TABLE]

Consequently, Chebyshev’s inequality shows that

[TABLE]

Similarly, we have $\mathbb{E}\sum_{i=1}^{n}\boldsymbol{d}_{i}=dn$ and $\mathrm{Var}\sum_{i=1}^{n}\boldsymbol{d}_{i}=\sum_{i=1}^{n}\mathrm{Var}(\boldsymbol{d})=O_{\varepsilon,n}(n),$ whence

[TABLE]

Since $|{\mathcal{C}}|\geq\sum_{i=1}^{n}\boldsymbol{d}_{i}-\sum_{j\geq 1}j\boldsymbol{M}_{j}$ by construction, the assertion follows from (5.10) and (5.11). ∎

Further, letting $\ell_{*}=\lceil\exp(1/\varepsilon^{4})\rceil$ and $\delta_{*}=\exp(-1/\varepsilon^{4})$, consider the event

[TABLE]

The following simple lemma is an application of Proposition 2.4.

Lemma 5.6**.**

For sufficiently large $\mathcal{T}=\mathcal{T}(\varepsilon)>0$ we have ${\mathbb{P}}\left[{\boldsymbol{A}^{\prime}\in{\mathcal{E}}}\right]>\exp(-1/\varepsilon^{4})$.

Proof.

Lemma 5.5 provides that $|{\mathcal{C}}|\geq\varepsilon n/2$ a.a.s. Moreover, since $\mathbb{E}[\boldsymbol{d}]=O_{\varepsilon,n}(1)$ we find $L=L(\varepsilon)>0$ such that the event $\mathcal{L}=\left\{{\sum_{i=1}^{n}\boldsymbol{d}_{i}\boldsymbol{1}\{\boldsymbol{d}_{i}>L\}<\varepsilon\delta_{*}^{2}n/(16\ell_{*})}\right\}$ has probability at least $1-\delta_{*}/8$. Thus, we may condition on the event $\mathcal{A}=\mathcal{L}\cap\{|{\mathcal{C}}|\geq\varepsilon n/2\}$.

Let $\hat{\boldsymbol{x}}_{1},\ldots,\hat{\boldsymbol{x}}_{\ell_{*}}$ be a sequence of independently and uniformly chosen variables from $x_{1},\ldots,x_{n}$. Consider a set $\mathcal{W}\subseteq\{x_{1},\ldots,x_{n}\}^{\ell_{*}}$. How can we estimate the probability that $(\boldsymbol{x}_{1},\ldots,\boldsymbol{x}_{\ell_{*}})\in\mathcal{W}?$ Either one of the variables $\boldsymbol{x}_{1},\ldots,\boldsymbol{x}_{\ell_{*}}$ has degree greater than $L$; on the event $\mathcal{A}$ this occurs with probability at most $\delta_{*}^{2}/16$. Or all of $\boldsymbol{x}_{1},\ldots,\boldsymbol{x}_{\ell_{*}}$ have degree at most $L$. Then the probability that $(\boldsymbol{x}_{1},\ldots,\boldsymbol{x}_{\ell_{*}})\in\mathcal{W}$ is not much greater than the probability that $(\hat{\boldsymbol{x}}_{1},\ldots,\hat{\boldsymbol{x}}_{\ell_{*}})\in\mathcal{W}$. To be precise, since $\hat{\boldsymbol{x}}_{1},\ldots,\hat{\boldsymbol{x}}_{\ell_{*}}$ are chosen uniformly and there are at least $\varepsilon n/2$ cavities, the probabilities differ by no more than a factor of $(2L/\varepsilon)^{\ell_{*}}$. Hence, on the event $\mathcal{A}$ we have

[TABLE]

Applying (5.13) to the set $\mathcal{W}$ of proper relations and invoking Proposition 2.4 completes the proof. ∎

Further, consider the event

[TABLE]

Lemma 5.7**.**

We have ${\mathbb{P}}\left[{{\mathcal{E}}^{\prime}}\right]=1-o_{n}(1).$

Proof.

This follows from the choice of the parameters in (5.1), Lemma 1.11 and Lemma 5.5. ∎

To prove Lemmas 5.3 and 5.4 we need an explicit description of the vector $\boldsymbol{\gamma}$ that captures the degrees of the checks adjacent to the new variable node $x_{n+1}$. Since $\boldsymbol{\gamma}$ is defined in terms of the the ‘big’ Tanner graph $\boldsymbol{G}_{n+1,\boldsymbol{M}^{+}}$, $\boldsymbol{\gamma}$ and the random variables are stochastically dependent. However, the next lemma shows that this dependence is very weak. Additionally, the lemma shows that the law of $\boldsymbol{\gamma}$ can be expressed easily in terms of the sequence $(\hat{\boldsymbol{k}}_{i})_{i\geq 1}$ of independent copies of $\hat{\boldsymbol{k}}$ from (1.10). Indeed, let

[TABLE]

Also let $\hat{\boldsymbol{\Delta}}=(\hat{\boldsymbol{\Delta}}_{j})_{j\geq 1}$ be a family random variables, mutually independent and independent of everything else, with distributions

[TABLE]

Further, let $\Sigma^{\prime}$ be the $\sigma$-algebra generated by $\boldsymbol{G}^{\prime}$, $\boldsymbol{A}^{\prime}$, $\boldsymbol{M}^{-}$ and $(\boldsymbol{d}_{i})_{i\in[n]}$. We write ${\boldsymbol{\gamma}}\mid\Sigma^{\prime},{\boldsymbol{\Delta}}\mid\Sigma^{\prime}$ for the conditional versions of ${\boldsymbol{\gamma}},{\boldsymbol{\Delta}}$ given $\Sigma^{\prime}$.

Lemma 5.8**.**

With probability $1-\exp(-\Omega_{\varepsilon,n}(1/\varepsilon))$ over the choice of $\boldsymbol{G}^{\prime}$, $\boldsymbol{A}^{\prime}$, $\boldsymbol{M}^{-}$ and $(\boldsymbol{d}_{i})_{i\in[n]}$ we have

[TABLE]

Proof.

We begin by studying the unconditional distributions of ${\boldsymbol{\gamma}}$ and ${\boldsymbol{\Delta}}$.

Let $\zeta=(\sum_{i\geq 1}i\boldsymbol{M}_{i}^{+})/(\sum_{i=1}^{n+1}\boldsymbol{d}_{i})$. Proceeding as in the proof of Lemma 5.5, we conclude that ${\mathbb{P}}\left[{1-2\varepsilon\leq\zeta\leq 1-\varepsilon/2}\right]=1-o_{n}(1)$. Further, given $1-2\varepsilon\leq\zeta\leq 1-\varepsilon/2$ we can think of $\boldsymbol{G}_{n+1,\boldsymbol{M}^{+}}$ as being generated by the following experiment.

1. (i)

Choose a set $\boldsymbol{C}\subseteq\bigcup_{h=1}^{n+1}\left\{{x_{h}}\right\}\times[\boldsymbol{d}_{h}]$ of size $(1-\zeta)\sum_{i=1}^{n+1}\boldsymbol{d}_{i}$ uniformly at random. 2. (ii)

Create a random perfect matching $\boldsymbol{\Gamma}^{\star}$ of the complete bipartite graph with vertex classes

[TABLE] 3. (iii)

Obtain $\boldsymbol{G}^{\star}$ with variable nodes $x_{1},\ldots,x_{n+1}$ and check nodes $a_{i,j}$, $i\geq 1$, $j\in[\boldsymbol{M}_{i}^{+}]$ by inserting an edge between $x_{h}$ and $a_{i,j}$ for any edge of $\boldsymbol{\Gamma}^{\star}$ that links $\{x_{h}\}\times[\boldsymbol{d}_{h}]$ to $\{a_{i,j}\}\times[i]$.

In other words, in the first step we designate the set of ${\mathcal{C}}=\boldsymbol{C}$ of cavities and in the next two steps we connect the non-cavities randomly.

By way of this alternative description we can easily get a grip on the degree of $x_{n+1}$. Indeed, given that $\boldsymbol{d}_{n+1}\leq\varepsilon^{-1/2}$, the probability that one of the clones $\{n+1\}\times[\boldsymbol{d}_{n+1}]$ ends up in $\boldsymbol{C}$ is $O_{\varepsilon}(\varepsilon^{1/2})$. Hence, the actual degree $\boldsymbol{d}^{\star}_{n+1}$ of $x_{n+1}$ in $\boldsymbol{G}^{\star}$ satisfies

[TABLE]

Regarding the degrees of the checks adjacent to $x_{n+1}$, by the principle of deferred decisions we can construct $\boldsymbol{\Gamma}^{\star}$ by matching one variable clone at a time, starting with the clones $\{x_{n+1}\}\times[\boldsymbol{d}_{n+1}]$. Clearly, in this process the probability that a specific clone of $x_{n+1}$ links to a specific check is proportional to the degree of that check. Therefore, since $\mathbb{E}\sum_{i\geq 1}i\boldsymbol{M}_{i}^{+}=O_{\varepsilon,n}(1)$, we find a fixed number $L$ such that with probability $1-O_{\varepsilon,n}(\varepsilon^{-1})$ all checks adjacent to $x_{n+1}$ have degree at most $L$. Further, Chebyshev’s inequality shows that $\boldsymbol{M}_{i}^{+}=(1-\varepsilon){\mathbb{P}}\left[{\boldsymbol{k}=i}\right]dn/k+o_{n}(n)$ for all $i\leq L$ and $\sum_{i\geq 1}i\boldsymbol{M}_{i}^{+}=(1-\varepsilon)dn+o_{n}(n)$ a.a.s. In effect, if $\boldsymbol{d}_{n+1}\leq\varepsilon^{-1/2}$, the $\boldsymbol{d}_{n+1}$ choices of the checks are asymptotically independent, and the distribution of the individual check degrees that $x_{n+1}$ joins is at total variation distance $o_{n}(1)$ of the distribution $\hat{\boldsymbol{k}}$. In summary, given $\boldsymbol{M}_{i}^{+}=(1-\varepsilon){\mathbb{P}}\left[{\boldsymbol{k}=i}\right]dn/k+o_{n}(n)$ for all $i\leq L$ and $\sum_{i\geq 1}i\boldsymbol{M}_{i}^{+}=(1-\varepsilon)dn+o_{n}(n)$ we have

[TABLE]

Moreover, it is immediate from (5.1) that the unconditional ${\boldsymbol{\Delta}}$ is distributed as $\hat{\boldsymbol{\Delta}}$.

To complete the proof we are going to argue that $\boldsymbol{M}^{-},\boldsymbol{d}_{1},\ldots,\boldsymbol{d}_{n}$ and ${\boldsymbol{\gamma}},{\boldsymbol{\Delta}}$ are asymptotically independent. Arguing along the lines of the previous paragraph, we find that for large $L=L(\varepsilon)>0$ the event

[TABLE]

occurs with probability ${\mathbb{P}}[\mathcal{K}]\geq 1-\exp(-1/\varepsilon^{2})$. Consequently, the event

[TABLE]

satisfies ${\mathbb{P}}[\mathcal{L}]\geq 1-\exp(-1/\varepsilon)$. Moreover, since $\boldsymbol{M}$ comprises independent Poisson variables, the event

[TABLE]

satisfies ${\mathbb{P}}[\mathcal{M}]=1-o_{n}(1)$. In summary,

[TABLE]

Further, we claim that for any outcomes $(M^{-},d_{1},\ldots,d_{n})\in\mathcal{L}\cap\mathcal{M}$ and $(\gamma,\Delta)\in\mathcal{K}$,

[TABLE]

Indeed, on the event $\mathcal{M}$ we have $\boldsymbol{M}_{i}^{-}=\mathbb{E}[\boldsymbol{M}_{i}]+O_{n}(\sqrt{n}\ln n)=\Omega_{n}(n)$ for any $i\leq L$ in the support of $\boldsymbol{k}$, the local limit theorem for the Poisson distribution yields

[TABLE]

Finally, given $\boldsymbol{M}=M^{-}+\gamma$ and $\boldsymbol{\Delta}=\Delta$ we have $\boldsymbol{M}_{i}^{+}=(1-\varepsilon){\mathbb{P}}\left[{\boldsymbol{k}=i}\right]dn/k+o_{n}(n)$ for all $i\leq L$ and $\sum_{i\geq 1}i\boldsymbol{M}_{i}^{+}=(1-\varepsilon)dn+o_{n}(n)$. Therefore, by the principle of deferred decisions, once we condition on a likely outcomes $M^{-}$ of $\boldsymbol{M}^{-}$, $\boldsymbol{d}_{1},\ldots,\boldsymbol{d}_{n}$ and of $\boldsymbol{\Delta}$, the conditional probability of obtaining $\boldsymbol{\gamma}=\gamma$ is close to the unconditional probability:

[TABLE]

Hence, (5.20) follows from (5.19) and (5.21).

Finally, to complete the proof we combine (5.19) and (5.20) to conclude that with probability $1-\exp(-\Omega_{\varepsilon,n}(1/\varepsilon))$,

[TABLE]

The assertion follows from (5.18) and (5.22). ∎

5.3. Proof of Lemma 5.3

The proof comprises several steps, each relatively simple individually. Let

[TABLE]

Then the total number of new non-zero entries upon going from $\boldsymbol{A}^{\prime}$ to $\boldsymbol{A}^{\prime\prime\prime}$ is bounded by $Y+Y^{\prime}$. Let

[TABLE]

Claim 5.9**.**

We have ${\mathbb{P}}\left[{{\mathcal{E}}^{\prime\prime}}\right]=1-O_{\varepsilon,n}(\varepsilon)$.

Proof.

Since (5.1) yields $\mathbb{E}[X],\mathbb{E}[Y]=O_{\varepsilon,n}(1)$, Markov’s inequality yields ${\mathbb{P}}\left[{X>1/\varepsilon}\right]=O_{\varepsilon,n}(\varepsilon)$ and ${\mathbb{P}}\left[{Y>1/\varepsilon}\right]=O_{\varepsilon,n}(\varepsilon)$. Further, we can bound the probability that a check of degree $i$ is adjacent to $x_{n+1}$ by $i\boldsymbol{d}_{n+1}/n$, because one of the $i$ clones of the check has to be matched to one of the $\boldsymbol{d}_{n+1}$ clones of $x_{n+1}$ and $\sum_{i=1}^{n}\boldsymbol{d}_{i}\geq n$. Hence,

[TABLE]

Thus, the assertion follows from Markov’s inequality. ∎

Going from $\boldsymbol{G}^{\prime}$ to $\boldsymbol{G}^{\prime\prime\prime}$ we add checks $a_{i,j}^{\prime\prime\prime}$, $i\geq 1$, $j\in[{\boldsymbol{\gamma}}_{i}]$ and $b_{i,j}^{\prime\prime\prime}$, $i\geq 1$, $j\in[\boldsymbol{M}_{i}^{+}-\boldsymbol{M}_{i}^{-}-{\boldsymbol{\gamma}}_{i}]$. Let

[TABLE]

comprise all the variable nodes adjacent to the new checks, except for $x_{n+1}$. Further, let

[TABLE]

be the event that the variables of $\boldsymbol{G}^{\prime}$ where the new checks attach are all distinct.

Claim 5.10**.**

We have ${\mathbb{P}}\left[{{\mathcal{E}}^{\prime\prime\prime}\mid{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}}\right]=1-o_{n}(1).$

Proof.

Given ${\mathcal{E}}^{\prime}$ there are $\Omega_{n}(n)$ cavities in total, while the maximum number belonging to any one variable is $O_{n}(\sqrt{n})$. Further, given ${\mathcal{E}}^{\prime\prime}$ we merely pick a bounded number $Y+Y^{\prime}=O_{\varepsilon,n}(1/\varepsilon)$ of these cavities randomly as neighbours of the new checks. Thus, the probability of hitting the same variable twice is $o_{n}(1)$. ∎

Claim 5.11**.**

We have $\mathbb{E}\left[{\left|{\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime})}\right|(1-\boldsymbol{1}{\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}\cap{\mathcal{E}}^{\prime\prime\prime})}\right]=o_{\varepsilon,n}(1)$.

Proof.

Clearly $\left|{\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime})}\right|\leq X+\boldsymbol{d}_{n+1}+1$ because going from $\boldsymbol{A}^{\prime}$ to $\boldsymbol{A}^{\prime\prime\prime}$ we add one column and at most $X+\boldsymbol{d}_{n+1}$ new rows. Consequently, as $\mathbb{E}[X^{2}],\mathbb{E}[\boldsymbol{d}_{n+1}^{2}]=O_{\varepsilon,n}(1)$, the Cauchy-Schwarz inequality yields

[TABLE]

Furthermore, Lemma 5.6 and Claims 5.7–5.10 readily imply that

[TABLE]

The assertion follows from (5.23)–(5.25). ∎

We obtain $\boldsymbol{G}^{\prime\prime\prime}$ by adding checks $a_{i,j}^{\prime\prime\prime}$ adjacent to $x_{n+1}$ and $b_{i,j}^{\prime\prime\prime}$ not adjacent to $x_{n+1}$. Recall that $\boldsymbol{\alpha}$ signifies the fraction of frozen cavities. Further, let $\Sigma^{\prime\prime}\supset\Sigma^{\prime}$ be the $\sigma$-algebra generated by $\boldsymbol{G}^{\prime}$, $\boldsymbol{A}^{\prime}$, $\boldsymbol{M}_{-}$, $(\boldsymbol{d}_{i})_{i\in[n+1]}$, ${\boldsymbol{\gamma}},\boldsymbol{M}$ and ${\boldsymbol{\Delta}}$. The random variable $\boldsymbol{\alpha}$ and the events ${\mathcal{E}},{\mathcal{E}}^{\prime},{\mathcal{E}}^{\prime\prime}$ are $\Sigma^{\prime\prime}$-measurable, but ${\mathcal{E}}^{\prime\prime\prime}$ is not. Indeed, given $\Sigma^{\prime\prime}$ the specific cavities of $\boldsymbol{G}^{\prime}$ that the new checks $a_{i,j}^{\prime\prime\prime},b_{i,j}^{\prime\prime\prime}$ join are still random.

Claim 5.12**.**

On the event ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}$ we have

[TABLE]

Proof.

Let

[TABLE]

be the set of all the new checks connected to $x_{n+1}$ and let

[TABLE]

be the set of all the new checks not connected to $x_{n+1}$. Let $\tilde{\boldsymbol{B}}$ be the $\left\{{0,1}\right\}$-matrix whose rows are indexed by $\mathcal{A}\cup\mathcal{B}$ and whose columns are indexed by $V_{n}=\{x_{1},\ldots,x_{n}\}$ such that for each $a\in\mathcal{A}\cup\mathcal{B}^{\prime}$ and each $x\in V_{n}$ the corresponding entry equals one iff $x\in\partial_{\boldsymbol{G}^{\prime\prime\prime}}a$. Further, obtain $\boldsymbol{B}$ from $\tilde{\boldsymbol{B}}$ by replacing each one-entry by the entry supplied by $\chi$ that represents the respective new edge of the Tanner graph. If the event ${\mathcal{E}}^{\prime\prime\prime}$ occurs, then each column of $B$ contains at most one non-zero entry and each row contains at least one non-zero entry. In effect, $\boldsymbol{B}$ has full rank, i.e.,

[TABLE]

Further, let $\boldsymbol{B}_{*}$ be the matrix obtained from $\boldsymbol{B}$ by replacing all entries in the $x$-column by zero for every $x\in\mathfrak{F}(\boldsymbol{A}^{\prime})$. Finally, let $\boldsymbol{C}\in\mathbb{F}^{\mathcal{A}\cup\mathcal{B}}$ be a column vector whose entries $\boldsymbol{C}_{a}$, $a\in\mathcal{A}$, are the entries from $\chi$ representing the edges of the Tanner graph $\boldsymbol{G}^{\prime\prime\prime}$ incident with $x_{n+1}$ and whose remaining entries $\boldsymbol{C}_{b}$, $b\in\mathcal{B}$, are equal to zero.

By construction, on the event ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}\cap{\mathcal{E}}^{\prime\prime\prime}$ we have

[TABLE]

Moreover, on ${\mathcal{E}}^{\prime}$ the set $\mathcal{X}^{\prime\prime\prime}$ of non-zero columns of $\boldsymbol{B}$ has size at most $|\mathcal{X}^{\prime\prime\prime}|\leq Y+Y^{\prime}\leq 2/\varepsilon$, while there are at least $\varepsilon dn/2$ cavities. As a consequence, even though the sequence of cavities that the new checks join are drawn without replacement, this sequence is at total variation distance $o_{n}(1)$ from a sequence of independent samples from the distribution $P$. Therefore, on ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}$ the conditional probability given ${\mathcal{E}}^{\prime\prime\prime}$ that $\mathcal{X}^{\prime\prime\prime}$ forms a proper relation is bounded by $O_{\varepsilon,n}(\exp(-1/\varepsilon^{4}))$. Hence, Lemma 2.5 implies that on ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}$,

[TABLE]

On ${\mathcal{E}}^{\prime\prime\prime}$ the matrix $\boldsymbol{Q}=\left({\boldsymbol{B}_{*}\ \boldsymbol{C}}\right)$ is a block matrix that decomposes into the $\mathcal{A}$-rows $\boldsymbol{Q}_{\mathcal{A}}$ and the $\mathcal{B}$-rows $\boldsymbol{Q}_{\mathcal{B}}$. Hence, $\operatorname{rk}(\boldsymbol{Q})=\operatorname{rk}(\boldsymbol{Q}_{\mathcal{A}})+\operatorname{rk}(\boldsymbol{Q}_{\mathcal{B}})$. To complete the proof, we claim that

[TABLE]

where, as we recall, $\boldsymbol{\alpha}$ is the probability that a cavity chosen from $p(\,\cdot\,)$ is frozen. Indeed, the probability that a $\mathcal{B}$-row of $\boldsymbol{B}$ that contains precisely $i$ non-zero entries gets zeroed out completely in $\boldsymbol{B}_{*}$ equals $\boldsymbol{\alpha}^{i}+o_{n}(1)$ and there are $\boldsymbol{M}_{i}^{+}-\boldsymbol{M}_{i}^{-}-{\boldsymbol{\gamma}}_{i}$ such rows; hence (5.27).

Similarly, the probability that an $\mathcal{A}$-row of $\boldsymbol{B}$ with $i-1$ non-zero entries gets zeroed out completely in $\boldsymbol{B}_{*}$ equals $\boldsymbol{\alpha}^{i-1}+o_{n}(1)$ and there are ${\boldsymbol{\gamma}}_{i}$ such rows. Hence, the expected rank of the $\mathcal{A}$-rows of $\boldsymbol{B}_{*}$ equals $\sum_{i\geq 1}\left({1-\boldsymbol{\alpha}^{i-1}}\right){\boldsymbol{\gamma}}_{i}+o_{n}(1)$, which is the first summand in (5.28). Moreover, the presence of the $\boldsymbol{C}$-column adds one to the rank of $\boldsymbol{Q}_{\mathcal{A}}$ unless not a single one of the $\mathcal{A}$-rows of $\boldsymbol{B}$ gets zero out, which occurs with probability $\prod_{i\geq 1}\left({1-\boldsymbol{\alpha}^{i-1}}\right)^{{\boldsymbol{\gamma}}_{i}}+o_{n}(1)$. Hence, we obtain (5.28). Finally, the assertion follows from (5.26)–(5.28). ∎

Proof of Lemma 5.3.

Let $\mathfrak{E}={\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}\cap{\mathcal{E}}^{\prime\prime\prime}$. Combining Claims 5.9–5.12, we see that

[TABLE]

[TABLE]

Since on $\mathfrak{E}$ all degrees $i$ with $\boldsymbol{M}_{i}^{+}-\boldsymbol{M}_{i}^{-}-{\boldsymbol{\gamma}}_{i}>0$ are bounded and Chebyshev’s inequality shows that $\boldsymbol{M}_{i}\sim\mathbb{E}[\boldsymbol{M}_{i}]=\Omega_{n}(n)$ for any fixed $i$ a.a.s., (5.3) yields $\boldsymbol{M}_{i}^{-}=\boldsymbol{M}_{i}-{\boldsymbol{\gamma}}_{i}$ a.a.s. Hence, (5.29) turns into

[TABLE]

Further, since $\sum_{i\geq 1}{\boldsymbol{\gamma}}_{i}\leq\boldsymbol{d}_{n+1}$ and $\mathbb{E}[\boldsymbol{d}_{n+1}]=O_{\varepsilon,n}(1)$, we obtain

[TABLE]

Similarly, Claim 5.9, Lemma 5.8 and the construction (5.16) of $\hat{\boldsymbol{\Delta}}$ yield

[TABLE]

Finally, the assertion follows from (5.30), (5.31) and (5.32). ∎

5.4. Proof of Lemma 5.4

The argument resembles the one from the proof of Lemma 5.3 but the details are considerably more straightforward as we merely add checks to obtain $\boldsymbol{A}^{\prime\prime}$ from $\boldsymbol{A}^{\prime}$. As before we consider the events ${\mathcal{E}},{\mathcal{E}}^{\prime}$ from (5.12) and (5.14) Moreover, recalling that the total number of new non-zero entries when going from $\boldsymbol{A}^{\prime}$ to $\boldsymbol{A}^{\prime\prime}$ is bounded by $\boldsymbol{d}_{n+1}$, we introduce ${\mathcal{E}}^{\prime\prime}=\left\{{\boldsymbol{d}_{n+1}\leq 1/\varepsilon}\right\}.$

Claim 5.13**.**

We have ${\mathbb{P}}\left[{{\mathcal{E}}^{\prime\prime}}\right]=1-O_{\varepsilon,n}(\varepsilon^{2}).$

Proof.

This follows from the assumption $\mathbb{E}[\boldsymbol{d}_{n+1}^{2}]=O_{\varepsilon,n}(1)$ and Chebyshev’s inequality. ∎

Further, similarly as in the proof of Lemma 5.3 we consider the set

[TABLE]

of variable nodes adjacent to the new checks. Let ${\mathcal{E}}^{\prime\prime\prime}$ be the event that none of the variable nodes in $\mathcal{X}$ is connected with the set of new checks by more than one edge.

Claim 5.14**.**

We have ${\mathbb{P}}\left[{{\mathcal{E}}^{\prime\prime\prime}\mid{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}}\right]=1-o_{n}(1).$

Proof.

Given ${\mathcal{E}}^{\prime}$ there are $\Omega_{n}(n)$ cavities in total, with each variable node contributing no more than $O_{n}(\sqrt{n})$ cavities. Moreover, given ${\mathcal{E}}^{\prime\prime}$ we choose $O_{\varepsilon,n}(1/\varepsilon)$ of cavities randomly to attach the new checks. Consequently, the probability of twice choosing a cavity with the same underlying variable is $o_{n}(1)$. ∎

Claim 5.15**.**

We have $\mathbb{E}\left[{\left|{\operatorname{nul}(\boldsymbol{A}^{\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime})}\right|(1-\boldsymbol{1}{\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}\cap{\mathcal{E}}^{\prime\prime\prime})}\right]=o_{\varepsilon,n}(1)$.

Proof.

We have $\left|{\operatorname{nul}(\boldsymbol{A}^{\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime})}\right|\leq\boldsymbol{d}_{n+1}$ as we add at most $\boldsymbol{d}_{n+1}$ rows. Because $\mathbb{E}[\boldsymbol{d}_{n+1}]=O_{\varepsilon,n}(1)$, Claim (5.13) and the Cauchy-Schwarz inequality yield

[TABLE]

Moreover, Lemma 5.6, Lemma 5.7 and Claim 5.14 show that

[TABLE]

[TABLE]

The assertion follows from (5.33) and(5.34). ∎

The matrix $\boldsymbol{A}^{\prime\prime}$ results from $\boldsymbol{A}^{\prime}$ by adding checks $a_{i,j}^{\prime\prime}$, $i\geq 1$, $j\in[\boldsymbol{M}_{i}-\boldsymbol{M}_{i}^{-}]$ that are connected to random cavities of $\boldsymbol{A}^{\prime}$. Moreover, as before let $\Sigma^{\prime\prime}\supset\Sigma^{\prime}$ be the $\sigma$-algebra generated by $\boldsymbol{G}^{\prime}$, $\boldsymbol{A}^{\prime}$, $\boldsymbol{M}_{-}$, $(\boldsymbol{d}_{i})_{i\in[n+1]}$, ${\boldsymbol{\gamma}},\boldsymbol{M}$ and ${\boldsymbol{\Delta}}$. Then ${\mathcal{E}},{\mathcal{E}}^{\prime},{\mathcal{E}}^{\prime\prime}$ are $\Sigma^{\prime\prime}$-measurable, but ${\mathcal{E}}^{\prime\prime\prime}$ is not.

Claim 5.16**.**

On the event ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}$ we have $\mathbb{E}\left[{(\operatorname{nul}(\boldsymbol{A}^{\prime\prime})-\operatorname{nul}(\boldsymbol{A}^{\prime}))\boldsymbol{1}{\mathcal{E}}^{\prime\prime\prime}\mid{\Sigma^{\prime\prime}}}\right]=o_{\varepsilon,n}(1)-\sum_{i\geq 1}(1-\boldsymbol{\alpha}^{i})(\boldsymbol{M}_{i}-\boldsymbol{M}_{i}^{-}).$

Proof.

Let $\mathcal{A}$ be the set of all the new checks $a_{i,j}^{\prime\prime}$, $i\geq 1$, $j\in[\boldsymbol{M}_{i}-\boldsymbol{M}_{i}^{-}]$. Let $\tilde{\boldsymbol{B}}$ be the $\left\{{0,1}\right\}$-matrix whose rows are indexed by $\mathcal{A}$ and whose columns are indexed by $V_{n}=\{x_{1},\ldots,x_{n}\}$ such that for each $a\in\mathcal{A}$ and each $x\in V_{n}$ the corresponding entry equals one iff $x\in\partial_{\boldsymbol{G}^{\prime\prime}}a$. Further, obtain $\boldsymbol{B}$ by substituting each one-entry of $\tilde{\boldsymbol{B}}$ by the appropriate non-zero field element from $\chi$. If ${\mathcal{E}}^{\prime\prime\prime}$ occurs, then $\boldsymbol{B}$ has rank $\operatorname{rk}(\boldsymbol{B})=|\mathcal{A}|=\sum_{i\geq 1}\boldsymbol{M}_{i}^{+}-\boldsymbol{M}_{i}$, because no column contains two non-zero entries and each row contains at least one non-zero entry. Further, let $\boldsymbol{B}_{*}$ be the matrix obtained from $\boldsymbol{B}$ by replacing all entries in the $x$-column by zero if $x\in\mathfrak{F}(\boldsymbol{A}^{\prime})$ is frozen to zero in $\boldsymbol{A}^{\prime}$.

On the event ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}\cap{\mathcal{E}}^{\prime\prime\prime}$ we have

[TABLE]

Moreover, on ${\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}$ the set $\mathcal{X}^{\prime\prime}$ of non-zero columns of $\boldsymbol{B}$ has size at most $|\mathcal{X}^{\prime\prime}|\leq\boldsymbol{d}_{n+1}\leq 1/\varepsilon$, while there are at least $\varepsilon dn/2$ cavities. Hence, on ${\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}\cap{\mathcal{E}}^{\prime\prime\prime}$ the probability that $\mathcal{X}^{\prime\prime}$ forms a proper relation is bounded by $\exp(-1/\varepsilon^{4})$. Therefore, Lemma 2.5 implies that

[TABLE]

Further, since an $\boldsymbol{\alpha}$-fraction of cavities are frozen, a row of $\boldsymbol{B}$ with $i$ non-zero entries gets zeroed out completely in $\boldsymbol{B}_{*}$ with probability $\boldsymbol{\alpha}^{i}+o_{n}(1)$. Consequently,

[TABLE]

Finally, the assertion follows from (5.36) and (5.37). ∎

Proof of Lemma 5.4.

Let $\mathfrak{E}={\mathcal{E}}\cap{\mathcal{E}}^{\prime}\cap{\mathcal{E}}^{\prime\prime}\cap{\mathcal{E}}^{\prime\prime\prime}$. Combining Claims 5.15–5.16, we obtain

[TABLE]

Since on $\mathfrak{E}$ all degrees $i$ with $\boldsymbol{M}_{i}^{+}-\boldsymbol{M}_{i}^{-}>0$ are bounded a.a.s. and $\boldsymbol{M}_{i}^{-}=\Omega_{n}(n)$ a.a.s., we conclude that $\boldsymbol{M}_{i}-\boldsymbol{M}_{i}^{-}={\boldsymbol{\gamma}}_{i}$ for all $i\geq 1$ a.a.s. Hence, (5.38) turns into

[TABLE]

Further, because $\sum_{i\geq 1}{\boldsymbol{\gamma}}_{i}\leq\boldsymbol{d}_{n+1}$ and $\mathbb{E}[\boldsymbol{d}_{n+1}]=O_{\varepsilon,n}(1)$,

[TABLE]

The assertion follows from (5.39) and (5.40). ∎

5.5. Proof of Lemma 5.2

Once more we break the proof down into a few relatively simple steps.

Claim 5.17**.**

We have $\mathbb{E}[\operatorname{nul}(\boldsymbol{A}^{\prime\prime})]=\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{n,\boldsymbol{M}})]+o_{n}(1)$.

Proof.

The choice of the random variables in (5.1) and Lemma 1.11 ensure that the event ${\mathcal{E}}=\{\sum_{i\geq 1}i\boldsymbol{M}_{i}\leq dn/k\}$ has probability $1-o_{n}(1/n)$. Further, given ${\mathcal{E}}$ the random variables $\operatorname{nul}(\boldsymbol{A}^{\prime\prime})$ and $\operatorname{nul}(\boldsymbol{A}_{n,\boldsymbol{M}})$ are identically distributed by the principle of deferred decisions. Because the nullity of either matrix is bounded by $n$ deterministically, the claim follows. ∎

To compare $\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})$ and $\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})$ we consider the event

[TABLE]

Claim 5.18**.**

We have ${\mathbb{P}}\left[{{\mathcal{E}}^{+}}\right]=1-o_{n}(1/n)$.

Proof.

This follows from the definition (5.2) of the random variables $\boldsymbol{M}_{i}^{+}$ and Lemma 1.11. ∎

Further, consider the event

[TABLE]

Claim 5.19**.**

We have ${\mathbb{P}}\left[{\mathcal{W}}\right]=1-o_{n}(1)$.

Proof.

This follows from the assumption that $\mathbb{E}[\boldsymbol{d}^{2}],\mathbb{E}[\boldsymbol{k}^{2}]$ are bounded. ∎

Moreover, let $\mathcal{U}$ be the event that $x_{n+1}$ does not partake in any multi-edges of $\boldsymbol{G}_{n,\boldsymbol{M}^{+}}$.

Claim 5.20**.**

We have ${\mathbb{P}}\left[{\mathcal{U}\mid\mathcal{W}\cap{\mathcal{E}}^{+}}\right]=1-o_{n}(\ln^{-6}n)$.

Proof.

Given $\mathcal{W}\cap{\mathcal{E}}^{+}$ variable node $x_{n+1}$ has target degree at most $\ln n$ and all check degrees are bounded by $n/\ln^{9}n$. Hence, the probability that $x_{n+1}$ joins the same check twice is $O_{n}(\ln^{-7}n)$. ∎

The next claim shows that $\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})$, $\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})$ can be coupled identically on the ‘bulk’ event ${\mathcal{E}}^{+}\cap\mathcal{U}\cap\mathcal{W}$.

Claim 5.21**.**

Given ${\mathcal{E}}^{+}\cap\mathcal{U}\cap\mathcal{W}$ the random variables $\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})$ and $\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})$ are identically distributed and thus

[TABLE]

Proof.

By construction, on ${\mathcal{E}}^{+}\cap\mathcal{U}\cap\mathcal{W}$ the random matrices $\boldsymbol{A}^{\prime\prime\prime}$ and $\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}}$ are identically distributed, and hence so are their nullities. ∎

In light of Claims 5.18 and 5.21 we are left to bound the difference of the nullities on ${\mathcal{E}}^{+}\setminus(\mathcal{U}\cap\mathcal{W})$.

Claim 5.22**.**

There is a coupling of $\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}}$ and $\boldsymbol{A}^{\prime\prime\prime}$ on ${\mathcal{E}}^{+}$ such that $\left|{\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})-\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})}\right|\leq 2\sum_{i\geq 1}i({\boldsymbol{\Delta}}_{i}+{\boldsymbol{\gamma}}_{i}).$

Proof.

We estimate the number of edges of the Tanner graph $\boldsymbol{G}_{n+1,\boldsymbol{M}^{+}}$ incident with the checks $a_{i,j}$, $\boldsymbol{M}_{i}^{-}<j\leq\boldsymbol{M}_{i}^{+}$ or the new variable $x_{n+1}$ of $\boldsymbol{G}_{n+1,\boldsymbol{M}^{+}}$. By construction, there are at most $\sum_{i\geq 1}i({\boldsymbol{\Delta}}_{i}+{\boldsymbol{\gamma}}_{i})$ such edges. Similarly, there are no more than $\sum_{i\geq 1}i({\boldsymbol{\Delta}}_{i}+{\boldsymbol{\gamma}}_{i})$ edges incident with the new checks $a_{i,j}^{\prime\prime\prime}$, $b_{i,j}^{\prime\prime\prime}$ added to $\boldsymbol{A}^{\prime}$ to obtain $\boldsymbol{A}^{\prime\prime\prime}$. By the principle of deferred decisions on ${\mathcal{E}}^{\prime\prime}$ we can couple the Tanner graphs of $\boldsymbol{A}^{\prime\prime\prime}$ and $\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}}$ such that they coincide on all the edges that join variables $x_{1},\ldots,x_{n}$ and checks $a_{i,j}$, $j\leq\boldsymbol{M}_{i}^{-}$, and hence the matrices themselves so that they coincide on all the corresponding matrix entries. Consequently, $\boldsymbol{A}^{\prime\prime\prime}$ and $\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}}$ differ in no more than $2\sum_{i\geq 1}i({\boldsymbol{\Delta}}_{i}+{\boldsymbol{\gamma}}_{i})$ entries. ∎

We proceed to bound the difference of the nullities on ${\mathcal{E}}^{+}\setminus\mathcal{W}$.

Claim 5.23**.**

We have $\mathbb{E}\left[{\sum_{i\geq 1}i({\boldsymbol{\Delta}}_{i}+{\boldsymbol{\gamma}}_{i})\boldsymbol{1}{{\mathcal{E}}^{+}\setminus\mathcal{W}}}\right]=o_{n}(1)$.

Proof.

The event ${\mathcal{E}}^{+}\setminus\mathcal{W}$ is contained in the union of the three events

[TABLE]

[TABLE]

To bound the contribution of $\mathcal{Q}_{1}$, consider ${\boldsymbol{m}}_{\varepsilon,n}^{+}=\sum_{i\geq 1}\boldsymbol{M}_{i}^{+}\,\sim\,{\rm Po}((1-\varepsilon)d(n+1)/k)$. We claim that, with the copies $(\boldsymbol{k}_{i})_{i\geq 1}$ of $\boldsymbol{k}$ independent of everything else,

[TABLE]

[TABLE]

Indeed, the last equality sign follows from the first because $\mathbb{E}[\boldsymbol{k}^{2}]=O_{n}(1)$ and the first equality sign follows because ${\boldsymbol{m}}_{\varepsilon,n}^{+}$ is independent of $\boldsymbol{d}_{n+1}$ and the $\boldsymbol{k}_{i}$. Further, to obtain the first inequality we consider the ${\boldsymbol{m}}_{\varepsilon,n}^{+}$ checks one by one. The degree of the $i$th check is distributed as $\boldsymbol{k}_{i}$. We discard it unless $\boldsymbol{k}_{i}\geq\log n$. But if $\boldsymbol{k}_{i}\geq\log n$, then the probability that $\boldsymbol{k}_{i}$ is adjacent to $x_{n+1}$ is bounded by $O_{n}(\boldsymbol{k}_{i}\boldsymbol{d}_{n+1}/\sum_{h=1}^{n+1}\boldsymbol{d}_{h})$ and $\sum_{h=1}^{n+1}\boldsymbol{d}_{h}\geq n$. Thus, we obtain (5.42). Further, we observe that (5.42) yields ${\mathbb{P}}\left[{\mathcal{Q}_{1}}\right]\leq\mathbb{E}\sum_{i\geq 1}i{\boldsymbol{\gamma}}_{i}\boldsymbol{1}\mathcal{Q}_{1}=O_{n}(\log^{-2}n).$ Hence, as $\mathbb{E}\sum_{i\geq 1}i{\boldsymbol{\Delta}}_{i}=O_{n}(1)$ we obtain

[TABLE]

Combining (5.42) and (5.43), we conclude that

[TABLE]

Regarding $\mathcal{Q}_{2}$, we deduce from the bound $\mathbb{E}[\boldsymbol{d}_{n+1}^{r}]=O_{n}(1)$ for an $r>2$ that

[TABLE]

Moreover, since the ${\boldsymbol{\Delta}}_{i}$ are independent of $\boldsymbol{d}_{n+1}$ and $\mathbb{E}\sum_{i\geq 1}i{\boldsymbol{\Delta}}_{i}=O_{n}(1)$, we obtain $\mathbb{E}\left[{\sum_{i\geq 1}i{\boldsymbol{\Delta}}_{i}\boldsymbol{1}\mathcal{Q}_{2}}\right]$

$=o_{n}(1)$. Hence, (5.45) yields

[TABLE]

Moving on to $\mathcal{Q}_{3}$ and recalling the definition (5.2) of $\boldsymbol{\Delta}$, we find

[TABLE]

Moreover, on $\mathcal{Q}_{3}$ we have $\sum_{i\geq 1}i{\boldsymbol{\gamma}}_{i}\leq\log^{2}n$ because $\boldsymbol{d}_{n+1}\leq\log n$ and ${\boldsymbol{\gamma}}_{i}=0$ for all $i\geq\log n$. Consequently, since the ${\boldsymbol{\Delta}}_{i}$ are mutually independent and $\sum_{i\geq 1}\mathbb{E}[i{\boldsymbol{\Delta}}_{i}]=O_{n}(1)$, (5.47) yields

[TABLE]

Finally, the assertion follows from (5.44), (5.46) and (5.48). ∎

Proof of Lemma 5.2.

The first assertion concerning $\boldsymbol{A}^{\prime\prime}$ and $\boldsymbol{A}_{n,\boldsymbol{M}}$ follows from Claim 5.17.

Concerning $\boldsymbol{A}^{\prime\prime\prime}$ and $\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}}$, Claim 5.18 shows that it suffices to couple $\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})\boldsymbol{1}{\mathcal{E}}^{+}$ and $\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})\boldsymbol{1}{\mathcal{E}}^{+}$, because both random variables are bounded by $n+1$. Indeed, thanks to Claim 5.21 we merely need to couple $\operatorname{nul}(\boldsymbol{A}^{\prime\prime\prime})\boldsymbol{1}{\mathcal{E}}^{+}\setminus(\mathcal{U}\cap\mathcal{W})$ and $\operatorname{nul}(\boldsymbol{A}_{n+1,\boldsymbol{M}^{+}})\boldsymbol{1}{\mathcal{E}}^{+}\setminus(\mathcal{U}\cap\mathcal{W})$, and Claim (5.22) supplies a coupling such that

[TABLE]

Hence, it suffices to show that

[TABLE]

Indeed, in light of Claim 5.23 we merely need to estimate $\sum_{i\geq 1}i({\boldsymbol{\Delta}}_{i}+{\boldsymbol{\gamma}}_{i})\boldsymbol{1}{\mathcal{E}}^{+}\cap\mathcal{W}\setminus\mathcal{U}$. But since on ${\mathcal{E}}^{+}\cap\mathcal{W}$ we have $\sum_{i\geq 1}i({\boldsymbol{\Delta}}_{i}+{\boldsymbol{\gamma}}_{i})\leq\ln^{4}n$, Claim 5.20 yields

[TABLE]

Finally, the assertion follows from Claim 5.23 and (5.49)–(5.51). ∎

6. Proof of Theorem 1.2

We describe how to extend the proof of [76] to ${\boldsymbol{G}}$. First, we will work on $G={\boldsymbol{G}}_{0,n}$ (defined in Section 2.2), the configuration model for ${\boldsymbol{G}}$. By Lemma 4.3, properties that holds with probability $1-o_{n}(1)$ for $G$ also hold with probability $1-o_{n}(1)$ for ${\boldsymbol{G}}$. Second, using the terminology in [76], variable nodes in $G$ are called vertices, and each check node corresponds to a hyperedge in the following sense: if $f_{a}$ is a check node adjacent with variable nodes $\{v_{a_{1}},\ldots,v_{a_{h}}\}$ for some $h\geq 1$, then the set of vertices $\{v_{a_{1}},\ldots,v_{a_{h}}\}$ is called a hyperedge. A check node with size 0 corresponds to an isolated hyperedge with size 0, i.e. this hyperedge does not contain any vertex.

Suppose ${\mathbb{P}}({\boldsymbol{d}}\geq 2)>0$ as otherwise the theorem holds trivially by Remark 1.3(b). We first prove Theorem 1.2 in the case $\boldsymbol{k}\geq 1$. Consider the parallel stripping process where all vertices of degree less than 2 are deleted in each step, together with the hyperedges (if any) incident with them. Take a random vertex $v\in[n]$. Let $\lambda_{t}$ be the probability that $v$ survives after $t$ iterations of the stripping process. It is easy to see that $\lambda_{t}$ is monotonically non-increasing and thus $\lambda=\lim_{t\to\infty}\lambda_{t}$ exists. For any vertex $u\in[n]$, let $\partial^{j}(u)$ denote the set of vertices of distance $j$ from $u$. Recall that there exists a constant $\sigma>0$ such that ${\mathbb{E}}{\boldsymbol{d}}^{2+\sigma}<\infty$ and ${\mathbb{E}}\boldsymbol{k}^{2+\sigma}<\infty$ by our assumptions on ${\boldsymbol{d}}$ and $\boldsymbol{k}$. We claim that

Claim 6.1**.**

*With high probability, the maximum degree and the maximum size of hyperedges in $G$ are at most

$(n\log n)^{1/(2+\sigma)}$, and for every $u\in[n]$ and for all fixed $R$, $|\cup_{j\leq R}\partial^{j}(u)|=O_{n}(n^{1/(2+\sigma)}\log^{2}n)$.*

Let $H_{t}$ be the subgraph of $G$ obtained after $t$ iterations of the parallel stripping process. Consider Doob’s martingale $({\mathbb{E}}(H_{t}\mid e_{1},\ldots,e_{j}))_{0\leq j\leq m}$ where random hyperedges are added in the order $e_{1},\ldots,e_{m}$ using the configuration model, and $m$ denotes the number of hyperedges in $G$. By Claim 6.1, swapping two clones in the configuration model would affect $H_{t}$ by $O_{n}(n^{1/(2+\sigma)}\log^{2}n)$, as each altered hyperedge can only affect the vertices (if surviving the first $t$-th iteration or not) within its $t$-neighbourhood. Standard concentration arguments (see, for instance, the proof of [89, Theorem 2.19]) based on Azuma’s inequality (with Lipschitz constant $Cn^{1/(2+\sigma)}\log^{2}n$ for some fixed $C>0$) produce that $||H_{t}|-\lambda_{t}n|=O_{n}(n^{(4+\sigma)/(4+2\sigma)}\log^{3}n)=o_{n}(n)$. Next we deduce an expression for $\lambda_{t}$. Consider a random hypertree $T$ iteratively built as follows. The root of $T$ is $v$, which is incident to $d_{v}$ hyperedges of size $\boldsymbol{k}_{1},\ldots,\boldsymbol{k}_{d_{v}}$ where the $\boldsymbol{k}_{i}$s are i.i.d. copies of $\hat{\boldsymbol{k}}$ where

[TABLE]

Then the $i$-th hyperedge is incident to other $\boldsymbol{k}_{i}-1$ vertices (other than $v$) whose degrees are i.i.d. copies of $\hat{\boldsymbol{d}}$, where

[TABLE]

This builds the first neighbourhood of $v$ in $T$. Iteratively we can build the $r$-neighbourhood of $v$ in $T$ for any fixed $r$. It follows from the following claim that the $r$-neighbourhood of $v$ in $G$ converges in distribution to the $r$-neighbourhood of $T$, as $n\to\infty$, for any fixed $r\geq 1$. This is because when uniformly picking a random variable clone (or check clone), the degree of the corresponding variable node (or check node) has the distribution in (6.2) (or (6.1)). Let $S$ be a set of vertices in $G$. We say $S$ induces a cycle if there is a closed walk $x_{0}x_{1}\ldots x_{\ell}=x_{0}$ such that all $x_{i}\in S$, and every pair of consecutive vertices in the walk are contained in a hyperedge in $G$.

Claim 6.2**.**

With high probability, for all fixed $R\geq 1$, $\cup_{j\leq R}\partial^{j}(v)$ induces no cycles.

(The proofs of Claim $6.1$ and $6.2$ can be found at the end of this section.) If $v$ survives $t$ iterations of the stripping process then at least two hyperedges incident with $v$ survives after $t$ iterations of the stripping process. On the other hand, let $x$ be a hyperedge of size at least 1 and let $u$ be a vertex incident with $x$. Let $\rho_{t}$ denote the probability that $u$ is incident with at least one hyperedge other than $x$ which survives after $t$ iterations of the stripping process. We will deduce a recursion for $\rho_{t}$ and then deduce $\lambda_{t}$ from $\rho_{t}$. Note that the degree of $u$ follows the distribution from (6.2). Then, ignoring an $o_{n}(1)$ error accounting for the probability of the complement of the events in Claims 6.1 and 6.2:

[TABLE]

and

[TABLE]

noting that

[TABLE]

Consequently,

[TABLE]

Let $g(x)=1-\frac{1}{d}D^{\prime}(1-\frac{K^{\prime}(x)}{k})$. Then $g^{\prime}(x)=\frac{1}{dk}D^{\prime\prime}(1-\frac{K^{\prime}(x)}{k})K^{\prime\prime}(x)$ which is non-negative over $[0,1]$. We also have $\phi(x)=g(x)-x$, where $\phi$ is given in (1.4). Since $\phi(1)=-D^{\prime}(0)/d\leq 0$, $\phi^{\prime}(\rho)<0$ by the hypothesis, and $g(x)$ is nondecreasing in $[0,1]$, it follows that $|g^{\prime}(\rho)|<1$ and thus $\rho$ is an attractive fix point of $x=g(x)$. As $\rho_{0}=1$. It follows that $\rho_{t}\to\rho$ as $t\to\infty$. Consequently, for every $\hat{\varepsilon}>0$ there is sufficiently large $I$ such that $|\rho_{t}-\rho|<\hat{\varepsilon}$. Hence, after $I$ iterations of the parallel stripping process, the number of vertices remaining is $(\lambda+o(1))n+O_{n}(\hat{\varepsilon}n)$ where

[TABLE]

If $\rho=0$ then $\lambda=0$ by Remark 1.3(c). Our theorem for ${\boldsymbol{n}}^{*}$ follows by letting $I\to\infty$. Since $\boldsymbol{k}\geq 1$, $K(0)=0$ and thus ${\boldsymbol{m}}^{*}/n=\frac{d}{k}K(0)+o_{\hat{\varepsilon},n}(1)=o_{\hat{\varepsilon},n}(1)$. This establishes (1.6) when $\rho=0$.

Suppose $\rho>0$. It is sufficient to show that the 2-core is obtained after further removing $O_{n}(\hat{\varepsilon}n)$ vertices, following the same approach as [76, Lemma 4]. We briefly sketch it. Following the same argument as before, the probability that a random vertex has degree $j\geq 2$ after $I$ iterations of the stripping process is

[TABLE]

Similarly, the probability of a uniformly random hyperedge in $G$ having size $j\geq 1$ and surviving the first $I$ iterations of the stripping process is

[TABLE]

The number of vertices with degree less than 2 after $I$ iterations is bounded by $(\lambda_{I}-\lambda_{I+1})n+o_{n}(n)$. Hence, by choosing $I$ sufficiently large, we can make these quantities arbitrarily close to those with $\rho_{I}$ replaced by $\rho$. Now standard concentration arguments apply to show that the number of degree $j\geq 2$ vertices is $\gamma_{j}n+O_{n}(\hat{\varepsilon}n)$, where

[TABLE]

the number of vertices of degree less than 2 is $O_{n}(\hat{\varepsilon}n)$. Similarly, the number of remaining hyperedges of size $j$ is ${\mathbb{P}}(\boldsymbol{k}=j)\rho^{j}m+O_{n}(\hat{\varepsilon}n)$, and the total degree of the remaining vertices is

[TABLE]

Note that $\hat{\varepsilon}$ can be made arbitrarily small by choosing sufficiently large $I$.

Now we remove one hyperedge incident with a vertex with degree 1 at a time. Call this process SLOWSTRIP. Let $G_{t}$ denote the hypergraph obtained after $t$ steps of SLOWSTRIP and let $X_{t}$ denote the total degree of the vertices of degree 1 in $G_{t}$, Then, for all $t=O_{n}(\hat{\varepsilon}n)$ such that $X_{t}>0$:

[TABLE]

Note that in the first equation above, $-1$ accounts for the removal of one variable clone $x$ from the set of vertices of degree less than 2. The term $j{\mathbb{P}}(\boldsymbol{k}=j)\rho^{j}m/\rho K^{\prime}(\rho)m$ approximates the probability that $x$ is contained in a hyperedge of size $j$, up to an $O_{n}(\hat{\varepsilon})$ error. In that case, $j-1$ variable clones that lie in the same hyperedge as $x$ will be removed. For each of these $j-1$ deleted variable clones, if it lies in a variable of degree 2, then it results in one new variable node of degree 1. The probability for that to happen is approximated by $2\gamma_{2}n/D_{t}$, up to an $O_{n}(\hat{\varepsilon})$ error, where $D_{t}$ denotes the total degree of $G_{t}$ and by (6.4), $D_{t}=(dn/k)\rho K^{\prime}(\rho)+O_{n}(\hat{\varepsilon}n)$. For the second equation above, note that

[TABLE]

By the assumption that {\color[rgb]{0,0,0}\phi}^{\prime}(\rho)<0 we have

[TABLE]

Hence, ${\mathbb{E}}(X_{t+1}-X_{t}\mid G_{t})<-\delta$ for some $\delta>0$, by making $\hat{\varepsilon}$ sufficiently small (i.e. by choosing sufficiently large $I$). Then the standard Azuma inequality [42, Lemma 29] (with Lipschitz constant $(n\log n)^{1/(2+\sigma)}$ by Claim (6.1) will be sufficient to show that $X_{t}$ decreases to 0 after $O_{n}(\hat{\varepsilon}n)=o_{\hat{\varepsilon},n}(n)$ steps (See details in [76, Lemma 4]). The case $\rho>0$ of the theorem follows by

[TABLE]

as desired. This proves (1.6) when $\boldsymbol{k}\geq 1$.

Suppose now that $p_{0}={\mathbb{P}}(\boldsymbol{k}=0)>0$. Let $\bar{G}$ be the hypergraph obtained from $G$ by deleting all hyperedges with size 0. Let $\bar{m}$ denote the number of hyperedges in $\bar{G}$. Then, with probability $1-o_{n}(1)$, $\bar{m}\sim(1-p_{0})m$. The size of a uniformly random hyperedge in $\bar{G}$ has the same distribution as $\bar{\boldsymbol{k}}$, defined by $\boldsymbol{k}$ conditioned on $\boldsymbol{k}\geq 1$. Let $\bar{k}={\mathbb{E}}\bar{\boldsymbol{k}}$. Then, $\bar{k}=k/(1-p_{0})$. Let $\bar{K}(\alpha)$ be the probability generating function of $\bar{\boldsymbol{k}}$. Then, immediately

[TABLE]

Let $\bar{\Phi}$ be the function obtained from $\Phi$ by replacing $K(\alpha)$, $K^{\prime}(\alpha)$ and $k$ by $\bar{K}(\alpha)$, $\bar{K}^{\prime}(\alpha)$ and $\bar{k}$ respectively. It is straightforward to see that the set of stable points of $\Phi$ corresponds to the set of stable points of $\bar{\Phi}$. Thus, by letting $\bar{\rho}=\max\{x\in[0,1]:\bar{\Phi}^{\prime}(x)=1\}$ it follows then that $\bar{\rho}=\rho$. Applying (1.6) with $\bar{\boldsymbol{k}}\geq 1$ to $\bar{G}$,

[TABLE]

where $\bar{\boldsymbol{n}}^{*}$ and $\bar{\boldsymbol{m}}^{*}$ denote the numbers of vertices and hyperedges in $\bar{G}$. Since ${\boldsymbol{n}}^{*}=\bar{\boldsymbol{n}}^{*}$ and ${\boldsymbol{m}}^{*}=\bar{\boldsymbol{m}}^{*}+(1+o_{n}(1))p_{0}m=(1+o_{n}(1))(\bar{\boldsymbol{m}}^{*}+p_{0}dn/k)$, as the set of hyperedges of size 0 in $G$ remain in the 2-core of $G$, the equations (1.6) holds as well for the case that $p_{0}>0$.∎

*Proof of Claim 6.1. * Since both ${\mathbb{E}}{\boldsymbol{d}}^{2+\sigma}=O_{n}(1)$ and ${\mathbb{E}}\boldsymbol{k}^{2+\sigma}=O_{n}(1)$, the probability that ${\boldsymbol{d}}>(n\log n)^{1/(2+\sigma)}$ or $\boldsymbol{k}>(n\log n)^{1/(2+\sigma)}$ is $O_{n}(1/n\log n)$. The bound on the maximum degree and maximum size of the hyperedges in ${\boldsymbol{G}}$ follows by taking the union bound.

For any $u\in[n]$, let $N_{i}(u)=|\partial^{i}(u)|$. We will prove that with high probability for every $u$ and for every fixed $i$, $N_{i}(u)=O_{n}(n^{1/(2+\sigma)}\log^{2}n)$, which then completes the proof for Claim 6.1. We prove by induction. Let $d_{1},\ldots,d_{N_{i}(u)}$ denote the degrees of the vertices in $\partial^{i}(u)$. Then the number of hyperedges incident with these vertices is bounded by $M:=\sum_{j=1}^{N_{i}(u)}d_{j}$. By the construction of ${\boldsymbol{G}}$, each $M$ is stochastically dominated by $\sum_{j=1}^{N_{i}(u)}(1+o_{n}(1))\hat{\boldsymbol{d}}_{j}$ where $\hat{\boldsymbol{d}}_{j}$ are i.i.d. copies of $\hat{\boldsymbol{d}}$ whose distribution is given in (6.2). The $o_{n}(1)$ error is caused by the exposure of $\cup_{j\leq i}\partial^{j}(u)$ which contains $o_{n}(n)$ vertices by induction. Since ${\mathbb{E}}{\boldsymbol{d}}^{2+\sigma}=O_{n}(1)$, we have $\hat{d}:={\mathbb{E}}\hat{\boldsymbol{d}}=O_{n}(1)$. Note that ${\mathbb{E}}M=\hat{d}N_{i}(u)$. Applying the Chernoff bound to the sum of independent $[0,1]$-valued random variables we have

[TABLE]

Similarly, $N_{i+1}(u)$ is bounded by $\sum_{j=1}^{M}k_{i}$, where $k_{i}$ are the sizes of the hyperedges incident with the vertices in $\partial^{i}(u)$. Similarly, $\sum_{j=1}^{M}k_{i}$ is stochastically dominated by $(1+o_{n}(1))\sum_{j=1}^{M}\hat{\boldsymbol{k}}_{j}$ where $\hat{\boldsymbol{k}}_{j}$ are i.i.d. copies of $\hat{\boldsymbol{k}}$ whose distribution is defined in (6.1). Let $\hat{k}={\mathbb{E}}\hat{\boldsymbol{k}}$. Applying the Chernoff bound again we obtain that with probability at least $1-n^{-2}$, $N_{i+1}(u)<2\hat{k}M+n^{1/(2+\sigma)}\log^{2}n<4\hat{d}\hat{k}N_{i}(u)+(1+2\hat{k})n^{1/(2+\sigma)}\log^{2}n$. Apply this recursion inductively and the union bound on the failure probability, we obtain $N_{i}(u)=O_{n}(n^{1/(2+\sigma)}\log^{2}n)$, as desired. ∎

*Proof of Claim 6.2. * Fix $\varepsilon>0$. Choose $L=L(\varepsilon,r)$ sufficiently large so that the probability that $d_{v}>L$ is smaller than $\varepsilon$ (note that $v$ is a uniformly random vertex). Given $d_{v}\leq L$. Let $k_{1},\ldots,k_{d_{v}}$ be the sizes of the hyperedges incident to $v$. Similarly to the proof of Claim 6.1, $k_{j}$s are approximated by i.i.d. copies of $\hat{\boldsymbol{k}}$ defined in (6.1), up to an $1+o(1)$ multiplicative error. We can assume $L$ is sufficiently large so that with probability at least $1-\varepsilon$, $\sum_{i=1}^{d_{v}}k_{i}\leq L$. Inductively, we can make $L$ sufficiently large so that $|\partial^{i}(v)|\leq L$ for all $i\leq R$. Let ${\mathcal{E}}_{i}$ denote the set of hyperedges incident with vertices in $\partial^{i}(v)$, but not incident with any in $\partial^{i-1}(v)$. Cycles in $\partial^{i}(v)$ can appear in two ways: (a) two vertices in $\partial^{i}(v)$ are incident with the same hyperedge in ${\mathcal{E}}_{i}$; (b) two hyperedges in ${\mathcal{E}}_{i-1}$ are incident with the same vertex in $\partial^{i}(v)$. We will prove that with high probability, none of the two cases occurs for any fixed $i$. For (a), let $(d_{j})_{j\in\partial^{i}(v)}$ denote the degrees of the vertices in $\partial^{i}(v)$. The expected number of occurrences of pairs of vertices in (a) is

[TABLE]

Note that $|\partial^{j}(v)|\leq L$ for each $j\leq R$. This immediately implies that $d_{j}\leq L$ for all $j\in\partial^{i}(v)$. Hence, the above probability is $O_{n}(n^{-1})$. The probability that $|\partial^{i}(v)|\leq L$ fails is at most $R\varepsilon$ by our choice of $L$. Hence, the probability that (a) fails is at most $R\varepsilon+o_{n}(1)$. The treatment of (b) is analogous. Our claim now follows by letting $\varepsilon\to 0$. ∎

7. Proof of Theorem 1.4

Recall that

[TABLE]

For Theorem 1.4 and Remark 1.5, it is sufficient to prove that if condition (i) or (ii) is satisfied then (a) $\max_{\alpha\in[0,1]}\Phi(\alpha)=\max\{\Phi(0),\Phi(\rho)\}$; and (b) $\phi^{\prime}(\rho)<0$ unless

[TABLE]

Since $\Phi(\alpha)$ is continuous on $[0,1]$, the maximum occurs at either 0 or 1 or at a stable point.

Case A: $\mathrm{Var}(\boldsymbol{k})=0$. In this case, $\boldsymbol{k}=k$ always and thus $K(\alpha)=\alpha^{k}$. We must have $k\geq 1$ since otherwise $k=d=0$. If $k=1$ then $\phi^{\prime}(\alpha)=-1$ which implies (b) immediately. Moreover, $K^{\prime\prime}(\alpha)=0$ for all $\alpha\in[0,1]$ and thus $\Phi^{\prime}(\alpha)=(d/k)K^{\prime\prime}(\alpha)\phi(\alpha)=0$ for all $\alpha\in[0,1]$. This implies (a).

Next consider the case that $k=2$. Then, $\phi^{\prime\prime}(\alpha)=-\frac{1}{d}D^{\prime\prime\prime}(1-\alpha)<0$ on $(0,1)$ unless ${\boldsymbol{d}}\leq 2$. Consider the case that ${\text{supp}}{{\boldsymbol{d}}}\cap{\mathbb{N}}_{\geq 3}\neq\emptyset$. Then $\phi$ is concave and can have at most 2 roots. Obviously $\alpha=0$ is a root. Let $\rho$ denote the other root if exists. We must have $\phi^{\prime}(\rho)<0$ by the concavity of $\phi$. Hence, the maximum of $\Phi$ cannot be achieved at 1. Thus, the maximas of $\Phi$ can only be from $\{0,\rho\}$. This verifies (a) and (b). Now assume ${\boldsymbol{d}}\leq 2$. Then $\phi^{\prime\prime}(\alpha)=0$ on $[0,1]$. Hence $\phi^{\prime}(\alpha)=\phi^{\prime}(1)=-1+{\mathbb{P}}({\boldsymbol{d}}=2)/d<0$ for all $\alpha\in[0,1]$. Thus, $\phi(\alpha)$ is a line with a negative slope and has exactly one root at [math] on $[0,1]$. Hence $\rho=0$ and $\phi^{\prime}(\rho)<0$. This verifies (a) and (b).

Next we consider the case that $k\geq 3$. We have

[TABLE]

Hence,

[TABLE]

Recall that $\Phi^{\prime}(\alpha)=\frac{d}{k}K^{\prime\prime}(\alpha)\phi(\alpha)$. We have $K^{\prime\prime}(\alpha)>0$ for all $\alpha\in(0,1]$. By (7.3) we have $\Phi^{\prime}(1)\leq 0$ and thus the supremum of $\Phi(\alpha)$ can only occur at 0 or a stable point. In all of the following sub-cases, we will prove that $\phi^{\prime\prime}(\alpha)$ has at most 1 root in $[0,1]$ (except for some trivial cases that we discuss separately). It follows immediately that $\phi$ can have at most three roots on $[0,1]$ including the trivial one at $\alpha=0$. Now we prove that this implies claims (a) and (b).

If $\phi$ has only a trivial root, then so is $\Phi^{\prime}(\alpha)$. Thus, $\alpha=0$ is the unique maxima of $\Phi(\alpha)$ and $\rho=0$. This verifies (a). As $\phi^{\prime}(0)=-1$ we immediately have $\phi^{\prime}(\rho)<0$.

If $\phi$ has two roots, then the larger root is $\rho$. Since $\phi^{\prime}(0)<0$, in this case, $\phi$ is negative in $(0,\rho)$ and positive in $(\rho,1)$. This is only possible when $\phi(1)=0$ by (7.3), which requires ${\mathbb{P}}({\boldsymbol{d}}=1)=0$. In this case, $\rho=1$. Next we consider two further cases: (i) $2(k-1){\mathbb{P}}({\boldsymbol{d}}=2)>d$ corresponding to $\phi^{\prime}(1)>0$; (ii) $2(k-1){\mathbb{P}}({\boldsymbol{d}}=2)<d$ corresponding to $\phi^{\prime}(1)<0$. As $\phi$ has only two roots, case (ii) obviously cannot happen. Thus, it means that the only situation that $\phi$ has two roots would be ${\mathbb{P}}({\boldsymbol{d}}=1)=0$ and $2(k-1){\mathbb{P}}({\boldsymbol{d}}=2)>d$, as in (7.2). In this situation we are only required to verify (a). Note that $\phi$ is negative in $(0,1)$ as $\rho=1$. It follows then that $\Phi(\alpha)$ is a decreasing function in $(0,1)$. Hence, $\alpha=0$ is the unique maxima, as desired.

If $\phi$ has three roots, then there is a root $\rho^{*}$ between [math] and $\rho$. Then $\phi$ is negative in $(0,\rho^{*})$ and positive in $(\rho^{*},\rho)$. As $K^{\prime\prime}(\alpha)>0$ for all $\alpha\in(0,1]$, the sign of $\phi$ implies that $\rho^{*}$ is a local minima and $\rho$ is a local maxima. This verifies (a). Moreover, as $\phi$ is positive in $(\rho^{*},\rho)$ and $\phi(\rho)=0$, $\phi^{\prime}(\rho)<0$ follows immediately.

Case A1: $\mathrm{Var}(\boldsymbol{k})=0$ and $\mathrm{Var}({\boldsymbol{d}})=0$. In this case ${\boldsymbol{d}}=d$. Then $D(\alpha)=\alpha^{d}$. If $d\geq 3$ then

[TABLE]

Obviously, $\phi^{\prime\prime}(\alpha)$ has a unique root in $[0,1]$.

If $d=1$ then $\phi^{\prime}(\alpha)=-1$ and so $\phi$ has only a trivial root at $\alpha=0$; If $d=2$ then $\phi^{\prime\prime}(\alpha)>0$ in $(0,1)$ and so $\phi$ is convex and thus has only a trivial root at $\alpha=0$ by (7.3). Hence for $d\leq 2$, $\rho=0$ and is the unique maxima. Claims (a) and (b) hold trivially.

Case A2: $\mathrm{Var}(\boldsymbol{k})=0$ and ${\boldsymbol{d}}\sim{\bf Po}_{\geq r}(\lambda)$. In this case $D(\alpha)=h_{r}(\lambda\alpha)/h_{r}(\lambda)$, where

[TABLE]

Then, for all integers $t$,

[TABLE]

Since ${\mathbb{E}}{\boldsymbol{d}}=d$, it requires that $\lambda$ satisfies

[TABLE]

Thus,

[TABLE]

Solving $\phi^{\prime\prime}(\alpha)=0$ yields

[TABLE]

The right hand side above is obviously a constant function if $r\leq 2$. If $r\geq 3$, then $h_{r-3}(\lambda t)-h_{r-2}(\lambda t)=(\lambda t)^{r-3}/(r-3)!$, and $h_{r-3}(\lambda t)$ is a power series of $\lambda t$ with minimum degree $r-3$. Hence, by dividing $(\lambda t)^{r-3}/(r-3)!$ from both the numerator and the denominator, we immediately get that the right hand side of (7.8) is an increasing function. However the left hand side of (7.8) is a decreasing function. Hence (7.8) has at most one solution, implying that $\phi^{\prime\prime}(\alpha)$ has at most one root.

Case B: $\boldsymbol{k}\sim{\bf Po}_{\geq s}(\gamma)$. We must have $\gamma$ satisfy

[TABLE]

so that ${\mathbb{E}}\boldsymbol{k}=k$. Here $k>s$ is required (to guarantee the existence of $\gamma$ if $s\geq 1$, and to avoid triviality if $s=0$). Now we have $K(\alpha)=h_{s}(\gamma\alpha)/h_{s}(\gamma)$, where $h_{s}$ is defined as in (7.5) and (7.6). Thus,

[TABLE]

Hence,

[TABLE]

As before, we will prove that $\phi^{\prime\prime}(\alpha)$ has at most 1 root in $[0,1]$ (except for some trivial cases that will be discussed separately), which is sufficient to ensure (a) and (b).

Case B1: $\boldsymbol{k}\sim{\bf Po}_{\geq s}(\gamma)$ and $\mathrm{Var}({\boldsymbol{d}})=0$. In this case ${\boldsymbol{d}}=d$. Then $D(\alpha)=\alpha^{d}$. If $d\geq 3$ then solving $\phi^{\prime\prime}(\alpha)=0$ yields

[TABLE]

On the right hand side above, $1-h_{s-1}(\gamma\alpha)/h_{s-1}(\gamma)\geq 0$ and is a decreasing function of $\alpha$. We also have

[TABLE]

which is positive and a decreasing function of $\alpha$ if $s\geq 3$, and is equal to 1 if $s\leq 2$. Hence, the left hand side of (7.11) is an increasing function whereas the right hand side is a decreasing function. Hence $\phi^{\prime\prime}(\alpha)$ has at most one root.

If $d\leq 2$ the same argument as in Case A1 shows that claims (a) and (b) hold.

Case B2: $\boldsymbol{k}\sim{\bf Po}_{\geq s}(\gamma)$ and ${\boldsymbol{d}}\sim{\bf Po}_{\geq r}(\lambda)$. In this case $D(\alpha)=h_{r}(\lambda\alpha)/h_{r}(\lambda)$, and $\lambda$ necessarily satisfies (7.7). Then solving $\phi^{\prime\prime}(\alpha)=0$ yields

[TABLE]

The left hand side is an increasing function whereas the right hand side is the product of two functions, both of which are either equal to 1 or a positive decreasing function. Thus, $\phi^{\prime\prime}(\alpha)$ has at most one root.

Acknowledgment

We thank David Saad for a helpful conversation and Guilhem Semerjian for bringing [63] to our attention.

Appendix A Proof of Lemma 1.11

Since $\mathbb{E}[\boldsymbol{\lambda}^{r}]<\infty$, the event $\mathcal{M}=\left\{{\max_{i\in[s]}\boldsymbol{\lambda}_{i}\leq n/\ln^{9}n}\right\}$ has probability

[TABLE]

Moreover, fixing a small enough $\eta=\eta(\delta)>0$ and a large enough $L=L(\eta)>0$ and setting $Q_{j}=\sum_{i\in[s]}\boldsymbol{1}\{\lambda_{i}=j\}$, we obtain from the Chernoff bound that ${\mathbb{P}}\left[{\forall j\leq L:|Q_{j}-s{\mathbb{P}}\left[{\boldsymbol{\lambda}=j}\right]|>\sqrt{n}\ln n}\right]=o_{n}(1/n)$. Hence, by Bayes’ rule,

[TABLE]

In addition, let ${\mathcal{H}}=\left\{{h\in\mathbb{N}:(1+\eta)^{h-1}L\leq n/\ln^{9}n}\right\}$ and for $h\in{\mathcal{H}}$ let

[TABLE]

[TABLE]

Then the Chernoff bound and Bayes’ rule yield

[TABLE]

Finally, given $\mathcal{M}$ and $|Q_{j}-s{\mathbb{P}}\left[{\boldsymbol{\lambda}=j}\right]|\leq\sqrt{n}\ln n$ for all $j\leq L$ and $\left|{R_{h}-\bar{R}_{h}}\right|\leq\eta\bar{R}_{h}+\ln^{2}n$ for all $h\in{\mathcal{H}}$, we obtain

[TABLE]

Similarly, $\frac{1}{s}\sum_{i=1}^{s}\boldsymbol{\lambda}_{i}\geq\mathbb{E}\left[{\boldsymbol{\lambda}\boldsymbol{1}}\right]-\delta/2+o_{n}(1)$. Thus, the assertion follows from (A.1)–(A.3)

Appendix B Stochastic vs. linear independence

A precursor of Proposition 2.4 for finite field was obtained in [7, Lemma 3.1]. Instead of dealing with linear independence, that statement dealt with stochastic dependencies. Formally, given an $m\times n$-matrix $A$ over a finite field $\mathbb{F}$, let $\mu_{A}$ be the probability distribution on $\mathbb{F}^{n}$ defined by

[TABLE]

(This definition is nonsensical over infinite fields for the obvious reason that $|\ker A|\in\{1,\infty\}$.) Let $\boldsymbol{\sigma}=\boldsymbol{\sigma}_{A}\in\mathbb{F}^{n}$ denote a sample from $\mu_{A}$. The stochastic independence statement reads as follows.

Lemma B.1** ([7, Lemma 3.1]).**

For any $\delta>0$, $\ell>0$ and for any finite field $\mathbb{F}$ there exists $\mathcal{T}=\mathcal{T}(\delta,\ell,\mathbb{F})>0$ such that for any matrix $A$ over $\mathbb{F}$ the following is true. Choose $\boldsymbol{\theta}\in[\mathcal{T}]$ uniformly at random. Then with probability at least $1-\delta$ the matrix $A[\boldsymbol{\theta}]$ satisfies

[TABLE]

In words, for most sets $I$ of $\ell$ coordinates the joint distribution of the coordinates $(\boldsymbol{\sigma}_{i})_{i\in I}$ is close to a product distribution in total variation distance. Furthermore, the number $\boldsymbol{\theta}$ of rows that we add to $A$ is bounded in terms of $\varepsilon,\ell$ only; i.e., $\boldsymbol{\theta}$ does not depend on the size $m\times n$ of $A$ or on the matrix $A$ itself. Lemma B.1 and its proof are inspired by the ‘pinning lemma’ from [24].

The following lemma shows that how Proposition 2.4 implies Lemma B.1; in a nutshell, the lemma states that linear independence is stronger than stochastic independence.

Lemma B.2**.**

Let $A$ be an $m\times n$-matrix over a finite field $\mathbb{F}$. Unless $I\subseteq[n]$ is a proper relation of $A$ we have

[TABLE]

Proof.

Since for every $\tau\in\mathbb{F}^{I}$ we have

[TABLE]

we may assume that $I\cap\mathfrak{F}(A)=\emptyset$ by simply passing on to $I\setminus\mathfrak{F}(A)$ if necessary. Hence, the task reduces to proving (B.2) under the assumption that $I\subseteq[n]\setminus\mathfrak{F}(A)$ is no relation of $A$.

To prove this statement let $N=\operatorname{nul}(A)$ and suppose that $\xi_{1},\ldots,\xi_{N}\in\mathbb{F}^{n}$ form a basis of $\ker A$. Let $\Xi\in\mathbb{F}^{n\times N}$ be the matrix with columns $\xi_{1},\ldots,\xi_{N}$ and let $\Xi_{1},\ldots,\Xi_{N}$ signify the rows of $\Xi$. The homomorphism $z\in\mathbb{F}^{N}\to\ker A$, $z\mapsto\Xi z$ maps the uniform distribution on $\mathbb{F}^{N}$ to the uniform distribution $\mu_{A}$ on $\ker A$. Therefore, to prove (B.2) it suffices to prove that the projection of this homomorphism to the $I$-rows, i.e., the map $z\in\mathbb{F}^{N}\mapsto(\Xi_{i}z)_{i\in I}$ is surjective. Equivalently, we need to show that

[TABLE]

Assume for contradiction that (B.3) is violated. Then there exists a vector $z\in\mathbb{F}^{I}\setminus\left\{{0}\right\}$ such that $\sum_{i\in I}z_{i}\Xi_{i}=0$. This implies that for all $x\in\mathbb{F}^{n}$,

[TABLE]

As a consequence, there exists a row vector $y$ of length $m$ such that $(yA)_{j}=\boldsymbol{1}\left\{{i\in I}\right\}z_{i}$ for all $j\in[n]$. Hence, $\emptyset\neq{\text{supp}}(yA)\subseteq I$. Thus $I$ is a relation of $A$, in contradiction to our assumption that it is not. ∎

Thus, Lemma B.1 is an immediate consequence of Proposition 2.4 and Lemma B.2. Indeed, the proof of Proposition 2.4 renders the explicit bound $\mathcal{T}=\lceil 4\ell^{3}/\delta^{4}\rceil+1$ on the number of coordinates that need to get pegged. By comparison, the stochastic approach via the arguments from [7, 10] leads to a value of $\mathcal{T}$ that is exponential in $\ell$ (although it may be possible to improve this estimate via probabilistic arguments).

Appendix C A self-contained proof of the upper bound on the rank

The ‘$\leq$’-inequality in (1.3) was previously proved by Lelarge [63], who derived the bound from the Leibniz determinant formula and the formula for the matching number of random bipartite graphs from [15]. The proof of that formula, however, is far from straightforward. Therefore, as a point of interest in this section we show that another idea from mathematical physics, the interpolation method from spin glass theory [38, 45], can be harnessed to obtain a self-contained proof of the upper bound on the rank. The proof uses similar ideas as the proof of the lower bound outlined in Section 2. Thus, phrased in terms of the nullity, the aim in this section is to show that a.a.s.

[TABLE]

C.1. The interpolation method

The basic idea behind the interpolation method is to construct a family of random matrices $\boldsymbol{A}_{\varepsilon}(t)$ parametrised by ‘time’ $t$. At $t={\boldsymbol{m}}_{\varepsilon,n}$ we obtain precisely the matrix $\boldsymbol{A}_{\varepsilon,n}$. At the other extreme, $\boldsymbol{A}_{\varepsilon}(0)$ is a block diagonal matrix whose nullity can be read off easily. To establish the lower bound we will control the change of the nullity with respect to $t$. By comparison to applications of the interpolation method to other combinatorial problems (e.g., [11, 24, 38, 79]), the construction here is relatively elegant. In particular, throughout the interpolation we will be dealing with an actual random matrix, rather than some other, more contrived object.

Getting down to the details, apart from $t$ and $\varepsilon$ we need two further parameters: an integer $\mathcal{T}=\mathcal{T}(\varepsilon)\geq 0$ and a real $\beta\in[0,1]$, which, in order to obtain the optimal bound, we choose such that

[TABLE]

Further, let ${\boldsymbol{m}}_{\varepsilon,n}\,\sim\,{\rm Po}((1-\varepsilon)dn/k)$. Also let $(\boldsymbol{k}_{i},\boldsymbol{k}_{i}^{\prime},\boldsymbol{k}_{i}^{\prime\prime})_{i\geq 1}$ and $(\boldsymbol{d}_{i})_{i\geq 1}$ be copies of $\boldsymbol{k}$ and $\boldsymbol{d}$, respectively, mutually independent and independent of ${\boldsymbol{m}}_{\varepsilon,n}$. Additionally, choose $\boldsymbol{\theta}\in[\mathcal{T}]$ uniformly and independently of everything else. Finally, recall that $(\boldsymbol{\zeta}_{i},\boldsymbol{\xi}_{i})_{i\geq 1}$ are uniformly distributed on the unit interval and independent of all other randomness.

The Tanner graph $\boldsymbol{G}_{\varepsilon}(t)$ has variable nodes

[TABLE]

Moreover, let $\mathcal{F}_{t}$ be a random set that contains each of the variable nodes $x_{i,j,h}$ with probability $\beta$ independently. Then the check nodes are

[TABLE]

To define the edges of the Tanner graph let $\boldsymbol{\Gamma}_{\varepsilon}(t)$ be a random maximal matching of the complete bipartite graph with vertex sets

[TABLE]

For each matching edge $\{(x_{i},s),(a_{j},t)\}\in\boldsymbol{\Gamma}_{\varepsilon}(t)$ insert an edge between $x_{i}$ and $a_{j}$ into the Tanner graph and for each $\{(x_{i},s),b_{j,h}\}\in\boldsymbol{\Gamma}_{\varepsilon}(t)$ insert an edge between $x_{i}$ and $b_{j,h}$. Thus, $\boldsymbol{G}_{\varepsilon}(t)$ may contain multi-edges. Further, add an edge between $x_{i}$ and $p_{i}$ for $i=1,\ldots,\boldsymbol{\theta}$ and add an edge between $x_{i,j,h}$ and $b_{i,j}$ for each $h\in[\boldsymbol{k}_{i}^{\prime}-1]$ as well as an edge between every $x_{i,j,h}\in\mathcal{F}_{t}$ and the check $f_{i,j,h}$. Finally, let $\boldsymbol{A}_{\varepsilon}(t)$ be the random matrix induced by $\boldsymbol{G}_{\varepsilon}(t)$. Formally, with the rows indexed by the check nodes and the columns indexed by the variable nodes, we let

[TABLE]

All other entries of $\boldsymbol{A}_{\varepsilon}(t)$ are equal to zero.

The semantics is as follows. The checks $a_{i}$ will play exactly the same role as before, i.e., each is adjacent to $\boldsymbol{k}_{i}$ of the variable nodes $x_{1},\ldots,x_{n}$ a.a.s. By contrast, each $b_{i,j}$ is adjacent to precisely one of the variables $x_{1},\ldots,x_{n}$. In addition, $b_{i,j}$ is adjacent to the $\boldsymbol{k}_{i}^{\prime}-1$ variable nodes $x_{i,j,h}$, $h\in[\boldsymbol{k}_{i}^{\prime}-1]$. These variable nodes, in turn, are adjacent only to $b_{i,j}$ and to $f_{i,j,h}$ if $x_{i,j,h}\in\mathcal{F}$. The checks $f_{i,j,h}$ are unary, i.e., $f_{i,j,h}$ simply forces $x_{i,j,h}$ to take the value zero. Finally, each of the checks $p_{i}$ is adjacent to $x_{i}$ only, i.e., $p_{1},\ldots,p_{\boldsymbol{\theta}}$ just freeze $x_{1},\ldots,x_{\boldsymbol{\theta}}$.

For $t=1$ the Tanner graph contains ${\boldsymbol{m}}_{\varepsilon,n}\,\sim\,{\rm Po}((1-\varepsilon)dn/k)$ ‘real’ checks $a_{i}$ and none of the checks $b_{i,j}$ or $f_{i,j,h}$. In effect, $\boldsymbol{A}_{\varepsilon}(1)$ is distributed precisely as $\boldsymbol{A}_{\varepsilon}$ from Section 2.2. By contrast, at $t=0$ there are no checks $a_{i}$ involving several of the variables $x_{1},\ldots,x_{n}$. As a consequence, the Tanner graph decomposes into $n$ connected components, one for each of the $x_{i}$. In fact, each component is a tree comprising $x_{i}$, some of the checks $b_{j,h}$ and their proprietary variables $x_{j,h,s}$ along with possibly a check $f_{j,h,s}$ that freezes $x_{j,h,s}$ to zero. For $i\in[\boldsymbol{\theta}]$ there is a check $p_{i}$ freezing $x_{i}$ to zero as well. Thus, $\boldsymbol{A}_{\varepsilon}(0)$ is a block diagonal matrix consisting of $n$ blocks, one for each component. In effect, the rank of $\boldsymbol{A}_{\varepsilon}(0)$ will be easy to compute. Finally, for $0<t<1$ we have a blend of the two extremal cases. There will be some checks $a_{i}$ and some $b_{i,j}$ with their retainer variables and checks; see Figure 2.

We are going to trace the nullity of $\boldsymbol{A}_{\varepsilon}(t)$ as $t$ increases. But since the newly introduced variables $x_{i,j,h}$ inflate the nullity, we subtract the ‘obvious’ correction term to retain the same scale throughout the process. In addition, we need a correction term to make up for the greater total number of check nodes in $\boldsymbol{A}_{\varepsilon}(0)$ by comparison to $\boldsymbol{A}_{\varepsilon}({\boldsymbol{m}}_{\varepsilon,n})$. Thus, let

[TABLE]

The following two statements summarise the interpolation argument. First, we compute $\mathbb{E}[\mathcal{N}_{0}]$.

Proposition C.1**.**

For any fixed $\theta\geq 0$ we have $n^{-1}\mathbb{E}[\mathcal{N}_{0}]=D(1-K^{\prime}(\beta)/k)+dK^{\prime}(\beta)/k-d+o_{\varepsilon,n}(1).$

The next proposition provides monotonicity.

Proposition C.2**.**

For any $\varepsilon>0$ there exists $\mathcal{T}=\mathcal{T}(\varepsilon)>0$ such that with probability $1-o_{n}(1/n)$ uniformly for all $0\leq t<{\boldsymbol{m}}_{\varepsilon,n}$ we have $\mathbb{E}[\mathcal{N}_{t+1}+\mathcal{Y}_{t+1}\mid{\boldsymbol{m}}_{\varepsilon,n}]\geq\mathbb{E}[\mathcal{N}_{t}+\mathcal{Y}_{t}\mid{\boldsymbol{m}}_{\varepsilon,n}]+o_{\varepsilon,n}(1).$

As an immediate consequence of Propositions C.1 and C.2 we obtain the desired lower bound on the nullity.

Corollary C.3**.**

We have $\frac{1}{n}\mathbb{E}[\operatorname{nul}(\boldsymbol{A}_{\varepsilon})]\geq\max_{\alpha\in[0,1]}\Phi(\alpha)+o_{\varepsilon,n}(1).$

Proof.

Proposition C.2 implies that

[TABLE]

Further, by Proposition C.1,

[TABLE]

Hence, (C.2) yields

[TABLE]

and the assertion follows from (C.3). ∎

Combining Proposition 2.6, Proposition 2.8 and Corollary C.3 and the standard concentration for $\operatorname{nul}{\boldsymbol{A}_{\varepsilon}}$ from Lemma 4.7 completes the proof of (C.1). We proceed to prove Propositions C.1 and C.2.

C.2. Proof of Proposition C.1

Each component of $\boldsymbol{G}_{\varepsilon}(0)$ contains precisely one of the variable nodes $x_{1},\ldots,x_{n}$. In effect, $\boldsymbol{A}_{\varepsilon}(0)$ has a block diagonal structure, and the overall nullity is nothing but the sum of the nullities of the blocks. It therefore suffices to calculate the nullity of the block $\boldsymbol{B}_{s}$ representing the connected component of $x_{s}$. Indeed, because $\sum_{s=1}^{n}\left|{\partial^{2}x_{s}}\right|=\sum_{i\leq{\boldsymbol{m}}^{\prime}_{\varepsilon}(0)}\boldsymbol{k}_{i}^{\prime}(\boldsymbol{k}_{i}^{\prime}-1)$ and $\sum_{s=1}^{n}\left|{\partial^{2}x_{s}\cap\mathcal{F}_{0}}\right|=|\mathcal{F}_{0}|$ we have

[TABLE]

Consequently, since $\boldsymbol{\theta}=O_{n}(1)$ it suffices to prove that

[TABLE]

In fact, the second case in (C.4) simply follows from $\boldsymbol{N}_{s}\leq\boldsymbol{d}_{s}$ and $\mathbb{E}[\boldsymbol{d}_{s}]=O_{n}(1)$ for all $s$.

Hence, suppose that $s>\boldsymbol{\theta}$. As $|\boldsymbol{N}_{s}|\leq\boldsymbol{d}_{s}$ and $\mathbb{E}[\boldsymbol{d}_{s}^{r}]=O_{\varepsilon,n}(1)$ for an $r>2$ we find $\xi>0$ such that

[TABLE]

Moreover, let $\Xi=\sum_{i=1}^{{\boldsymbol{m}}_{\varepsilon}^{\prime}(0)}\boldsymbol{k}_{i}^{\prime}\boldsymbol{1}\left\{{\boldsymbol{k}_{i}^{\prime}>\varepsilon^{-8}}\right\}$, $\boldsymbol{M}_{j}^{\prime}=\sum_{i=1}^{{\boldsymbol{m}}_{\varepsilon}^{\prime}(0)}\boldsymbol{1}\{\boldsymbol{k}_{i}^{\prime}=j\}.$ Because $\mathbb{E}[\boldsymbol{k}^{2}]=O_{\varepsilon,n}(1)$ we have

[TABLE]

while $\boldsymbol{M}_{j}^{\prime}\sim(1-\varepsilon)dn{\mathbb{P}}\left[{\boldsymbol{k}=j}\right]/k$ for all $j\leq\varepsilon^{-8}$ a.a.s. by Chebyshev’s inequality. Hence, introducing the event

[TABLE]

we obtain from (C.5) and (C.6) that

[TABLE]

With $\boldsymbol{\gamma}\leq\boldsymbol{d}_{s}$ the actual degree of $x_{s}$ in $\boldsymbol{G}_{\varepsilon}(s)$, let $\boldsymbol{\kappa}_{1},\ldots,\boldsymbol{\kappa}_{\boldsymbol{\gamma}}$ be the degrees of the checks adjacent to $x_{s}$. We claim that given ${\mathcal{E}}_{s}$ and $\boldsymbol{d}_{s}$,

[TABLE]

Indeed, on ${\mathcal{E}}_{s}$ the probability that $x_{s}$ is adjacent to a check of degree greater than $\varepsilon^{-8}$ is

$O_{\varepsilon,n}(\boldsymbol{d}_{s}\Xi/\sum_{j\geq 3}j\boldsymbol{M}_{j}^{\prime})=o_{\varepsilon,n}(\varepsilon)$. Further, given ${\mathcal{E}}_{s}$ we have

[TABLE]

and thus ${\mathbb{P}}[\boldsymbol{\gamma}<\boldsymbol{d}_{s}\mid{\mathcal{E}}_{s}]=o_{\varepsilon,n}(\varepsilon^{1/2})$. Moreover, given $\boldsymbol{\gamma}=\boldsymbol{d}_{s}$, for each $i\in[\boldsymbol{d}_{s}]$ the probability that the $i$-th clone of $x_{s}$ gets matched to a check of degree $j\leq\varepsilon^{-8}$ is

[TABLE]

These events are asymptotically independent for the different clones. Thus, we obtain (C.8).

Finally, we can easily compute $\boldsymbol{N}_{s}$ given the vector $(\boldsymbol{\kappa}_{1},\ldots,\boldsymbol{\kappa}_{\boldsymbol{\gamma}})$. The matrix $\boldsymbol{B}_{s}$ has fairly simple structure. The first ${\boldsymbol{\gamma}}$ rows have a non-zero entry in the first column representing $x_{s}$. Additionally, for $i=1,\ldots,{\boldsymbol{\gamma}}$ the $i$th row contains $\boldsymbol{\kappa}_{i}-1$ further non-zero entries, and the columns where theses non-zero entries occur are disjoint for all $i$. Finally, at the bottom of the matrix there is a block freezing the variables in $\mathcal{F}_{0}\cap\partial^{2}x_{s}$ to zero. We therefore claim that the rank of the matrix works out to be

[TABLE]

To see this, let us first compute the rank of the matrix $\boldsymbol{B}_{s}^{\prime}$ without the first column. Then row $i\in[{\boldsymbol{\gamma}}]$ contributes to the rank unless all the variables in the corresponding equation other than $x_{s}$ belong to $\mathcal{F}_{0}$, an event that occurs with probability $\beta^{\boldsymbol{\kappa}_{i}-1}$; hence the first summand. In addition, the $|\mathcal{F}_{0}\cap\partial^{2}x_{s}|$ rows pegging variables to zero contribute to the rank (second summand). Furthermore, going back to $\boldsymbol{B}_{s}$, the first column adds to the rank unless non of the first ${\boldsymbol{\gamma}}$ rows of $\boldsymbol{B}_{s}^{\prime}$ gets zeroed out completely, an event that has probability $\prod_{i=1}^{{\boldsymbol{\gamma}}}(1-\beta^{\boldsymbol{\kappa}_{i}-1})$. Since

[TABLE]

substituting (C.9) in yields

[TABLE]

Combining (C.7), (C.8) and (C.10) completes the proof.

C.3. Proof of Proposition C.2

To couple the random variables $\mathcal{N}_{t+1}$ and $\mathcal{N}_{t}$ we need to investigate short linear relations among the cavities, i.e., the clones from $\bigcup_{i=1}^{n}\left\{{x_{i}}\right\}\times[\boldsymbol{d}_{i}]$ that are not incident to an edge of $\boldsymbol{\Gamma}_{\varepsilon}(t)$. Denote this set by ${\mathcal{C}}(t)$. Further, let $P_{t}$ be the distribution on the set of variables induced by drawing a random cavity, i.e.,

[TABLE]

and let $\boldsymbol{y}_{1},\boldsymbol{y}_{2}\ldots$ be independent samples from $P_{t}$.

Lemma C.4**.**

For any $\delta>0$ and $\ell>0$ there is $\mathcal{T}=\mathcal{T}(\delta,\ell)>0$ such that

[TABLE]

Proof.

The choice of ${\boldsymbol{m}}_{\varepsilon,n}$ guarantees that $|{\mathcal{C}}(t)|\geq\varepsilon n/2$ a.a.s. Moreover, since $\mathbb{E}[\boldsymbol{d}]=O_{\varepsilon,n}(1)$ we find $L=L(\varepsilon,\delta)>0$ such that the event $\mathcal{L}=\left\{{\sum_{i=1}^{n}\boldsymbol{d}_{i}\boldsymbol{1}\{\boldsymbol{d}_{i}>L\}<\varepsilon\delta^{2}n/16}\right\}$ has probability ${\mathbb{P}}[\mathcal{L}]\geq 1-\delta/8$. Therefore, we may condition on ${\mathcal{E}}=\mathcal{L}\cap\{|{\mathcal{C}}(t)|\geq\varepsilon n/2\}$.

Let $\boldsymbol{x}_{1},\ldots,\boldsymbol{x}_{\ell}$ be variables drawn uniformly with replacement from $V_{n}=\{x_{1},\ldots,x_{n}\}$. Then on the event ${\mathcal{E}}$ we have, for any $\ell$-tuple $y_{1},\ldots,y_{\ell}$ of variables,

[TABLE]

Consequently, because the distribution of $\boldsymbol{G}_{\varepsilon}(t)-\{p_{1},\ldots,p_{\boldsymbol{\theta}}\}$ is invariant under permutations of $x_{1},\ldots,x_{n}$, Remark 3.6 shows that ${\mathbb{P}}\left[{\boldsymbol{x}_{1}=y_{1},\ldots,\boldsymbol{x}_{\ell}=y_{\ell}\mid\boldsymbol{A}_{\varepsilon}(t)}\right]<\delta(\varepsilon/(2L))^{\ell}/2$, provided that $\mathcal{T}=\mathcal{T}(\delta,\ell)$ is large enough. ∎

We proceed to derive Proposition C.2 from Lemma C.4 and a coupling argument. Let $\boldsymbol{G}_{\varepsilon}^{\prime}(t)$ be the Tanner graph obtained from $\boldsymbol{G}_{\varepsilon}(t+1)$ by removing the check $a_{t+1}$, let $\boldsymbol{A}_{\varepsilon}^{\prime}(t)$ be the corresponding matrix and let

[TABLE]

Then clearly

[TABLE]

Let $\boldsymbol{\alpha}\in[0,1]$ be the fraction of frozen cavities in $\boldsymbol{G}_{\varepsilon}^{\prime}(t)$, with the convention that $\boldsymbol{\alpha}=0$ if the set ${\mathcal{C}}^{\prime}(t)$ of these cavities is empty.

Lemma C.5**.**

We have $\mathbb{E}\left|{\mathbb{E}[\mathcal{N}_{t+1}-\mathcal{N}_{t}^{\prime}\mid\boldsymbol{A}_{\varepsilon}(t)^{\prime},{\boldsymbol{m}}_{\varepsilon,n}]-(K(\boldsymbol{\alpha})-1)}\right|=o_{\varepsilon,n}(1)$.

Proof.

The random matrix $\boldsymbol{A}_{\varepsilon}(t+1)$ is obtained from $\boldsymbol{A}_{\varepsilon}^{\prime}(t)$ by inserting a new random check $a_{t+1}$. Pick $\zeta=\zeta(\varepsilon)>0$ small enough and $\delta=\delta(\zeta)>0$ smaller still. Since $|\operatorname{nul}(\boldsymbol{A}_{\varepsilon}^{\prime}(t))-\operatorname{nul}(\boldsymbol{A}_{\varepsilon}(t+1))|\leq 1$ and $\mathbb{E}[\boldsymbol{k}^{2}]=O_{\varepsilon,n}(1)$ we may condition on the event that $\boldsymbol{k}_{t+1}\leq\varepsilon^{-1}$. Similarly, Lemma C.4 shows that we may assume that the set $\mathcal{X}$ of variables of $\boldsymbol{G}_{\varepsilon}^{\prime}(t)$ where the new check node $a_{t+1}$ attaches does not form a proper relation, provided that $\mathcal{T}=\mathcal{T}(\varepsilon)$ is chosen sufficiently large. Therefore, Lemma 2.5 yields

[TABLE]

as claimed. ∎

Lemma C.6**.**

Let $Q(\alpha,\beta)=\mathbb{E}\left[{\boldsymbol{k}(\alpha\beta^{\boldsymbol{k}-1}-1)}\right]$ for $\alpha\in[0,1]$. Then $\mathbb{E}\left|{\mathbb{E}\left[{\mathcal{N}_{t}-\mathcal{N}_{t}^{\prime}\mid\boldsymbol{A}^{\prime}_{\varepsilon}(t),{\boldsymbol{m}}_{\varepsilon,n}}\right]-Q(\boldsymbol{\alpha},\beta)}\right|=o_{\varepsilon,n}(1).$

Proof.

The factor graph $\boldsymbol{G}_{t}(\varepsilon)$ is obtained from $\boldsymbol{G}_{t}^{\prime}(\varepsilon)$ by adding the checks $b_{{\boldsymbol{m}}_{\varepsilon,n}-t-1,h}$ for $h\in[\boldsymbol{k}^{\prime}_{{\boldsymbol{m}}_{\varepsilon}-t-1}]$, the corresponding variables $x_{{\boldsymbol{m}}_{\varepsilon,n}-t-1,h,j}$ and possibly their respective checks $f_{{\boldsymbol{m}}_{\varepsilon,n}-t-1,h,j}$. Since by construction

[TABLE]

and $\mathbb{E}[\boldsymbol{k}^{2}]=O_{\varepsilon,n}(1)$ we may condition on the event that $\boldsymbol{k}^{\prime}_{{\boldsymbol{m}}_{\varepsilon}-t-1}\leq\varepsilon^{-1}$. In effect, Lemma C.4 shows that we may assume the set $\mathcal{X}$ of cavities adjacent to the new checks $b_{{\boldsymbol{m}}_{\varepsilon,n}-t-1,h}$ does not form a proper relation, provided that $\mathcal{T}=\mathcal{T}(\varepsilon)$ is chosen large enough. Moreover, the number of frozen cavities in $\mathcal{X}$ is within $o_{n}(1)$ of a binomial distribution ${\rm Bin}(\boldsymbol{k}_{{\boldsymbol{m}}_{\varepsilon,n}-t-1}^{\prime},\boldsymbol{\alpha})$ in total variation. Therefore, Lemma 2.5 shows that

[TABLE]

as claimed. ∎

Lemma C.7**.**

We have $\mathbb{E}[\mathcal{Y}_{t+1}-\mathcal{Y}_{t}]=\mathbb{E}[(\boldsymbol{k}-1)(\beta^{\boldsymbol{k}}-1)]$.

Proof.

This is the result of a straightforward calculation. ∎

Proof of Proposition C.2.

Combining Lemmas C.5–C.7, we obtain

[TABLE]

Since $x^{k}-kxy^{k-1}+(k-1)y^{k}\geq 0$ for all $k\geq 1$, $x,y\in[0,1]$, the assertion follows from (C.11). ∎

Appendix D Verification of $({\boldsymbol{m}}-{\boldsymbol{m}}^{\prime})/n$ and $\operatorname{rk}(\boldsymbol{A}^{\prime})/n$

Let ${\boldsymbol{m}}_{j}$ denote the number of rows with exactly $j$ nonzero entries. With standard concentration arguments, we know that a.a.s. ${\boldsymbol{m}}_{0}\sim{\boldsymbol{m}}{\mathbb{P}}(\boldsymbol{k}=0)\sim(dn/k)K(0)$, and ${\boldsymbol{m}}_{1}\sim{\boldsymbol{m}}{\mathbb{P}}(\boldsymbol{k}=1)\sim(dn/k)K^{\prime}(0)$. Consequently, a.a.s. $({\boldsymbol{m}}-{\boldsymbol{m}}^{\prime})/n\sim d(1-K(0)-K^{\prime}(0))/k$.

For $\operatorname{rk}(\boldsymbol{A}^{\prime})$, let $X_{i}$ be the indicator variable that there exists a row with exactly one nonzero entry, and that nonzero entry occurs at the $i$-th column. Then $\operatorname{rk}(\boldsymbol{A}^{\prime})=\sum_{i=1}^{n}X_{i}$. Conditioning on ${\boldsymbol{m}}_{1}$ and $\boldsymbol{D}=\sum_{j}j{\boldsymbol{m}}_{j}$, we know that ${\mathbb{E}}X_{i}=\sum_{j}{\mathbb{P}}(\boldsymbol{d}=j)(1-({\boldsymbol{m}}_{1}/\boldsymbol{D})^{j})$ for every $i$. Since a.a.s. ${\boldsymbol{m}}_{1}/\boldsymbol{D}\sim K^{\prime}(0)/k$, the standard concentration results immediately yield that a.a.s. $\operatorname{rk}(\boldsymbol{A}^{\prime})/n\sim 1-\sum_{j}{\mathbb{P}}(\boldsymbol{d}=j)(K^{\prime}(0)/k)^{j}=1-D(1-K^{\prime}(0)/k)$.