Eigenvalues of the non-backtracking operator detached from the bulk

TL;DR

This paper analyzes the non-backtracking spectrum of stochastic block models in a dense regime, identifying a key eigenvalue inside the bulk and introducing a new perturbation theorem for quadratic eigenvalue problems.

Contribution

It provides a detailed spectral analysis of the non-backtracking operator in dense stochastic block models and introduces a novel Bauer-Fike variant for quadratic eigenvalue problems.

Findings

Existence of a real eigenvalue inside the bulk near a specific location.

Characterization of the non-backtracking spectrum in dense regimes.

Introduction of a new Bauer-Fike theorem variant for quadratic eigenvalue problems.

Abstract

We describe the non-backtracking spectrum of a stochastic block model with connection probabilities $p_{\mathrm{in}}, p_{\mathrm{out}} = \omega(\log n)/n$. In this regime we answer a question posed in Dall'Amico and al. (2019) regarding the existence of a real eigenvalue `inside' the bulk, close to the location $\frac{p_{\mathrm{in}}+ p_{\mathrm{out}}}{p_{\mathrm{in}}- p_{\mathrm{out}}}$. We also introduce a variant of the Bauer-Fike theorem well suited for perturbations of quadratic eigenvalue problems, and which could be of independent interest.

Figures

Equations

$$B_{(i,j),(k,\ell )}=A_{k,\ell }{1}_{j=k,i\ne \ell }.$$

$$p,q=\frac{\omega (\log n)}{n},p,q\to 0,C_1⩽\frac{|p-q|}{p+q}⩽C_2$$

$$\frac{np}{2}-p+\frac{nq}{2}=\frac{n(p+q)}{2}-p.$$

$$\alpha$$

$$\beta$$

$$\det (B-zI)=(z^2-1{)}^{|E|-|V|}\det (z^{2}I-zA+D-I),$$

$$H=\left[\begin{array}{cc}A& I-D\\ I& 0\end{array}\right]$$

$${\lambda }_1(B)=\alpha +O({\alpha }^{3/4}),{\lambda }_2(B)=\beta +O({\alpha }^{3/4}),$$

$${\lambda }_{2n-1}(B)=\frac{\alpha }{\beta }+o(1),\text{ and }{\lambda }_{2n}(B)=1.$$

$$H(r):=(r^2-1)I+D-rA,$$

$${\hat{H}}_0=\left[\begin{array}{cc}\hat{A}& \eta I\\ I& 0\end{array}\right]$$

$$0=\det (z^{2}M-z\hat{A}-X)$$

$$Q_{\hat{A},X}:=\left[\begin{array}{cc}\hat{A}& X\\ I& 0\end{array}\right],$$

$$\det (Q_{\hat{A},X}-zI)=\prod _{i=1}^n(z^2-{\hat{a}}_{i}z-x_i)$$

$$L_0=\left[\begin{array}{cc}\hat{A}& X\\ I& 0\end{array}\right],L=\left[\begin{array}{cc}\hat{B}& Y\\ I& 0\end{array}\right].$$

$$|\mu -\nu |⩽\sqrt{\kappa (P)}\sqrt{\Vert X-Y+\mu (\hat{A}-\hat{B})\Vert }.$$

$$\left({\cup }_{j\in \mathcal{K}}\mathcal{B}({\nu }_k,\epsilon )\right)\cap \left({\cup }_{j\notin \mathcal{K}}\mathcal{B}({\nu }_k,\epsilon )\right)=\varnothing ,$$

$$R_{\mu }={\mu }^{2}I-\mu \hat{B}-Y$$

$$=S_{\mu }+X+\mu \hat{A}-\mu \hat{B}-Y$$

$$=S_{\mu }(I+S_{\mu }^{-1}(X+\mu \hat{A}-\mu \hat{B}-Y)).$$

$$1⩽\Vert S_{\mu }^{-1}\Vert \cdot \Vert X+\mu \hat{A}-\mu \hat{B}-Y\Vert =\Vert S_{\mu }^{-1}\Vert \cdot \Vert X-Y+\mu (\hat{A}-\hat{B})\Vert .$$

$$\Vert S_{\mu }^{-1}\Vert ⩽\kappa (P)\times \underset{k\in [n]}{\max }|{\mu }^2-\mu {\lambda }_k-{\delta }_k{|}^{-1}.$$

$$1⩽\kappa (P)\times \frac{\Vert X-Y+\mu (\hat{A}-\hat{B})\Vert }{|{\mu }^2-\mu {\lambda }_k-{\delta }_k|}=\kappa (P)\times \frac{\Vert X-Y+\mu (\hat{A}-\hat{B})\Vert }{|\mu -{\alpha }_k||\mu -{\beta }_k|}$$

$$|\mu -{\alpha }_k||\mu -{\beta }_k|⩽\kappa (P)\Vert X-Y+\mu (\hat{A}-\hat{B})\Vert :=x.$$

$$L_0=\left[\begin{array}{cc}\hat{A}& X\\ I& 0\end{array}\right]L=\left[\begin{array}{cc}\hat{A}& Y\\ I& 0\end{array}\right]$$

$$|\mu -\nu |⩽\epsilon :=\sqrt{\kappa (P)\Vert X-Y\Vert }.$$

$$\mathbb{E}[\Vert A-\mathbb{E}A\Vert ]⩽(2+o(1))\sqrt{\alpha }.$$

$$\mathbb{P}\left(\left|\frac{\Vert A-\mathbb{E}A\Vert }{\sqrt{\alpha }}-\frac{\mathbb{E}[\Vert A-\mathbb{E}A\Vert ]}{\sqrt{\alpha }}\right|⩾t\right)⩽2e^{-c{\alpha }^2{t}^2}.$$

$$\Vert A-\mathbb{E}A\Vert$$

$$|{\lambda }_1(A)-\alpha |$$

$$\underset{k⩾3}{\max }|{\lambda }_k(A)|$$

$$\gamma =\frac{n(p+q)}{2}-p-1=\alpha -1$$

Full text

Y.Z. is partially supported by NSF DMS-1712630.

1. Introduction

For any real matrix $A$ with size $n\times n$, its non-backtracking operator $B$ is the real matrix indexed by the coordinates of the non-zero entries of $A$, and is defined by

[TABLE]

The non-backtracking matrix of a graph is the non-backtracking matrix of its adjacency matrix, and it is closely related to the Zeta function of the graph [29]. Its spectrum was first studied in the case of finite graphs and their universal covers [19, 7, 27, 20, 29, 5]. Recently, the non-backtracking operator attracted a lot of attention from random graph theory as a very powerful tool. In the spectral theory of random graphs, it was a key element in a new proof of the Alon-Friedman theorem for random regular graphs [11]. In the same vein it has been used later to study the eigenvalues of random regular hypergraphs [16], random bipartite biregular graphs [13] and homogeneous or inhomogeneous Erdős-Rényi graphs [21, 31, 12, 18, 8, 3, 9]. Very recently, the real eigenvalues were used to prove estimates on the vector-colouring number of a graph [6].

Most of the results focus on the eigenvalues of large magnitude, those which lie outside the bulk of the spectrum. They are known to be the ‘most informative’ eigenvalues, as they capture some essential features about the structure of the graph. For instance, in community detection, the appearance of certain outliers indicates when the community structure can be recovered [12, 21]. A cornerstone result was that even in the difficult dilute case, where the connection probabilities are of order $1/n$, reconstruction was feasible (under some condition) by looking at the eigenvalues of the non-backtracking matrix appearing outside the ‘bulk’ of eigenvalues.

It was recently observed in [15, Section 3.2] that in fact, there is a real eigenvalue isolated inside the bulk that corresponds to the ratio of the two largest eigenvalues outside the bulk, as displayed in the right panel of Figure 1. Recall that $B$ is non-Hermitian with a complex spectrum, so that ‘inside’ the bulk is understood as eigenvalues inside the circle of the spectrum. To the best of our knowledge, this phenomenon has not been rigorously studied yet. In this paper, we prove the existence of this real eigenvalue inside the bulk for the stochastic block model (SBM) in the regime where the mean degree goes to infinity faster than $\log n$.

Notations

Throughout the paper, we will adopt the conventional notations $a_{n}=o(b_{n})$ when $\lim_{n\to\infty}\frac{a_{n}}{b_{n}}=0$, $a_{n}=\omega(b_{n})$ when $\lim_{n\to\infty}\frac{a_{n}}{b_{n}}=\infty$ and $a_{n}=O(b_{n})$ when $|a_{n}/b_{n}|$ is bounded. All the results depend on the parameter $n$, the size of the graph, which is seen as large through $n\to\infty$. For any matrix $M$, we denote the spectral norm of $M$ as $\|M\|$.

1.1. Setting: the SBM in the logarithmic regime

Consider a stochastic block model $G(n,p,q)$ with an even number $n$ of vertices, two blocks of equal size $n/2$, and two probability parameters $p,q$: if $i,j$ are vertices in the same block then they are connected with probability $p$, and if they are in different blocks they are connected with probability $q$. We will place ourselves under the regime

[TABLE]

for some constants $C_{1},C_{2}\in(0,1)$. The last condition is technical: we will see that it is here to ensure that two separated outliers appear in the spectrum of the adjacency matrix, and are of the same order. This assumption is crucial in our perturbation analysis, see Remark 4.2 for further discussion.

It is known (see for example [10, Chapter 3]) that when $q=p=\omega(\log n/n)$, the $G(n,p,p)$ graph (the Erdős-Rényi model) is ‘almost regular’ in the sense that all degrees are concentrated around $(n-1)p$. In general, it can be shown that when $p,q=\omega(\log n)/n$ then $G(n,p,q)$ is almost regular with degrees concentrated around

[TABLE]

See Subsection 3.4 for a proof. For this reason, we will denote by $\alpha$ the mean degree and by $\beta$ the mean difference degree:

[TABLE]

where $d_{i}$ is the number of neighbors of vertex $i$, and $d^{\mathrm{out}}_{i}$ (resp. $d^{\mathrm{in}}_{i}$) is the number of neighbors of vertex $i$ which do not have the same type as $i$ (resp. which have the same type as $i$). Under assumption (A), $\beta$ can be either positive or negative, and $\alpha,\beta$ are of the same order.

Our assumptions (A) imply that the mean degree $\alpha$ is $\omega(\log n)$ and the mean difference degree $\beta$ has the same order as $\alpha$. Finally, the adjacency matrix of the graph is the $n\times n$ matrix $A$ defined by $A_{i,j}=\mathbf{1}_{i\sim j}$, where $i\sim j$ denotes the event that $i$ and $j$ are connected.

1.2. Main results

Let $G=(V,E)$ be any finite graph with adjacency matrix $A$. The Ihara-Bass formula gives a connection between the spectrum of $B$ defined in (1.1) and a quadratic eigenvalue problem: for any complex $z$,

[TABLE]

see [7, 20, 27]. The zeros of the polynomial $z\mapsto\det(z^{2}I-zA+D-I)$ are usually called, with an abuse of language, the non-backtracking spectrum of $B$, the two additional eigenvalues $\pm 1$ appearing with multiplicity $|E|-|V|$ in (1.2) being usually called trivial. The non-backtracking spectrum can be expressed as the eigenvalues of a smaller matrix $H$:

[TABLE]

where $D=\mathrm{diag}(A\mathbf{1})$ is the diagonal degree matrix. This representation of the spectrum of $B$ in terms of $H$ is extremely useful: to compute the spectrum of $B$, we do not have to construct the matrix $B$, and we can analyze the spectrum of $B$ directly from $H$.

Using these facts, we answer the question posed in [15] regarding the existence of an isolated eigenvalue inside the bulk, at least under the assumptions (A). We also give a detailed description of the non-backtracking spectrum that is similar to the one given in [31] for Erdős-Rényi graphs.

Theorem 1.1**.**

Let $B$ be the non-backtracking operator of a stochastic block model $G(n,p,q)$ satisfying assumption (A). We order the $2|E|$ eigenvalues of $B$ by decreasing modulus: $|\lambda_{1}(B)|\geqslant\dotsb\geqslant|\lambda_{2|E|}(B)|$.

With probability $1-o(1)$, the spectrum of $B$ can be described as follows. First, the smallest eigenvalues in modulus are the trivial eigenvalues $-1$ and $1$, each with multiplicity $|E|-|V|$.

Then, in the non-trivial eigenvalues $\lambda_{1}(B),\dotsc,\lambda_{2n}(B)$, there are four real eigenvalues which are isolated, two ‘outliers’

[TABLE]

and two ‘insiders’

[TABLE]

All the other eigenvalues $\lambda_{k}(B)$ with $k\in\{3,\dotsc,2n-3\}$ are located within distance $o(\sqrt{\alpha})$ of a circle of radius $\sqrt{\alpha-1}$. Moreover, the real parts of eigenvalues of $\frac{B}{\sqrt{\alpha}}$ are asymptotically distributed as the semi-circle distribution supported on $[-1,1]$.

To present our approach in the most efficient and clear way, we state and prove the theorem in the simplest regime, when there are only two blocks and when the community structure appears in the spectrum through the presence of an extra outlier near $\beta$. It is straightforward to check that our proof can be extended to a diversity of settings, when the mean degree is $\omega(\log n)$. We state this as an informal result:

Assume that with high probability the degree of each vertex is $(1+o(1))\alpha$ and the spectrum of the adjacency matrix has $k=O(1)$ outliers far outside $[-2\sqrt{\alpha},2\sqrt{\alpha}]$, say $\lambda_{1}\approx\alpha$, and $\lambda_{2},\dotsc,\lambda_{k}$. Assume $\lambda_{1},\dots,\lambda_{k}$ are are of the same order. Then with high probability its non-backtracking spectrum will have $k$ eigenvalues near $\lambda_{i}$ for $i\in[k]$, then $k$ eigenvalues near $\alpha/\lambda_{i}$, and all the other eigenvalues will be located within distance $o(\sqrt{\alpha})$ of the circle of radius $\sqrt{\alpha-1}$.

Remark 1.2*.*

The concentration results we have in Section 3 work for general inhomogeneous random graphs with outliers of the same order. Our modified Bauer-Fike theorem given in Theorem 2.2 works for general inhomogeneous random graphs as well, as long as each vertex has almost regular degree $(1+o(1))\alpha$. Therefore all the analysis in Section 4 can be extended to the case we mentioned above for $k$ outliers.

The presence of bulk insiders in Theorem 1.1 and in the preceding statement are illustrated in Figure 1 for a realization of an Erdős-Rényi graph and a realization of an SBM graph. Note that the description of the spectrum of $B$ in the preceding theorem is much more precise than Theorem 1.5 in [31]. This comes from the fact that their perturbation parameter $R=c\sqrt{\log n/p}$ goes to infinity (see Theorem 1.5 in [31] for the exact statement, where the scaling parameter is different from ours). Our method includes a tailored version of the Bauer-Fike theorem suited for perturbations of matrices like (1.3), which yields perturbation bounds that are better than the classical Bauer-Fike theorem in terms of the order of magnitude, and without which the existence of the two eigenvalues at $1$ and near $\beta/\alpha$ would not follow. We think such variants of the Bauer-Fike theorem could be of independent interest.

1.3. Bulk insiders and community detection

The real eigenvalue of $B$ inside the bulk is closely related to community detection problems for SBMs. An interesting heuristic spectral algorithm based on the Bethe Hessian matrix was proposed in [32, 26]. The Bethe Hessian matrix, sometimes called deformed Laplacian ([17, 15]), is defined as

[TABLE]

where $r\in\mathbb{R}$ is a regularizer to be carefully tuned. It is conjectured in [26] that a spectral algorithm based on the eigenvectors associated with the negative eigenvalues of $H(r)$ with $r=\sqrt{\rho(B)}$ is able to reach the information-theoretic threshold confirmed in [23, 12, 25, 24] for community detection in the dilute regime. In a subsequent work [15], the authors crafted a spectral algorithm based on $H(r)$ with $r=\alpha/\beta$ and empirically showed it outperforms already known spectral algorithms. Their choice of $r$ was motivated by the conjectured value of the real eigenvalue inside the bulk. The gain in using $H(r)$ instead of $B$ in spectral algorithms mainly comes from the fact that $H(r)$ has a smaller dimension than $B$, is Hermitian, and is easiest to build from $A$ — nearly no preprocessing is needed, in contrast with non-backtracking matrices [12, 21], self-avoiding path matrices [23] or graph powering matrices [1].

The relation between the Bethe Hessian matrix and the non-backtracking operator is given by the Ihara-Bass formula (see (1.2) above). Therefore, a good understanding of the real eigenvalues of the non-backtracking operator is the first step towards understanding the theoretical guarantee of the heuristic algorithms purposed in [26, 15].

Unfortunately, our proof techniques do not work in the dilute regime. In the regime studied in this paper, community detection problems are now very well understood and clustering based on the second eigenvector of $A$ has been proven to yield exact reconstruction (see [2]). Our result should instead be seen as a preliminary step in view of

1. proving the existence of bulk insiders in the dilute regime, 2) showing their usefulness in practical reconstruction. It will be helpful in practice to have a better understanding of the eigenvectors for the Bethe Hessian matrix, and we leave it as a future direction.

The key obstacle is the lack of concentration of degrees profiles, which tells us random graphs with bounded expected degrees are far away from being ‘roughly’ regular (see also the discussion in Subsection 3.4). Without this property, our perturbation analysis does not apply.

Organization of the paper

In Section 2, we first state some classical facts on the non-backtracking spectrum of graphs then we state and prove a perturbation theorem which is well suited for quadratic eigenvalue problems and improves the classical Bauer-Fike results. In Section 3, we gather several facts on stochastic block models. Then we study the spectrum of $H$ as in (1.3) and a suitably chosen perturbation of $H$ (defined later in (3.2)). In Section 4 we prove the main theorem.

2. Perturbation of the non-backtracking spectrum

2.1. The non-backtracking spectrum

When the graph is regular with degree $d$, the diagonal matrix satisfies $D=dI$, and we can relate the eigenvalues of $B$ with the eigenvalues of $A$ through exact algebraic relations as in the following elementary lemma.

Lemma 2.1**.**

Let $\hat{A}$ be a Hermitian matrix with eigenvalues $\lambda_{1},\dotsc,\lambda_{n}$ and let $\eta$ be a nonzero complex number. Then, the characteristic polynomial of the matrix

[TABLE]

is given by $\chi_{\hat{H}_{0}}(z)=\prod_{k=1}^{n}(z^{2}-\lambda_{k}z-\eta)$, and the eigenvalues of $\hat{H}_{0}$ are the $2n$ complex numbers (counted with multiplicities) which are solutions of $z^{2}-z\lambda_{k}-\eta=0$ for $k\in[n]$.

Similar exact relations have also been used when the graph has a very specific structure, like bipartite biregular (see [13]). When the graph $G$ does not exhibit such a simple structure, the relation between $A$ and $B$ becomes more involved. Several Ihara-Bass-like formulas are available (see for instance [32, 8, 4]), but they are usually hard to analyze.

As cleverly noted in [31], the spectrum of $H$ as in (1.3) is hard to describe in terms of the spectrum of $A$, but the spectrum of $\hat{H}_{0}$ in the preceding lemma is completely explicit in terms of the spectrum of $\hat{A}$, even if $\hat{A}$ has no specific structure. It is therefore quite natural to study the spectrum of $\hat{H}_{0}$ using the spectrum of $\hat{A}$, then use perturbation theorems to infer results on the ‘true’ non-backtracking spectrum, the spectrum of $H$. This is done in [31] through a combination of the Bauer-Fike theorem and a refinement of the Tao-Vu replacement principle [28].

The celebrated Bauer-Fike theorem says that* if a square matrix $\hat{A}$ is diagonalizable, say $\hat{A}=P\Delta P^{-1}$ for a diagonal matrix $\Delta$ and a non-singular matrix $P$, then under a perturbation $E$, every eigenvalue of the matrix $\hat{A}+E$ is within distance $\varepsilon$ of an eigenvalue of $\hat{A}$*, where $\varepsilon=\kappa(P)\|E\|$, and $\kappa(P)=\|P\|\|P^{-1}\|$ is the condition number (see for instance [12]).

We observe that the Bauer-Fike theorem, while optimal in the worst case, is indeed extremely wasteful when applied to $H$ and $H_{0}$. Taking into account the specific structure of $H$ and $H_{0}$ yields a better perturbation bound at virtually no cost, as shown in the next section.

2.2. Bauer-Fike theorems for quadratic eigenvalue problems

A quadratic eigenvalue problem (QEP) consists of finding the zeroes of the polynomial equation

[TABLE]

where $M,\hat{A},X$ are square matrices, and $M$ is non-singular. Such problems appear in a variety of contexts and there exists an extensive literature on them, mainly from a numerical point of view (see the survey [30]).

The triplet $(M,\hat{A},X)$ can be replaced with by the triplet $(I,\hat{A}M^{-1},XM^{-1})$ without changing the problem, so we will be interested in the case where $M=I$. In this case, one can easily check that the solutions of (2.1) are the eigenvalues of the $2n\times 2n$ matrix

[TABLE]

which is called a linearization of the problem. In this section, we will present extensions of the Bauer-Fike theorem for linearizations of quadratic eigenvalue problems.

If both matrices $\hat{A}$ and $X$ are diagonal, say $\hat{A}=\mathrm{diag}(\hat{a}_{i})$ and $X=\mathrm{diag}(x_{i})$, then it is easily seen through elementary linear algebra operations that

[TABLE]

and the eigenvalues of $Q_{\hat{A},X}$ are the $2n$ complex solutions of the collection of $n$ quadratic equations $z^{2}-\hat{a}_{i}z-x_{i}=0$ for $1\leq i\leq n$.

We say the matrices $\hat{A}$ and $X$ are co-diagonalizable if there is a common non-singular matrix $P$ such that $P\hat{A}P^{-1}$ and $PXP^{-1}$ are diagonal. If $\hat{A},X$ are co-diagonalizable, then the identity (2.2) still holds with $\hat{a}_{i}$ being the eigenvalues of $\hat{A}$ and $x_{i}$ being those of $X$. As a consequence, we say that that $Q_{\hat{A},X}$ is QEP-diagonalizable if $\hat{A}$ and $X$ are co-diagonalizable. This is equivalent to ask that the matrix $z^{2}I-z\hat{A}-X$ is diagonalizable for any $z\in\mathbb{C}$.

Our main tool for the perturbation analysis is the following theorem.

Theorem 2.2**.**

Let $\hat{A},\hat{B},X,Y$ be $n\times n$ matrices. We define

[TABLE]

Suppose $L_{0}$ is QEP-diagonalizable, with $\hat{A}$ and $X$ being diagonalized by the common matrix $P$. Then, for any eigenvalue $\mu$ of $L$, there is an eigenvalue $\nu$ of $L_{0}$ such that

[TABLE]

Moreover, ‘multiplicities are preserved’ in the following sense: Denote $\varepsilon(\mu)$ the RHS of (2.3) and $\varepsilon=\max_{\mu\in\mathrm{Spec}(L)}\varepsilon(\mu)$. If $\nu_{1},\dotsc,\nu_{n}$ are the eigenvalues of $L_{0}$ and $\mathcal{K}$ is a subset of $[n]$ such that

[TABLE]

where $\mathcal{B}(\nu_{k},\varepsilon)=\{z\in\mathbb{C}:|z-\nu_{k}|\leq\varepsilon\}$ for $1\leq k\leq n$, then the number of eigenvalues of $L$ in $\cup_{j\in\mathcal{K}}\mathcal{B}(\nu_{k},\varepsilon)$ is exactly equal to $|\mathcal{K}|$.

Remark 2.3*.*

Theorem 2.2 is stated for two general matrices $L_{0}$ and $L$. However, the inequality (2.3) will yield good perturbation bound only when we can control the difference between $\hat{A},\hat{B}$, the difference between $X,Y$, and the condition number $\kappa(P)$.

Proof.

Assume $\mu$ is an eigenvalue of $L$. The matrix $R_{\mu}:=\mu^{2}I-\mu\hat{B}-Y$ is then singular. Assume, in addition, that $\mu$ is not an eigenvalue of $L_{0}$. Then, the matrix $S_{\mu}:=\mu^{2}I-\mu\hat{A}-X$ is non-singular. We have

[TABLE]

As a consequence the matrix $I+S_{\mu}^{-1}(X+\mu\hat{A}-\mu\hat{B}-Y)$ is singular, which directly implies that $-1$ is an eigenvalue of $S_{\mu}^{-1}(X+\mu\hat{A}-\mu\hat{B}-Y)$. therefore by the definition of spectral norm,

[TABLE]

As noted before the statement of the theorem, if $L_{0}$ is QEP-diagonalizable then the matrix $S_{\mu}$ is indeed diagonalizable: if $\Sigma=\mathrm{diag}(\lambda_{i})$ is the diagonal matrix of eigenvalues of $\hat{A}$, $\Delta=\mathrm{diag}(\delta_{i})$ the diagonal matrix of eigenvalues of $X$, and $P$ their common diagonalization matrix, then $S_{\mu}=P^{-1}(\mu^{2}I-\mu\Sigma-\Delta)P$, and the eigenvalues of $S_{\mu}$ are the complex numbers $\mu^{2}-\mu\lambda_{k}-\delta_{k}$, so

[TABLE]

From this, we infer that there is a $k\in[n]$ such that $\|S_{\mu}^{-1}\|\leqslant\kappa(P)\times|\mu^{2}-\mu\lambda_{k}-\delta_{k}|^{-1}$. Let us denote $\alpha_{k}$ and $\beta_{k}$ the two complex solutions of $0=z^{2}-z\lambda_{k}-\delta_{k}$ (they are eigenvalues of $L_{0}$), then

[TABLE]

which implies that

[TABLE]

If $|\mu-\alpha_{k}|$ and $|\mu-\beta_{k}|$ were both strictly greater than $\sqrt{x}$, the preceding inequality would be violated. One of those distances is thus smaller than $\sqrt{x}$, thus proving (2.3).

The ‘multiplicities preserved’ part is then proven as usual with the complex argument principle, see for instance [14, Appendix A]. ∎

When applying the preceding result with $\hat{A}=\hat{B}$, one gets the following corollary.

Corollary 2.4**.**

Let

[TABLE]

where $\hat{A},X,Y$ are square matrices and are such that $L_{0}$ is QEP-diagonalizable with $\hat{A}$ and $X$ diagonalized by the common matrix $P$. Then, for any eigenvalue $\mu$ of $L$, there is an eigenvalue $\nu$ of $L_{0}$ such that

[TABLE]

Remark 2.5* (Comparison with classical Bauer-Fike).*

Casting the classical Bauer-Fike theorem in this setting would yield an error term of $\varepsilon^{\prime}=\kappa(Q)\|X-Y\|$, where $Q$ is the diagonalization matrix of $L_{0}$. We thus gain the whole square root, and we do not need to compute the condition number of $Q$. This improvement is remarkable when the matrix $\hat{A}$ is itself Hermitian, for in this case $P$ is unitary and $\kappa(P)=1$, thus reducing the error term to $\sqrt{\|X-Y\|}$. If we had invoked the classical Bauer-Fike theorem instead, the error term would be $\kappa(Q)\|X-Y\|$, which can be far bigger than $\|X-Y\|$. In fact, the matrix $L_{0}$ is not Hermitian in general, and its diagonalization matrix $Q$ might be either difficult to compute or ill-conditioned: in [31], the bound obtained by the authors is $\kappa(Q)\leqslant O(\sqrt{1/p})$, where $p$ is the connection probability for an Erdős-Rényi graph $G(n,p)$. Our version of the Bauer-Fike theorem shows that for QEP, the only parameters at stake in perturbations are those of the original matrices $\hat{A}$ and $X$, not those of the linearization of the QEP.

3. The stochastic block model in the logarithmic regime

In this section, we collect results from the literature on stochastic block models or inhomogeneous Erdő-Rényi graphs, based on which we prove several quick results for our models, as given in Proposition 3.1, Proposition 3.2 and Corollary 3.3.

3.1. Outliers of the adjacency matrix

The concentration of the spectral norm for the SBMs follows immediately from the spectral norm bounds given in [22, 8] for inhomogeneous random matrices and random graphs. Recall Assumption (A). The following statement can be found for example in Example 4.1 of [22]: assume $\alpha=\omega(\log n)$, then

[TABLE]

Also from Equation (2.4) in [8], there exists a constant $c>0$ such that

[TABLE]

Taking $t=\sqrt{\log n}/\alpha$ in the inequality above, we have with probability $1-2n^{-c}$ that

[TABLE]

Since all the eigenvalues of $\mathbb{E}[A]$ are $\{-p,\beta,\alpha\}$, and $p=o(1)$ under assumption (A), the Weyl eigenvalue inequalities for Hermitian matrices yields the following proposition.

Proposition 3.1**.**

Assume $\alpha=\omega(\log n)$, then with high probability the following holds:

[TABLE]

3.2. Spectrum of the partially derandomized matrix $H_{0}$

We will use the notation

[TABLE]

which is the ‘mean degree minus one’. We introduce the partial derandomization of $H$ (defined in (1.3)) as:

[TABLE]

As already mentioned in Lemma 2.1, by elementary operations on $H_{0}$, one finds that the characteristic polynomial of $H_{0}$ is indeed equal to

[TABLE]

The eigenvalues of $H_{0}$ hence come into conjugate pairs coming from eigenvalues of $A$. Those eigenvalues $\lambda_{k}$ of $A$ for which $|\lambda_{k}|<2\sqrt{\gamma}$ give rise to two complex conjugate eigenvalues

[TABLE]

and the other ones, the outliers $|\lambda_{k}|>2\sqrt{\gamma}$ of the spectrum of $A$, give rise to two ‘harmonic conjugate’ eigenvalues

[TABLE]

Next we obtain a description of the eigenvalues of $H_{0}$ from the discussion on the spectrum of $A$ in Proposition 3.1. The description is illustrated in the second panel of Figure 2, the first one depicting the same phenomenon but for Erdős-Rényi graphs (with only one outlier in the spectrum).

Proposition 3.2**.**

Under assumption (A) with high probability the following holds for $H_{0}$.

1. (1)

The two eigenvalues with greater modulus, $\lambda_{1}(H_{0})$ and $\lambda_{2}(H_{0})$, are real, and they satisfy

[TABLE] 2. (2)

The two eigenvalues with smaller modulus, $\lambda_{2n}(H_{0})$ and $\lambda_{2n-1}(H_{0})$, are real, and they satisfy

[TABLE] 3. (3)

All the other $2n-4$ eigenvalues have modulus smaller than $\sqrt{\alpha}+o(\sqrt{\alpha})$. Among them, complex eigenvalues lie on a circle of radius $\sqrt{\alpha-1}$ and real ones lie in the intervals $[\sqrt{\alpha}-o(\sqrt{\alpha}),\sqrt{\alpha}+o(\sqrt{\alpha})]$ and $[-\sqrt{\alpha}-o(\sqrt{\alpha}),-\sqrt{\alpha}+o(\sqrt{\alpha})]$.

Proof.

We use Proposition 3.1 and the link described before between the spectrum of $A$ and the spectrum of $H_{0}$ given in (3.4) and (3.5).

The greatest eigenvalue of $A$ is $\lambda_{1}=\lambda_{1}(A)=\alpha+O(\sqrt{\alpha})$, from (3.4), it gives rise to two real eigenvalues of $H_{0}$:

[TABLE]

The second greatest eigenvalue ( in absolute value) of $A$ is $\lambda_{2}=\lambda_{2}(A)=\beta+O(\sqrt{\alpha})$, which gives rise to two real eigenvalues of $H_{0}$:

[TABLE]

For the eigenvalues $\lambda_{k}$ of $A$ with $2\sqrt{\gamma}<|\lambda_{k}|\leqslant(2+o(1))\sqrt{\alpha}$, from (3.5), the same argument gives

[TABLE]

Finally, from (3.4), all eigenvalues of $A$ with $|\lambda_{k}|\leqslant 2\sqrt{\gamma}$ give rise to two complex conjugate eigenvalues of $H_{0}$ with magnitude $\sqrt{\gamma}=\sqrt{\alpha-1}$. This completes the proof. ∎

The preceding description in Proposition 3.2 also shows that $H_{0}$ is non-singular, and we can quickly describe the eigenvalues of $H_{0}^{-1}$. We will later show that the norm of $(H^{-1}-H_{0}^{-1})$ is very small in Section 4. The strategy is then to apply Theorem 2.2 to $H_{0}^{-1}$ and $H^{-1}$, which gives a more precise estimate on the location of the outliers in $H$. See Remark 4.3 for further discussion.

Corollary 3.3** (inverse spectrum).**

Under the assumption (A), with high probability, in the spectrum of the matrix $H_{0}^{-1}$ there are exactly two real outliers

[TABLE]

and all the other eigenvalues of $H_{0}^{-1}$ have modulus smaller than $(\sqrt{\alpha}(1+o(1))^{-1}=o(1)$.

Proof.

The location of the two real outliers in the spectrum of $H_{0}^{-1}$ comes from part (2) in Proposition 3.2. The location of all the other eigenvalues comes from part (1) and (3) in Proposition 3.2. ∎

We now turn to the description of the global behavior of the spectrum of $A$.

3.3. Limiting spectral distribution of A

If we have an SBM with two blocks of equal size, and $p,q=\omega(1/n)$, the empirical spectral distribution of $\frac{A}{\sqrt{\alpha}}$ will converge weakly to the semicircle law: for any bounded continuous test function $f$, almost surely

[TABLE]

This can be seen from the graphon representation of SBMs and the result for generalized Wigner matrices (Section 4 in [33]), since each row in $\mathbb{E}A$ has the same row sum or equivalently, each vertex has the same expected degree. If the degree is not homogeneous, then the limiting spectral distribution will not be the semicircle law. We recall the following result from [33] for generalized Wigner matrices, which includes the regime where the sparsity parameter is $\omega(1/n)$.

Theorem 3.4** (Theorem 4.2. in [33]).**

Let $A_{n}$ be a random Hermitian matrix such that entries on and above the diagonal are independent and satisfy the following conditions:

1. (1)

$\mathbb{E}[a_{ij}]=0,\quad\mathbb{E}|a_{ij}|^{2}=s_{ij}$. 2. (2)

$\frac{1}{n}\sum_{j=1}^{n}s_{ij}=1+o(1)$* for all $i\in[n]$.* 3. (3)

For any constant $\eta>0$, $\displaystyle\lim_{n\to\infty}\frac{1}{n^{2}}\sum_{1\leqslant i,j\leqslant n}\mathbb{E}[|a_{ij}|^{2}\mathbf{1}(|a_{ij}|\geqslant\eta\sqrt{n})]=0.$ 4. (4)

$\sup_{ij}s_{ij}\leqslant C$* for a constant $C>0$.*

Then the empirical spectral distribution of $\frac{A_{n}}{\sqrt{n}}$ converges weakly to the semicircle law almost surely, which means that on an event with probability $1$, the convergence (3.9) holds for any bounded continuous function $f$.

We obtain the following theorem for the adjacency matrix $A$ of an SBM, and also for $H_{0}$.

Theorem 3.5**.**

Assume $\alpha\to\infty$ and $p,q\to 0$. The empirical spectral distribution of $\frac{A}{\sqrt{\alpha}}$ converges weakly to the semicircle law supported on $[-2,2]$ almost surely.

Moreover, the empirical spectral distribution of $\frac{H_{0}}{\sqrt{\alpha}}$ converges weakly almost surely to a distribution on the circle of radius $1$, and the limiting distribution of the real part of the eigenvalues of $H_{0}$ is the semicircle law rescaled on $[-1,1]$.

Proof.

We first consider the centered and scaled matrix

[TABLE]

For $i\not=j$ we have

[TABLE]

Then for all $i\in[n]$,

[TABLE]

One can quickly check that all the conditions in Theorem 3.4 hold for $M$. Therefore the empirical spectral distribution of

[TABLE]

converges weakly to the semicircle law. Or equivalently the empirical spectral distribution of $\frac{A-(\mathbb{E}A+pI)}{\sqrt{\alpha}}$ converges weakly almost surely to the same distribution. Finally, since the rank of the matrix $(\mathbb{E}A+pI)$ is $2$ and by the Cauchy interlacing theorem for the eigenvalue of Hermitian matrices, the empirical spectral distribution of $\frac{A}{\sqrt{\alpha}}$ converges weakly to the semicircle law almost surely.

We now turn to the second part of the theorem. Note that from (3.3), one eigenvalue $\lambda_{i}(A)$ corresponds to two eigenvalues of $H_{0}$ momentarily denoted by $\mu_{2i-1}(H_{0}),\mu_{2i}(H_{0})$, and such that

[TABLE]

The empirical spectral distribution of the real parts of eigenvalues of $H_{0}/\sqrt{\alpha}$ satisfies

[TABLE]

which converges weakly almost surely to the semicircle law rescaled on $[-1,1]$ by the first part of the theorem. ∎

3.4. Concentration of the degrees

We finally describe the degrees in the SBM. Let us note $d_{i}$ the degree of vertex $i$ in the SBM graph. Under the assumption (A), we have $\alpha=\omega(\log n)$, and the degrees are highly concentrated in the following sense.

Lemma 3.6**.**

With high probability $\displaystyle\max_{i\in[n]}|d_{i}-\alpha|=o(\alpha).$

Remark 3.7*.*

Note that Lemma 3.6 is no longer true in other regimes. When $\alpha=O(\log n)$, the event $d_{i}=(1+o(1))\alpha$ for all $i\in[n]$ does not happen with high probability (see for example [10, Chapter 3]). Then the diagonal degree matrix $D$ is not close to $\alpha I$, which is a barrier for our perturbation analysis to work.

Lemma 3.6 can be found in the literature, but we provide a proof for completeness. We recall Bernstein’s inequality: let $Y_{n}=\sum_{i=1}^{n}X_{i}$ where $X_{i}$ are independent random variables such that $|X_{i}|\leqslant b$. Define $\sigma_{n}^{2}:=\mathrm{Var}(Y_{n})$. Then for any $x>0$,

[TABLE]

Now we prove Lemma 3.6.

Proof.

Each $d_{i}$ has the same distribution with mean $\alpha$, hence we can apply the union bound and get

[TABLE]

Let us write $d_{1}=X_{2}+\dotsb+X_{n/2}+X_{n/2+1}+\dotsb+X_{n}$, where the $X_{i}$ are independent, and $X_{i}$ is a Bernoulli random variable with parameter $p$ if $i\in\{2,\dotsc,n/2\}$ and $q$ if $i\in\{n/2+1,\dotsc,n\}$. Those variables are all bounded by $1$ so we can take $b=1$ in Bernstein’s inequality. The variance is

[TABLE]

From (3.10) and Bernstein’s inequality we have

[TABLE]

Let $h(n)$ be any sequence of positive numbers. The choice $x=\frac{\alpha}{h(n)}$ then leads to

[TABLE]

Since we know $\alpha=\omega(\log n)$, any choice of $h(n)$ growing to $\infty$ slowly enough will be sufficient; for instance if $\alpha=\log(n)f(n)$ with $f(n)\to\infty$, we take $h(n)=f(n)^{1/3}$ and we obtain that $\max_{i\in[n]}|d_{i}-\alpha|=o(\alpha)$ with probability $1-o(1)$. ∎

4. Proof of Theorem 1.1

4.1. Existence of bulk insiders

In this section we prove the existence of the isolated eigenvalues inside the bulk. To do this, we compare the spectrum of $H$ (defined in (1.3)) and $H_{0}$ (defined in (3.2)). We also need to compare the spectrum of $H^{-1}$ and $H_{0}^{-1}$ to have a more refined estimate compared to [31]. See Remark 4.3 for further discussion.

Fix any non-singular square matrix $X$. One can easily check that

[TABLE]

and by conjugation the spectrum of this matrix is the same as the spectrum of the matrix

[TABLE]

Let us introduce the matrices

[TABLE]

These matrices have the same spectrum as (respectively) $H^{-1}$ and $H_{0}^{-1}$ and we are going to apply Theorem 2.2 to them. First, one has to note that the spectrum of $K$ is indeed bounded away from zero. More precisely, all the eigenvalues of $H$ are bounded below by $1$, as explained in the following statement (Theorem 3.7 in [5], the same result for finite graphs was first given in [20]): let $d_{\min}\geq 2$ and $d_{\max}$ be the minimal and maximal degrees of some finite or infinite graph $G$. Then the spectrum of $B$ is included in

[TABLE]

We see from Lemma 3.6 that with high probability all the degrees in our graph are greater than $2$, hence every eigenvalue of $H$ has modulus greater than $1$, thus ensuring that every eigenvalue $\mu$ of $H^{-1}$ has $|\mu|\leqslant 1$. We now apply Theorem 2.2 to $K$ and $K_{0}$. It is easily seen from (4.2) that $K_{0}$ is QEP-diagonalizable and the change-of-basis matrix $P$ is unitary since $A$ is Hermitian. We take

[TABLE]

From Theorem 2.2, we have

[TABLE]

where the last line holds with high probability from the description of the spectrum of $A$ in Proposition 3.1. It turns out that $\varepsilon=o(1)$, as a consequence of the following lemma.

Lemma 4.1**.**

For $X$ and $Y$ defined in (4.4), with high probability $\|X-Y\|=o(\alpha^{-1})$.

Proof.

Since $X,Y$ are diagonal matrices, we have

[TABLE]

By Lemma 3.6, with high probability $\max_{i\in[n]}|d_{i}-\alpha|=o(\alpha)$ and this implies the lemma. ∎

We thus have $\varepsilon=o(1)$ and now we can combine the ‘multiplicities preserved’ part of Theorem 2.2 and the description of the spectrum of $K_{0}$ in Corollary 3.3.

Remark 4.2*.*

Recall $\zeta_{1},\zeta_{2}$ from Corollary 3.3. The crucial fact here is that $\zeta_{1}\approx 1$ and $\zeta_{2}\approx\beta/\alpha$ are of order $1$ and in particular they are bounded away from [math], which is guaranteed by the third inequality in our assumption (A).

Theorem 2.2 implies that there is exactly one eigenvalue of $K$ in $\mathcal{B}(\zeta_{1},\varepsilon)$, one in $\mathcal{B}(\zeta_{2},\varepsilon)$ and all the other eigenvalues have modulus $o(1)$. In other words, there are exactly two eigenvalues $\xi_{1},\xi_{2}$ of $H$ such that

[TABLE]

and all the other ones have inverse modulus $o(1)$. By the continuity of $x\mapsto x^{-1}$, we have exactly two eigenvalues of $H$,

[TABLE]

which are of order $1$, and all the other eigenvalues of $H$ have inverse modulus $\omega(1)$.

Since [math] is always an eigenvalue of the Laplacian $D-A$, we have

[TABLE]

which implies $1$ is always an eigenvalue of $H$. So $\xi_{1}$ is indeed exactly equal to $1$, otherwise we have three eigenvalues of $H$: $1,\xi_{1}$ and $\xi_{2}$ that are of order $1$, a contradiction to (4.5).

Moreover, $\xi_{2}$ must be a real eigenvalue of $H$, otherwise from the fact that the spectrum of $B$ is symmetric with respect to the real line, we would see two eigenvalues of $K$ in the ball $\mathcal{B}(\zeta_{2},\varepsilon)$, which is a contradiction to Theorem 2.2. This completes the proof of (1.5).

4.2. Existence of the outliers

We now simply apply Theorem 2.2 to the matrices $H$ defined in (1.3) and $H_{0}$ defined in (3.2). Here, $A=B$ and in fact we are in the setting of Corollary 2.4 with $X=(\alpha-1)I$ and $Y=D-I$. Hence

[TABLE]

with high probability from Lemma 3.6. From the description of the spectrum of $H_{0}$ in Proposition 3.2, we see that there are two outliers located near $\beta$ and $\alpha$ and all other eigenvalues have order $O(\sqrt{\alpha})$. From this and the ‘multiplicities preserved’ part in Corollary 2.4, we see that $H$ has two outliers located within distance $o(\sqrt{\alpha})$ of

[TABLE]

By the symmetry of the spectrum with respect to the real line, those two outliers are real numbers. This completes the proof of (1.4).

Remark 4.3*.*

Note that we could also use this strategy to infer the existence of the bulk insiders: in fact, the result would yield the existence of two eigenvalues located in the balls $\mathcal{B}(1,\varepsilon)$ and $\mathcal{B}(\alpha/\beta,\varepsilon)$. These eigenvalues would be detached from the bulk of eigenvalues of $H$, which lie within distance $o(\sqrt{\alpha})$ of the circle of radius $\sqrt{\alpha}$; however, no further information can be inferred, since $o(\sqrt{\alpha})$ can go to infinity as well. This is the reason why we had to compare $H^{-1}$ with $H_{0}^{-1}$, which has two effects: first, it isolates the two ‘insiders’ of $H_{0}$ and the other eigenvalues close to zero; and secondly, it turns out that the norm of $H^{-1}-H_{0}^{-1}$ is very small. In addition, our use of the specific Bauer-Fike theorem designed for QEP (Theorem 2.2) yields more precise results than [31].

4.3. Global spectral distribution

We now prove the ‘bulk’ part of Theorem 1.1. The strategy is the same as [31] and we borrow their main theorem.

Theorem 4.4** (Corollary 3.3. in [31]).**

Let $M_{m}$ and $P_{m}$ be $m\times m$ matrices with entries in complex numbers, and let $f(z,m)\geq 1$ be a real function depending on $z,m$. Let $\mu_{M}$ be the empirical spectral distribution of any square matrix $M$. Assume that

[TABLE]

is bounded in probability, and

[TABLE]

in probability, and for almost every complex number $z\in\mathbb{C}$,

[TABLE]

with probability tending to $1$, then $\mu_{M_{m}}-\mu_{M_{m}+P_{m}}$ converges in probability to zero.

Recall $H$ from (1.3) and $H_{0}$ from (3.2). Take

[TABLE]

in Theorem 4.4. If all the conditions in Theorem 4.4 hold, then the ‘bulk’ part of Theorem 1.1 follows from our Theorem 3.5.

The condition (4.6) follows verbatim from the proof of Lemma 3.7. in [31]. Condition (4.7) follows from Lemma 3.9. in [31] and our Lemma 3.6. Condition (4.8) follows from Lemma 3.9. in [31]. This completes the proof of global spectral distribution part of Theorem 1.1.

Acknowledgements

We would like to thank Lorenzo Dall’Amico for sharing the conjecture in [15] and helpful discussions. We also thank Ke Wang for a careful reading of this paper and her useful suggestions. We are grateful to the organizers of the conference Random Matrices and Random Graphs at CIRM, during which this work was initiated. Y.Z. is partially supported by NSF DMS-1949617.