Joint CLT for top eigenvalues of sample covariance matrices of separable high dimensional long memory processes

TL;DR

This paper derives the joint central limit theorem for the top eigenvalues of sample covariance matrices from high-dimensional long memory processes with separable dependence, extending previous single-eigenvalue results.

Contribution

It extends prior work by establishing the joint CLT for multiple top eigenvalues in high-dimensional long memory settings with Toeplitz dependence structures.

Findings

Joint CLT for top eigenvalues of sample covariance matrices.

Spectral gap properties for largest eigenvalues.

Delocalization of eigenvectors associated with top eigenvalues.

Abstract

For $N,n\in\mathbb N$, consider the sample covariance matrix $$S_N(T)=\frac{1}{N}XX^*$$ from a data set $X=C_N^{1/2}ZT_n^{1/2}$, where $Z=(Z_{i,j})$ is a $N\times n$ matrix having i.i.d. entries with mean zero and variance one, and $C_N, T_n$ are deterministic positive semi-definite Hermitian matrices of dimension $N$ and $n$, respectively. We assume that $(C_N)_N$ is bounded in spectral norm, and $T_n$ is a Toeplitz matrix with its largest eigenvalues diverging to infinity. The matrix $X$ can be viewed as a data set of an $N$-dimensional long memory stationary process having separable dependence structure. As $N,n\to \infty$ and $Nn^{-1} \to r\in (0,\infty)$, we establish the asymptotics and the joint CLT for $(\lambda_1(S_N(T)),\cdots, \lambda_m(S_N(T))^\top$ where $\lambda_j(S_N(T))$ denotes the $j$th largest eigenvalue of $S_N(T)$, and $m$ is a fixed integer. For the CLT, we first study the case where the entries of $Z$ are Gaussian, and then we generalize the result to some more generic cases. This result substantially extends our previous result in Merlev\`ede et al. 2019, where we studied $\lambda_1(S_N(T))$ in the case where $m=1$ and $X=ZT_n^{1/2}$ with $Z$ having Gaussian entries. In order to establish this CLT, we are led to study the first order asymptotics of the largest eigenvalues and the associated eigenvectors of some deterministic Toeplitz matrices. We are specially interested in the autocovariance matrices of long memory stationary processes. We prove multiple spectral gap properties for the largest eigenvalues and a delocalization property for their associated eigenvectors.

Equations

$$S_N(T)=\frac{1}{N}X{X}^{\ast }$$

$$\gamma (h)=\frac{L_1(|h|)}{(1+|h|{)}^{\rho }},$$

$$\phi (x)=\frac{L_2(|x{|}^{-1})}{|x{|}^{1-\rho }},\text{for }x\in [-\pi ,\pi ],$$

$$S:=\frac{1}{N}X{X}^{\ast }=\frac{1}{N}{C}_N^{1/2}Z{T}_n{Z}^{\ast }{C}_N^{1/2}.$$

$$S_N(\Gamma ):=\frac{1}{N}{C}_N^{1/2}Z{\Gamma }_n{Z}^{\ast }{C}_N^{1/2}.$$

$$U=\left\{(s(z),g_{\Gamma }(z),g_C(z)):\Im s(z)<0,\Im (z{g}_{\Gamma }(z))<0,\Im (z{g}_C(z))<0\right\}$$

$$\{\begin{array}{l}s(z)=z^{-1}(1-r)+z^{-1}r\int \frac{1}{1-g_C(z)x}\mathrm{d}{\nu }_{\Gamma }(x),\\ s(z)=z^{-1}\int \frac{1}{1-g_{\Gamma }(z)x}\mathrm{d}{\nu }_C(x),\\ s(z)=z^{-1}+g_{\Gamma }(z)g_C(z).\end{array}$$

$${\lambda }_j(T_n)\sim 2\Gamma (\rho )\sin \left(\frac{\rho \pi }{2}\right)n^{1-\rho }{L}_2(n){\lambda }_j({\mathcal{K}}^{(\rho )})$$

$${\lambda }_j(S_N(T))\sim {\lambda }_j(T_n)\frac{\mathrm{tr}{C}_N}{N}.$$

$$\sqrt{N}{\left(\begin{array}{ccc}\frac{{\lambda }_1(S_N(T))}{{\lambda }_1(T_n)}-{\theta }_1& \cdots & \frac{{\lambda }_m(S_N(T))}{{\lambda }_m(T_n)}-{\theta }_m\end{array}\right)}^{\top }\stackrel{\mathcal{D}}{\to }\mathcal{N}(0,{\sigma }^2{I}_m)$$

$${\lambda }_1(A)\ge {\lambda }_2(A)\ge \dots$$

$$d_{\mathcal{L}\mathcal{P}}(P,Q):=\inf \{\epsilon :P(A)\le Q(A^{\epsilon })+\epsilon ,Q(A)\le P(A^{\epsilon })+\epsilon ,\forall A\in \mathcal{B}({\mathbb{R}}^m)\},$$

$$A^{\epsilon }:=\{x\in {\mathbb{R}}^m:\exists y\in A,\text{ s.t. }\Vert x-y\Vert <\epsilon \}.$$

$$({\mathcal{K}}^{(\rho )}f)(x)={\int }_0^1\frac{f(y)}{|x-y{|}^{\rho }}\mathrm{d}y,\text{ for }f\in L^2(0,1).$$

$$\underset{n\to \infty }{\lim }\frac{{\lambda }_j(T_n)}{n^{1-\rho }{L}_1(n)}={\lambda }_j({\mathcal{K}}^{(\rho )})>0.$$

$$\underset{n\to \infty }{\lim }\frac{{\lambda }_2(T_n)}{{\lambda }_1(T_n)}<1.$$

$$\underset{n\to \infty }{\lim }\frac{{\lambda }_{j+1}(T_n)}{{\lambda }_j(T_n)}<1.$$

$$\dim \ker \left(\lambda I-{\mathcal{K}}^{(\rho )}\right)=1.$$

$$\underset{n\to \infty }{\lim }\underset{1\le k\le n}{\sup }\left\{\left|\sqrt{n}{u}_{j,k}-f_j\left(\frac{k}{n}\right)\right|\right\}=0.$$

$$\Vert u_j{\Vert }_{\infty }\le \frac{1+\Vert f_j{\Vert }_{\infty }}{\sqrt{n}},$$

$$\Vert u_j{\Vert }_{\infty }\to 0.$$

$$L_1(h)\sim L_2(h)2\Gamma (\rho )\sin \left(\frac{\rho \pi }{2}\right)\text{as }h\to \infty ,$$

$$\Vert \frac{T_n}{n^{1-\rho }{L}_2(n)}-\frac{T_n^{\prime }}{n^{1-\rho }}\Vert \underset{n\to \infty }{\to }0.$$

$$\underset{n\to \infty }{\lim }\frac{{\lambda }_j(T_n)}{n^{1-\rho }{L}_2(n)}=2\Gamma (\rho )\sin \left(\frac{\rho \pi }{2}\right){\lambda }_j({\mathcal{K}}^{(\rho )})>0.$$

$$\Vert u_j-u_j^{\prime }{\Vert }_{\infty }\le \Vert u_j-u_j^{\prime }\Vert \underset{n\to \infty }{\to }0.$$

$$\int x\mathrm{d}{\mu }^{{\tilde{T}}_n}(x)=\frac{\mathrm{tr}{\tilde{T}}_n}{n}=o(1/\sqrt{n}).$$

$$\int x^2\mathrm{d}{\mu }^{{\tilde{T}}_n}(x)=\frac{\mathrm{tr}{\tilde{T}}_n^2}{n}=o(1/\sqrt{n}).$$

$$S_N(\Gamma ):=\frac{1}{N}{C}_N^{1/2}Z{\Gamma }_n{Z}^{\ast }{C}_N^{1/2},$$

$$c_1\ge c_2\ge \cdots \ge c_N\text{and}{t}_1\ge t_2\ge \cdots \ge t_n$$

$$\mathbb{E}{Z}_{i,j}=0,\mathbb{E}|Z_{i,j}{|}^2=1\text{and}\mathbb{E}|Z_{i,j}{|}^4<\infty ,$$

Full text

[TABLE]

from a data set $X=C_{N}^{1/2}ZT_{n}^{1/2}$, where $Z=(Z_{i,j})$ is a $N\times n$ matrix having i.i.d. entries with mean zero and variance one, and $C_{N},T_{n}$ are deterministic positive semi-definite Hermitian matrices of dimension $N$ and $n$, respectively. We assume that $(C_{N})_{N}$ is bounded in spectral norm, and $T_{n}$ is a Toeplitz matrix with its largest eigenvalues diverging to infinity. The matrix $X$ can be viewed as a data set of an $N$-dimensional long memory stationary process having separable dependence structure.

As $N,n\to\infty$ and $Nn^{-1}\to r\in(0,\infty)$, we establish the asymptotics and the joint CLT for $(\lambda_{1}(S_{N}(T)),\cdots,\lambda_{m}(S_{N}(T))^{\top}$ where $\lambda_{j}(S_{N}(T))$ denotes the $j$th largest eigenvalue of $S_{N}(T)$, and $m$ is a fixed integer. For the CLT, we first study the case where the entries of $Z$ are Gaussian, and then we generalize the result to some more generic cases. This result substantially extends our previous result in [28], where we studied $\lambda_{1}(S_{N}(T))$ in the case where $m=1$ and $X=ZT_{n}^{1/2}$ with $Z$ having Gaussian entries.

In order to establish this CLT, we are led to study the first order asymptotics of the largest eigenvalues and the associated eigenvectors of some deterministic Toeplitz matrices. We are specially interested in the autocovariance matrices of long memory stationary processes. We prove multiple spectral gap properties for the largest eigenvalues and a delocalization property for their associated eigenvectors.

1 Introduction

Background and related work.

For $N,n\in\mathbb{N}$, we set $X=C_{N}^{1/2}ZT_{n}^{1/2}$, where $Z=(Z_{i,j})$ is a $N\times n$ matrix having i.i.d. entries with mean zero and variance one, $C_{N}$ is a $N\times N$ deterministic positive semi-definite Hermitian matrix, and $T_{n}=(\gamma(i-j))_{1\leq i,j\leq n}$ be a positive semi-definite Toeplitz matrix. Then $X$ models a sample data set of an $N$-dimensional stationary process with a separable dependence structure. If the entries of $T_{n}$ satisfies the following power decay condition:

[TABLE]

or the spectral density $\varphi$ of $T_{n}$ has a power singularity at [math]:

[TABLE]

where $\rho\in(0,1)$ and $L_{1},L_{2}$ are positive, locally bounded functions which are slowly varying at $\infty$, then the process is long range dependent (LRD) or has long memory (LM) (see for example [31]). Note that the conditions (1.1) and (1.2) are not equivalent but tightly related, see Section 2.2.5 of [31].

From the sample data matrix $X$, we construct the sample covariance matrix

[TABLE]

Let $m\geq 1$ be a fixed integer. We will study the asymptotics and joint CLT of the largest eigenvalues of $S$ as $N,n\to\infty$ and $r_{n}:=Nn^{-1}\to r\in(0,\infty)$.

For convenience of further discussion, we define the notation $S_{N}(\Gamma)$ by

[TABLE]

where $\Gamma_{n}$ is a $n\times n$ deterministic positive semi-definite Hermitian matrix. Then $S$ in (1.3) will be denoted as $S_{N}(T)$, and if $\Gamma_{n}=I_{n}$ is identity, $S_{N}(I)$ is the classical sample covariance matrix.

The classical sample covariance model $S_{N}(I)$ has been intensively studied in the last decades. These studies are mostly concentrated on the global behaviors of the spectrum, including limiting spectral distribution (LSD) ([27, 37, 21, 40, 35, 34]) and CLT for linear spectral statistics ([3, 4, 18, 29]); and also the local behaviors of individual eigenvalues ([7, 19, 20, 15, 24, 8, 23, 5, 6]).

Recently several models of matrices $S_{N}(I)$ with $C_{N}$ having a small number of divergent eigenvalues have been considered, in the context of principal component analysis (PCA) [22, 33, 38, 11] and long memory processes [28]. Although the assumptions in these various works differ, many results coincide with the degenerated case of Bai and Yao [6] after normalization (see for example [28]).

The model $S_{N}(I)$ assumes that the columns of the data matrix $X$ are i.i.d. However this is not always the case in the practical applications. The separable model $S_{N}(\Gamma)$ introduces a special type of correlations between columns, or different weights on columns, achieving certain balance between generality and simplicity. So it attracts more and more attention nowadays. A first result on this model is due to Zhang [43] on the LSD of $S_{N}(\Gamma)$. She proved that if the empirical spectral distribution (ESD) $\mu^{\Gamma_{n}}$ of $\Gamma_{n}$ and the ESD $\mu^{C_{N}}$ of $C_{N}$ converge weakly to $\nu_{\Gamma}$ and $\nu_{C}$ respectively, then as $N,n\to\infty$, the ESD $\mu^{S_{N}(\Gamma)}$ of $S_{N}(\Gamma)$ will converge weakly to a non-random probability measure $\mu$ for which if $\nu_{C}=\delta_{0}$ or $\nu_{\Gamma}=\delta_{0}$, then $\mu=\delta_{0}$; otherwise for each $z\in\mathbb{C}^{+}:=\{z\in\mathbb{C}\,:\,\Im z>0\}$, the Cauchy-Stieltjes transform $s(z)$ of $\mu$, together with another two functions, denoted by $g_{\Gamma}(z)$ and $g_{C}(z)$, $(s(z),g_{\Gamma}(z),g_{C}(z))$ is the unique solution in the set

[TABLE]

to the following system of equations

[TABLE]

Later, Paul and Silverstein [30] proved in the case where $\Gamma_{n}$ is diagonal, that almost surely, for large enough $N,n$, there is no eigenvalue of $S_{N}(\Gamma)$ in any closed interval outside the support of the limiting spectral distribution (LSD). This is an extension of the results of [2] for $S_{N}(I)$. Couillet and Hachem [12] studied the analytical properties of the LSD $\mu$ when $\nu_{\Gamma}\neq\delta_{0}$ and $\nu_{C}\neq\delta_{0}$, including the determination of the support of $\mu$, extending the work of Choi and Silverstein [36] for $S_{N}(I)$ to the separable model $S_{N}(\Gamma)$. The CLT for linear spectral statistics has also been studied by Bai et al. [1] and Li et al. [25].

Regarding the extreme eigenvalues of $S_{N}(\Gamma)$, far less is known compared to the classical model $S_{N}(I)$. In [39], Yang proved the edge universality under the condition that the densities of the LSD’s $\nu_{\Gamma},\nu_{C}$ have a regular square-root behavior at the rightmost edge (soft edge). With this result, if we find the fluctuations of the largest eigenvalue $\lambda_{1}(S_{N}(\Gamma))$ at a soft edge in the case where the entries $Z_{i,j}$ are Gaussian, then the fluctuations at a soft edge in general cases will be determined. However, even in the Gaussian case, these fluctuations are still unknown.

The general spiked eigenvalues of $S_{N}(\Gamma)$ are also a new topic in the recent studies. In [42], a very particular case of this problem has been touched. One can refer to Remark 2.4 for the relations between the concerned results of [42] and the ours. During the preparation of the present paper, we learned that a newly submitted paper [14] treated the general spike separable model $S_{N}(\Gamma)$ with $C_{N},\Gamma_{n}$ general Hermitian matrices having a finite number of spikes. The authors studied the asymptotics and large deviations (instead of joint CLT) of spiked eigenvalues of $S_{N}(\Gamma)$, and the associated eigenvectors. The main restriction of [14] is that they assumed that $C_{N}$ and $\Gamma_{n}$ are all bounded in spectral norm, and that the spectrums of both $C_{N}$ and $\Gamma_{n}$ do not concentrate at zero, also that the number of spikes is finite. These assumptions exclude our model $S_{N}(T)$ from applying their results.

Introduction to the results.

The present paper aims at studying the asymptotics and the joint CLT of $m$ (with $m\geq 1$ an arbitrary fixed integer) largest eigenvalues of $S_{N}(T)$. The basic idea is analogous to the previous article [28] but we extend substantially the results of that paper.

In order to study the largest eigenvalues of $S_{N}(T)$, the asymptotics of largest eigenvalues and the associated eigenvectors of $T_{n}$ satisfying (1.1) or (1.2) will be studied. In [28] we proved that the ESD of $T_{n}$ converges weakly to a non-compact supported measure, thus the largest eigenvalues of $T_{n}$ diverges to infinity. We also studied the asymptotic behavior of largest eigenvalues of $T_{n}$ if it satisfies (1.1), and proved that $\lambda_{j}(T_{n})\sim n^{1-\rho}L_{1}(n)\lambda_{j}(\mathcal{K}^{(\rho)})$ as $n\to\infty$ and $j\geq 1$ is fixed, where $\mathcal{K}^{(\rho)}$ is a compact operator defined in (2.1) below. By proving the simplicity of $\lambda_{1}(\mathcal{K}^{(\rho)})$, we proved also the spectral gap property for the largest eigenvalue of $T_{n}$, that is, $\lambda_{2}(T_{n})/\lambda_{1}(T_{n})$ is bounded by a constant smaller than $1$. However, it is well known that the two conditions (1.1) and (1.2) are not equivalent without the quasi-monotone conditions on $L_{1}$ or $L_{2}$. For example, Gubner [17] gave counterexamples in both directions of implication. In this paper, we will prove that if $T_{n}$ satisfies (1.2), without assuming quasi-monotonicity of $L_{2}$, we have

[TABLE]

as $n\to\infty$, where $\Gamma$ is the Gamma-function. We will also prove that all nonzero eigenvalues of $\mathcal{K}^{(\rho)}$ are simple, using a method totally different from [28]. In consequence, the spectral gap property holds for any finite number of largest eigenvalues of $T_{n}$. Furthermore, we will study the relation between the eigenvectors of $T_{n}$ and the eigenfunctions of $\mathcal{K}^{(\rho)}$, and prove that the eigenvectors associated with the largest eigenvalues of $T_{n}$ are delocalized. These results may have independent interests.

Using the results on Toeplitz matrices, we study the asymptotics and fluctuations of largest eigenvalues of $S_{N}(T)$ as $N,n\to\infty$ with $N/n\to r\in(0,\infty)$. We first prove the following asymptotic behavior of $\lambda_{j}(S_{N}(T))$ for any fixed $j$. Assume that the entries of $Z$ have finite fourth moment, and some other conditions, then for any $j\geq 1$, in probability,

[TABLE]

Moreover, if the entries $Z_{i,j}$’s are Gaussian, (1.7) holds almost surely.

Then we will build the joint CLT of largest eigenvalues for the generic model $S_{N}(\Gamma)$ which implies in particular the following results: suppose that $T_{n}$ satisfies 1.1 or 1.2, and $Z$ has i.i.d. standard Gaussian entries, with $C_{N}$ satisfies some mild conditions, then as $N,n\to\infty$ with $N/n\to r\in(0,\infty)$,

[TABLE]

where $\sigma^{2}=2$ when the entries of $Z$ are real Gaussian, and $\sigma^{2}=1$ when the entries of $Z$ are complex Gaussian, and $\theta_{j}$ are determined by some equations. We then generalize the result to the non-Gaussian case.

If the entries of $Z$ are not Gaussian, it is more complicated. The CLT of eigenvalues of $S_{N}(T)$ depends also on the eigenvectors of $T_{n}$ and $C_{N}$. We will prove a general theorem implying that, when $C_{N}$ is diagonal, and the parameter $\rho$ of $T_{n}$ defined in (1.1) or (1.2) belongs to $(0,3/4)$, that is, the decay of the correlation of the process is sufficiently slow, then (1.8) still holds with the same $\theta_{j}$ and $\sigma^{2}=1+|\mathbb{E}Z_{1,1}^{2}|^{2}$, where $Z_{1,1}$ is the entry of $Z$ in the first row and first column. Note that if $Z_{1,1}$ is real with variance one, or if $Z_{1,1}$ is complex with $\mathbb{E}Z_{1,1}^{2}=0$, we will get the same CLT as the Gaussian case. This phenomenon is due to the delocalization of the largest eigenvectors of Toeplitz matrices $T_{n}$.

Organizations.

This paper is organized as follows. In Section 2 we state our main theorems. This section is divided in three parts. In 2.1, we state the results on Toeplitz matrices $T_{n}$; in 2.2, we state the asymptotics of the largest eigenvalues of $S_{N}(\Gamma)$; in 2.3 we state the CLT for largest eigenvalues of $S_{N}(\Gamma)$ in the case where $C_{N},\Gamma_{n}$ are diagonal, or where $Z_{i,j}$ are Gaussian; in 2.4, we present some generalizations of the CLT with non-diagonal $\Gamma_{n}$. The other sections contains the proofs of these results.

Notations.

For a Hermitian operator or matrix $A$, we denote its real eigenvalues by decreasing order as

[TABLE]

For a matrix or a vector $A$, we use $A^{\top}$ to denote the transpose of $A$, and $A^{*}$ the conjugate transpose of $A$. For a $N\times N$ matrix $A$, we denote the ESD of $A$ by $\mu^{A}$, which is defined by $\mu^{A}:=N^{-1}\sum_{i=1}^{N}\delta_{\lambda_{i}(A)}$, where $\delta_{x}$ is the Dirac measure at $x$.

The kernel of a linear operator $A:X\rightarrow X$ is denoted by $\ker A$. The spectrum of $A$ is denoted by $\operatorname*{Spec}(A)$. We denote the $L^{p}$ or $l^{p}$ norm by $\|\cdot\|_{p}$. For a matrix or a linear operator $A$, the operator norm of $A$ induced by vector norm $\|\cdot\|_{p}$ is denoted by $\|A\|_{p}$, and we recall that $\|A\|_{p}:=\sup_{\|v\|_{p}=1}\|Av\|_{p}$. The $L^{2}$ or $l^{2}$ norm will be abbreviated as $\|\cdot\|$. We say that a function $f$ or a vector $v$ is "normalized" or "unit length" when $\|f\|=1$ or $\|v\|=1$. When functions or vectors are said to be "orthonormal", they will be implicitly considered as elements of a Hilbert space.

For two probability measures $P$ and $Q$ on $\mathbb{R}^{m}$, we denote their Lévy-Prokhorov distance by $d_{\mathcal{LP}}(P,Q)$ which is defined by

[TABLE]

where $A^{\varepsilon}$ is defined by

[TABLE]

It is well known that this distance metrizes the weak convergence. For two random variables $X,Y\in\mathbb{R}^{m}$ with distributions $\mu_{X},\mu_{Y}$, respectively, we sometimes write $d_{\mathcal{LP}}(X,Y)$, $d_{\mathcal{LP}}(X,\mu_{Y})$ or $d_{\mathcal{LP}}(\mu_{X},Y)$ which all mean $d_{\mathcal{LP}}(\mu_{X},\mu_{Y})$.

Given $x\in\mathbb{R}$, we denote by $\lfloor x\rfloor$ the integer satisfying $\lfloor x\rfloor\leq x<\lfloor x\rfloor+1$. Given two sequences of non-negative numbers $x_{n},y_{n}$, $x_{n}\sim y_{n}$ means that $\lim_{n\to\infty}x_{n}/y_{n}=1$. If $X_{n},X$ are random variables, the notation $X_{n}=o_{P}(1)$ means that $\lim_{n\to\infty}X_{n}=0$ in probability. The notations $X_{n}\xrightarrow{\mathcal{D}}X$ and $X_{n}\xrightarrow{\mathcal{P}}X$ denote convergence in distribution and in probability, respectively. If $\mu_{n},\mu$ are measures, we denote with a slight abuse of notation $\mu_{n}\xrightarrow{\mathcal{D}}\mu$ for the weak convergence of $\mu_{n}$ to $\mu$.

In order to estimate some quantities we need sometimes split it into several parts. When we use $P_{1},P_{2},\dots$ to denote these parts, their definition is limited in the same proof, the same section or subsection.

Definition 1**.**

We say that a sequence of events $E_{n}$ hold with high probability, if $\mathbb{P}(E_{n})=1-o(1)$; with low probability, if $\mathbb{P}(E_{n})=o(1)$; with overwhelming probability, if for any $M>0$, $\mathbb{P}(E_{n})=1-o(n^{-M})$; with tiny probability, if for any $M>0$, $\mathbb{P}(E_{n})=o(n^{-M})$.

The cardinal of a set $B$ is denoted by $\#B$. In the proofs we use $K$ to denote a constant that may take different values from one place to another. If the constant depends on some parameter $p$, we denote the constant by $K_{p}$.

Acknowledgement.

Part of this work was completed during my PhD study, and was supported financially by Université Paris-Est and Labex Bézout. I would like to thank gratefully my Ph-D advisers Professor Florence MÈRLEVEDE and Professor Jamal NAJIM. Also thanks to Professor Jianfeng YAO in the University of Hong Kong for fruitful discussions.

2 Main theorems

2.1 Spectral properties of Toeplitz matrices

We collect our results on Toeplitz matrices in this section. Let $T_{n}=\left(\gamma(i-j)\right)_{i,j=1}^{n}$ satisfy (1.1). Let $\mathcal{K}^{(\rho)}$ be the operator defined on $L^{2}(0,1)$ by

[TABLE]

In [28], we have established the relation between the eigenvalues of $T_{n}$ and the eigenvalues of $\mathcal{K}^{(\rho)}$. From the proof of Theorem 2.3 of [28] we know that the operator $\mathcal{K}^{(\rho)}$ is compact and positive semi-definite. It has infinitely many positive eigenvalues. And if $T_{n}$ is defined by (1.1), then for any $j\geq 1$, we have

[TABLE]

Using the min-max formula for the largest eigenvalue and an argument by absurd, we have also proved in [28] that $\lambda_{1}(\mathcal{K}^{(\rho)})$ is simple, so that we proved the spectral gap property for the largest two eigenvalues of $T_{n}$:

[TABLE]

In this paper, using a different method, we will prove that all non-zero eigenvalues of $\mathcal{K}^{(\rho)}$ are simple. As a consequence we prove the multiple spectral gap property for any $j$th largest eigenvalue of $T_{n}$:

[TABLE]

Proposition 2.1**.**

All non-zero eigenvalues of the operator $\mathcal{K}^{(\rho)}$ defined by (2.1) with $\rho\in(0,1)$ are simple, and the associated eigenfunctions are continuous in $[0,1]$.

We note that $\mathcal{K}^{(\rho)}$ is self-adjoint, so for any non-zero eigenvalue $\lambda$, its algebraic multiplicity equals to its geometric multiplicity, which is defined as $\dim\ker(\lambda I-\mathcal{K}^{(\rho)})$. For more information about algebraic multiplicity, see [26]. So here to say that a non-zero eigenvalue $\lambda$ is simple, means

[TABLE]

In the next proposition, we provide a quantitative description of the eigenvectors associated with $\lambda_{j}(T_{n})$ for any fixed $j$.

Proposition 2.2**.**

Let $T_{n}$ satisfy (1.1). For any $j\geq 1$, let $f_{j}$ be the normalized eigenfunction of $\mathcal{K}^{(\rho)}$ associated with $\lambda_{j}(\mathcal{K}^{(\rho)})$, and $u_{j}=(u_{j,1},\dots,u_{j,n})^{\top}$ be a normalized eigenvector of $T_{n}$ associated with $\lambda_{j}(T_{n})$. Then, up to a change of sign, we have

[TABLE]

From this proposition we deduce the delocalization of eigenvector $u_{j}$ associated with $\lambda_{j}(T_{n})$ for any fixed $j\geq 1$. Indeed by (2.4), for large enough $n$, we have

[TABLE]

and because $f_{j}$ is continuous on $[0,1]$, we have $\|f_{j}\|_{\infty}<\infty$. Thus we conclude that

[TABLE]

The above propositions also applies to some Toeplitz matrices satisfying (1.2). It is well known that if $T_{n}$ satisfies (1.2) with quasi-monotonic $L_{2}$ (see Section 2.2.5 of [31] for definition), then $T_{n}$ also satisfies (1.1) with

[TABLE]

where $\Gamma$ is the Gamma-function. In particular, if $L_{2}\equiv 1$, then $L_{1}(h)$ tends to a constant as $h\to\infty$, and (2.2), Proposition 2.2 hold.

Without the condition of quasi-monotonicity on $L_{1}$ or $L_{2}$, the conditions (1.1) and (1.2) are not equivalent. See [17] for counterexamples in both directions. However, thanks to the following theorem, the above results can be extended to $T_{n}$ defined by (1.2) with general slowly varying function $L_{2}$.

Theorem 2.3**.**

Let $T_{n}$ and $T^{\prime}_{n}$ be $n\times n$ Toeplitz matrices both satisfying (1.2) with the same $\rho\in(0,1)$, with an arbitrary slowly varying function $L_{2}$ for $T_{n}$, and with $L_{2}^{\prime}\equiv 1$ for $T^{\prime}_{n}$. Then

[TABLE]

In consequence, for any fixed $j\geq 1$, we have

[TABLE]

Moreover, if $u_{j}$ (resp. $u_{j}^{\prime}$) is the eigenvector of $T_{n}$ (resp. $T^{\prime}_{n}$) associated with the $j$th largest eigenvalue, then

[TABLE]

Let $\tilde{T}_{n}:=T_{n}/\|T_{n}\|$. The following proposition provides the decay of moments of the ESD $\mu^{\tilde{T}_{n}}$ for $T_{n}$ satisfying (1.1) or (1.2). This result shows that the $T_{n}$ with parameter $\rho\in(0,3/4)$ satisfies A9 or A10 below, and is needed in the proof of Theorem 2.10.

Proposition 2.4**.**

Let $T_{n}$ be $n\times n$ Toeplitz matrix satisfying (1.1) or (1.2), and let $\tilde{T}_{n}=T_{n}/\|T_{n}\|$.

If $\rho\in(0,1/2)$, then

[TABLE] 2. 2.

If $\rho\in[1/2,3/4)$, then

[TABLE]

2.2 Convergence of largest eigenvalues of separable sample covariance matrix

For $N,n\in\mathbb{N}$, let

[TABLE]

where $C_{N},\Gamma_{n}$ are $N\times N$ and $n\times n$ deterministic positive semi-definite Hermitian matrices, and $Z$ is a $N\times n$ matrix having i.i.d. entries $Z_{i,j}$. Let

[TABLE]

be the eigenvalues of $C_{N}$ and $\Gamma_{n}$ respectively. Let $m\geq 1$ be a fixed integer. We assume that the following assumptions hold:

1. A1

The entries $Z_{i,j}$ satisfy

[TABLE] 2. A2

The spectral norm $C_{N}$ is bounded in $N$, and the ESD $\mu^{C_{N}}$ of $C_{N}$ converges weakly as $N\to\infty$, to a probability measure $\nu_{C}\neq\delta_{0}$. 3. A3

There exists a decreasing sequence of positive numbers

[TABLE]

converging to [math] such that for any $j\geq 1$, we have

[TABLE]

Note that under A3, we have $\mu^{\Gamma_{n}}\xrightarrow{\mathcal{D}}\delta_{0}$, and for any $\varepsilon>0$,

[TABLE]

For further use, we will prove a concentration inequality for the largest eigenvalues of $S_{N}(\Gamma)$ with the following conditions.

1. A4

The matrices $C_{N}$ are diagonal:

[TABLE] 2. A5

The matrices $\Gamma_{n}$ are diagonal:

[TABLE] 3. A6

(Bound condition) There exists a sequence of positive numbers $\varepsilon_{n}\to 0$ such that almost surely for large enough $N,n$,

[TABLE]

Remark 2.1*.*

We take two examples for which the bound condition A6 holds. The first case is where $\mathbb{E}|Z_{i,j}|^{6+\epsilon}<\infty$ for some $\epsilon>0$. In this case, we have

[TABLE]

where we have assumed that the convergence rate of $\varepsilon_{n}$ to [math] is slower than any preassigned rate. Then by Borel-Cantelli’s Lemma, the bound condition holds. The second case is where $\mathbb{E}|Z_{i,j}|^{4}<\infty$, and $Z_{i,j}$ does not depend on $N,n$ for any fixed $i,j$. In other words, $Z_{i,j}$ are all from an infinite double array $(Z_{i,j})_{i,j\geq 1}$. In this case, by the truncation lemma 2.2 of [41], the bound condition holds.

Recall that we use $N,n\to\infty$ to denote $N,n\to\infty,N/n\to r\in(0,\infty)$.

Proposition 2.5**.**

Let $S_{N}(\Gamma)$ be defined as (2.11). Under A1, A2 and A3, for any $j\geq 1$, we have

[TABLE]

Moreover, if $Z_{i,j}$’s are real or complex Gaussian, or if A4, A5 and A6 hold, then the above convergence holds almost surely.

Remark 2.2*.*

The almost sure convergence under A4, A5 and A6 is in fact a byproduct of Lemma 4.1 which is needed in the proof of CLT 2.7. However this does not allow to conclude the a.s. convergence when $Z_{i,j}$’s are Gaussian. Indeed if the entries of $Z$ are i.i.d real Gaussian variables, and if $C_{N}$ or $\Gamma_{n}$ are complex and non-diagonal, then we cannot diagonalize $C_{N}$ or $\Gamma_{n}$ because the real Gaussian vectors are not unitary invariant. Thus we will proceed an independent proof for Gaussian case with help of a Gaussian concentration inequality.

Applying the above generic result to the special case of $S_{N}(T/\|T\|)$, we obtain the following result:

Corollary 2.6**.**

Let $T_{n}$ be a sequence of Toeplitz matrices satisfying (1.1) or (1.2). Let $S_{N}(T)$ be defined as before. Then if A1, A2 hold, for any fixed $j\geq 1$ we have

[TABLE]

Moreover, if $Z_{i,j}$’s are standard real or complex Gaussian, then the above convergence holds almost surely.

2.3 CLT for largest eigenvalues: Diagonal & Gaussian case

In this section, we assume that $C_{N}$, $\Gamma_{n}$ are diagonal, and study the CLT for largest eigenvalues of $S_{N}(\Gamma)$. As a corollary, we obtain the result for Gaussian case.

1. A7

The sixth moment of the entries is finite:

[TABLE] 2. A8

The $m$ largest eigenvalues of $\Gamma_{n}$ satisfy the multiple spectral gap property:

[TABLE]

For $x,z\in\mathbb{C}$ we define

[TABLE]

For $j=1,\dots,m$, let $\theta_{j}$ be the largest solution of the equation

[TABLE]

Remark 2.3*.*

Note that if not all $c_{i}$’s are [math], and if $z\in\mathbb{R}$, then from the graph of the function $x\mapsto G(x,z)$, we see that the equation $G(x,z)=1$ on $x$ admits $\#\{zc_{i}\,:\,c_{i}\neq 0,i=1,\dots,n\}$ real solutions.

Moreover, we prove that the largest solution of (2.13) tends to $\int x\,\mathrm{d}\nu_{C}(x)$. Indeed, we know that under the assumptions A3 and A8, we have

[TABLE]

and the assumption A2 ensures that

[TABLE]

Also note that for every fixed $\theta\neq 0$,

[TABLE]

Thus for any $0<\epsilon<\int x\,\mathrm{d}\nu_{C}(x)$, let $\theta^{(1)}=\int x\,\mathrm{d}\nu_{C}(x)-\epsilon$, $\theta^{(2)}=\int x\,\mathrm{d}\nu_{C}(x)+\epsilon$, then we can see that asymptotically the largest solution of the equation (2.13) is between $\theta^{(1)}$ and $\theta^{(2)}$.

We define

[TABLE]

Theorem 2.7**.**

Under A1, A2, A3, A4, A5 and A7, A8, we have

[TABLE]

For general non-diagonal $C_{N}$ and $\Gamma_{n}$, note that if $Z_{i,j}$ are standard complex Gaussian, or if $Z_{i,j}$ are standard real Gaussian and $C_{N},\Gamma_{n}$ are both real, then the eigenvalues of $S_{N}(\Gamma)$ have the same joint distribution with the eigenvalues of

[TABLE]

Therefore the CLT 2.7 applies to the Gaussian case no matter whether $C_{N},\Gamma_{n}$ are diagonal. More particularly, applying the above result to $S_{N}(T)$ with $T_{n}$ the Toeplitz matrix defined as before, we get the following corollary.

Corollary 2.8**.**

Let $T_{n}$ be a sequence of Toeplitz matrices satisfying (1.1) or (1.2). Let $S_{N}(T)$ be defined as before. Let $\Lambda_{m}(T)$ be defined by replacing $\Gamma$ with $T$ in (2.14). Assume that A1 and A2 hold. Then, if $Z_{i,j}$ are standard complex Gaussian, or if $Z_{i,j}$ are standard real Gaussian and $C_{N}$ is real, we have

[TABLE]

where $\sigma^{2}=2$ in real Gaussian case, and $\sigma^{2}=1$ in complex Gaussian case.

If $C_{N}=I_{N}$, then we can see that

[TABLE]

In general, $\theta_{j}$ has not a closed expression. However, $\theta_{j}$ can be expressed as a power series of $N^{-1}\sum_{k\neq j}\frac{t_{k}}{t_{j}-t_{k}}$.

Proposition 2.9**.**

For a fixed $N$, let $m_{k}:=\operatorname*{tr}C_{N}^{k}/N$, let the coefficients $B_{0},B_{1},\dots$ be defined by the recurrent formula

[TABLE]

Suppose that not all the eigenvalues of $C_{N}$ are zero. Then the power series

[TABLE]

is the solution of the equation

[TABLE]

Its radius of convergence $R$ satisfies

[TABLE]

where we make the convention that $\inf\emptyset=+\infty$.

From this proposition, we have, under the conditions A2, A3 and A8, for large enough $N,n$,

[TABLE]

By the recurrent formula, we obtain

[TABLE]

So we have

[TABLE]

Remark 2.4*.*

In [28, Example 2.3], we have given the various orders of $\sqrt{N}(\theta_{j}-1)$ when $C_{N}=I_{N}$, so in general $\theta_{j}$ can not be replaced by a finite form. However in some particular cases, we can replace $\theta_{j}$ by a partial sum of its Taylor’s expansion. For example, when $\Gamma_{n}$ satisfies A9 below, we have $\sqrt{N}(\theta_{j}-m_{1})\to 0$. Thus we can replace $\theta_{j}$ by $m_{1}$. One can see that the model in [42] is in this case when their $\mathbf{\Pi}=\mathbf{I}$ (Theorem 3 of [42]), because their major spiked population eigenvalues are asymptotically $\lambda_{k}\sim 4T^{2}/(\pi(2k-1))^{2}$ as $T\to\infty$, where $T$ denotes the dimension (Lemma 1 and 2 of [42]). That is, with our notations,

[TABLE]

And by calculating $n^{-1/2}\operatorname*{tr}(\mathbf{C}^{*}\mathbf{C})/n^{2}$ where $\mathbf{C}$ is defined in (2.4) of [42], we have

[TABLE]

Similarly, when $\Gamma_{n}$ satisfies A10 below, $\theta_{j}$ can be replaced by

[TABLE]

2.4 CLT for largest eigenvalues: Some generalizations

In this section we generalize the CLT to non-diagonal $\Gamma_{n}$. We continue to assume the other assumptions, and moreover, we assume that one of the two following assumptions holds:

1. A9

[TABLE] 2. A10

[TABLE]

Remark 2.5*.*

Under A3, because almost all the eigenvalues of $\Gamma_{n}$ (except for at most a finite number of them) are smaller than $1$, the condition A9 is stronger than A10. They are some indicators who measure the degree of concentration of the eigenvalues near zero. If $\Gamma_{n}$ satisfies A9, then its eigenvalues are more concentrated near [math] than the case where it just satisfies A10.

From Proposition 2.4, we can see that for $\rho\in(-3/4,0)$, the normalized Toeplitz matrix $\tilde{T}_{n}$ satisfies A10, and for $\rho\in(-1/2,0)$, $\tilde{T}_{n}$ satisfies A9.

Theorem 2.10**.**

Under A1, A2, A3, A4, A8, and either A9 or A10, we have

[TABLE]

Where $\Sigma_{m}^{(N)}=\frac{\operatorname*{tr}C_{N}^{2}}{N}(I_{m}+(\sigma_{i,j})_{i,j=1}^{m})$ with

[TABLE]

and $u_{j}:=(u_{j,1},\dots,u_{j,n})^{\top}$ is a normalized eigenvector associated with $\lambda_{j}(\Gamma_{n})$.

Remark 2.6*.*

In view of the expression (2.16), it is not clear that the covariance matrix $\Sigma_{m}^{(N)}$ converges. In order to avoid any cumbersome assumption enforcing this convergence, we express the CLT with the help of Lévy-Prokhorov’s distance. If however it happens that $\Sigma_{m}^{(N)}$ converges to some matrix $\Sigma_{m}$, then we conclude the CLT in the following usual form

[TABLE]

From Proposition 2.2 and Theorem 2.3, if $T_{n}$ is a Toeplitz matrix satisfying (1.1) or (1.2), we can see that the eigenvectors $u_{j}$ of $T_{n}$ are delocalized, i.e. $\|u_{j}\|_{\infty}=o(1)$. So we have

[TABLE]

Also because $T_{n}$ are real, we have

[TABLE]

So we get the following result:

Corollary 2.11**.**

Let $T_{n}$ be a sequence of Toeplitz matrices satisfying (1.1) or (1.2). Let $S_{N}(T)$ be defined as before. Suppose that A1, A2, A4 hold. If one of the following is satisfied:

The parameter $\rho$ belongs to $(0,1/2)$ and $\mathbb{E}|Z_{i,j}|^{4}<\infty$; 2. 2.

The parameter $\rho$ belongs to $[1/2,3/4)$ and $\mathbb{E}|Z_{i,j}|^{8}<\infty$,

then we have

[TABLE]

3 Proofs of the theorems on Toeplitz matrices

In Section 3.1, 3.2 and 3.3 we focus on Toeplitz matrices satisfying (1.1). In Section 3.4 we treat the Toeplitz matrices satisfying (1.2). And in Section 3.5 we prove Proposition 2.4 for Toeplitz matrices satisfying either (1.1) or (1.2).

3.1 Some preparation

Let $T_{n}$ satisfy (1.1). Note that by the definition of slowly varying function, $\gamma(h)$ is positive for $h$ sufficiently large.

For $p\in[1,\infty]$, and for $n$ sufficiently large such that $\gamma(n)\neq 0$, we define a finite-rank operator $\mathcal{K}_{n}^{(\gamma)}$ acting on $L^{p}(0,1)$ by

[TABLE]

The operator $\mathcal{K}^{(\rho)}$ in (2.1) is also well-defined for any $f\in L^{p}(0,1)$ by the integral formula:

[TABLE]

The operators $\mathcal{K}_{n}^{(\gamma)}$ and $\mathcal{K}^{(\rho)}$ acting on $L^{p}(0,1)$ are bounded, see [28, Lemma 5.4]. Moreover, from Lemma 5.4 of [28], we have the convergence

[TABLE]

The convergence (3.3) has many useful consequences in this proof. The first consequence is that the operator $\mathcal{K}^{(\rho)}$ is compact on $L^{p}(0,1)$ for any $p\in[1,\infty]$.

For each $p\in[1,\infty]$, $\mathcal{K}^{(\rho)}$ (resp. $\mathcal{K}_{n}^{(\gamma)}$) has its spectrum as an operator acting on $L^{p}(0,1)$. The following proposition shows that its non-zero eigenvalues and the associated eigenfunctions are invariant as $p$ changes.

Proposition 3.1**.**

The non-zero eigenvalues and the associated eigenfunctions of $\mathcal{K}^{(\rho)}:L^{p}(0,1)\rightarrow L^{p}(0,1)$ and $\mathcal{K}_{n}^{(\gamma)}:L^{p}(0,1)\rightarrow L^{p}(0,1)$ do not change when $p$ runs across $[1,\infty]$.

Proof.

We only prove the result for $\mathcal{K}^{(\rho)}$. The same argument applies to $\mathcal{K}_{n}^{(\gamma)}$.

We only need to prove that, for any $p\in[1,\infty)$, the operator $\mathcal{K}^{(\rho)}:L^{p}(0,1)\rightarrow L^{p}(0,1)$ has the same non-zero eigenvalues and associated eigenfunctions as $\mathcal{K}^{(\rho)}:L^{\infty}(0,1)\rightarrow L^{\infty}(0,1)$. As we have already noticed that $\mathcal{K}^{(\rho)}$ is compact on $L^{p}(0,1)$ and on $L^{\infty}(0,1)$, the desired result is a direct application of Theorem 4.2.15 in [13].

Indeed, we recall that two Banach spaces $B_{1}$ and $B_{2}$ or their associated norms are said to be compatible if $B=B_{1}\cap B_{2}$ is dense in each of them, and the following condition is satisfied: if $f_{n}\in B$, $\|f_{n}-f\|_{B_{1}}\to 0$ and $\|f_{n}-g\|_{B_{2}}\to 0$, then $f=g\in B$. The operators $A_{i}:B_{i}\rightarrow C_{i}$ with $i=1,2$ and $C_{1},C_{2}$ two Banach spaces, are said to be consistent if $A_{1}f=A_{2}f$ for all $f\in B_{1}\cap B_{2}$. Then we can verify that $L^{p}(0,1)$ and $L^{\infty}(0,1)$ are compatible, and $\mathcal{K}^{(\rho)}$ defined by an integral formula is obviously consistent. Then Theorem 4.2.15 in [13] applies. ∎

According to the above proposition, when we talk about the non-zero eigenvalues and the associated eigenfunctions of these operators, we do not need to specify the space.

3.2 Proof of Proposition 2.1

Let $\lambda\neq 0$ be an eigenvalue of $\mathcal{K}^{(\rho)}$ and $f$ be an associated eigenfunction. We now prove that $f$ is continuous on $[0,1]$.

Note that $f$ satisfies the equation

[TABLE]

and from Proposition 3.1, $f$ also belongs to $L^{\infty}(0,1)$. So for any $x_{0}\in[0,1]$, one has

[TABLE]

and the integral on the RHS tends to [math] when $|x-x_{0}|\to 0$.

We now prove that all non-zero eigenvalues of $\mathcal{K}^{(\rho)}$ are simple. We need the following key lemma. It says that any normalized eigenfunction of $\mathcal{K}^{(\rho)}$ associated with a non-zero eigenvalue, taken at $x=1$, has the absolute value $\sqrt{1-\rho}$.

Lemma 3.2**.**

Let $\lambda>0$ be a non-zero eigenvalue of $\mathcal{K}^{(\rho)}$, and let $f$ be a normalized eigenfunction associated with $\lambda$. Then $f$ satisfies

[TABLE]

A result similar to the above lemma first appeared in [32] for a general but square integrable kernel $k(x-y)$, see Theorem 3 of [32]. Note that thanks to the explicit formula of $\mathcal{K}^{(\rho)}$, the result of Lemma 3.2 is stronger than [32]. Directly using Theorem 3 of [32], we can only conclude that for $\rho\in(-1/2,0)$, for any non-zero eigenvalue $\lambda$ of $\mathcal{K}^{(\rho)}$, there exists a group of orthonormal eigenfunctions $f_{\lambda,1},\dots,f_{\lambda,m}$ associated with $\lambda$, where $m$ is the multiplicity of $\lambda$, such that

[TABLE]

However we will notice that this result is not sufficient to prove the simplicity of eigenvalues.

Whenever Lemma 3.2 is proved, we can prove the simplicity of any non-zero eigenvalue $\lambda$ of $\mathcal{K}^{(\rho)}$ by contradiction. Assume to the contrary that $\lambda>0$ had multiplicity $m\geq 2$, then we could choose two orthonormal eigenfunctions $f_{\lambda,1},f_{\lambda,2}$ associated with $\lambda$. From Lemma 3.4, without loss of generality we can assume that $f_{\lambda,1}(1)=f_{\lambda,2}(1)=\sqrt{1-\rho}$. Then the function

[TABLE]

is also a normalized eigenfunction of $\mathcal{K}^{(\rho)}$. But this function satisfies

[TABLE]

which is a contradiction to Lemma 3.2.

Thus it remains to prove Lemma 3.2.

Proof of Lemma 3.2.

We follow the outline of the proof in [32]. For any $\tau>0$, we define $\mathcal{K}^{(\rho)}_{\tau}$ the operator on $L^{2}(0,\tau)$ by

[TABLE]

By a change of variable, it is easy to see that a function $f\in L^{2}(0,1)$ is an eigenfunction of $\mathcal{K}^{(\rho)}$ associated with an eigenvalue $\lambda$ if and only if $f(\frac{\cdot}{\tau})$ is an eigenfunction of $\mathcal{K}^{(\rho)}_{\tau}$ associated with the eigenvalue $\lambda\tau^{1-\rho}$. By this fact, a positive number $\lambda$ is an eigenvalue of $\mathcal{K}^{(\rho)}$ with multiplicity $m$ if and only if $\lambda\tau^{1-\rho}$ is an eigenvalue of $\mathcal{K}^{(\rho)}_{\tau}$ with the same multiplicity $m$ for all $\tau>0$.

Suppose that $f$ is a normalized eigenfunction of $\mathcal{K}^{(\rho)}$ associated with non-zero eigenvalue $\lambda>0$. Then for any $\tau>1$ we have the following two equations

[TABLE]

and

[TABLE]

We define the function $g$ on $[0,\infty)$ by

[TABLE]

then $g$ is a continuous extension of $f$ on $[0,\infty)$. Multiply the two sides of (3.6) by $\overline{f(x)}$, and integrate for $x\in[0,1]$, we get

[TABLE]

Note that by the boundedness of $f$, Fubini Theorem applies to the RHS, thus changing the order of two integrations and taking into account the definition of $g$, we get

[TABLE]

It is easy to see from (3.9) that

[TABLE]

Letting $\tau\to 1^{+}$ on the two sides of (3.10), and noting that the continuity of $f$ on $[0,1]$ implies the uniform convergence of $f(\frac{x}{\tau})$ to $f(x)$, we get

[TABLE]

and the result follows. ∎

3.3 Proof of Proposition 2.2

Let $j\geq 1$ be an integer. By the proof of Theorem 2.3 in [28], we have $\lambda_{j}(\mathcal{K}^{(\rho)})>0$. Also by Proposition 2.1, we have $\lambda_{j-1}(\mathcal{K}^{(\rho)})>\lambda_{j}(\mathcal{K}^{(\rho)})>\lambda_{j+1}(\mathcal{K}^{(\rho)})$. Let $f_{j}$ be a normalized eigenfunction associated with $\lambda_{j}(\mathcal{K}^{(\rho)})$. In the sequel, we shall rely on the spectral projections (to be defined later) to construct an eigenvector of $T_{n}$ associated with $\lambda_{j}(T_{n})$ and prove that such an eigenvector approximates $f_{j}$ in the sense of (2.4).

Let $\varepsilon=\frac{1}{2}\min(\lambda_{j-1}(\mathcal{K}^{(\rho)})-\lambda_{j}(\mathcal{K}^{(\rho)}),\lambda_{j}(\mathcal{K}^{(\rho)})-\lambda_{j+1}(\mathcal{K}^{(\rho)}))$ and $\mathscr{C}$ be the circle centered at $\lambda_{j}(\mathcal{K}^{(\rho)})$ and of radius $\varepsilon$ on complex plane. We take $n$ sufficiently large such that $\|\mathcal{K}_{n}^{(\gamma)}-\mathcal{K}^{(\rho)}\|_{\infty}<\varepsilon$. So we have $|\lambda_{k}(\mathcal{K}_{n}^{(\gamma)})-\lambda_{k}(\mathcal{K}^{(\rho)})|<\varepsilon$ for all $1\leq k\leq n$, which implies that only the eigenvalues $\lambda_{j}(\mathcal{K}_{n}^{(\gamma)})$ and $\lambda_{j}(\mathcal{K}^{(\rho)})$ are enclosed by $\mathscr{C}$ and all the other eigenvalues are outside $\mathscr{C}$. We define the spectral projections

[TABLE]

By Riesz decomposition Theorem (c.f. for example [13, Theorem 1.5.4 and Theorem 4.3.19]), $P_{n}$ (resp. $P$) is a projection onto the eigenspace of $\mathcal{K}_{n}^{(\gamma)}$ (resp. $\mathcal{K}^{(\rho)}$)) corresponding to $\lambda_{j}(\mathcal{K}_{n}^{(\gamma)})$ (resp. $\lambda_{j}(\mathcal{K}^{(\rho)})$). To those who are unfamiliar with Riesz’ Theorem, we explain the arguments with $\mathcal{K}^{(\rho)}$ and $P$. Indeed, from Riesz’ Theorem, $P$ is a finite rank projection which commutes with $\mathcal{K}_{\rho}$. Let $\mathcal{R}(P)$ be the range of $P$, then $\mathcal{R}(P)$ is an invariant space of $\mathcal{K}^{(\rho)}$ (due to the commutativity of the projection $P$ and $\mathcal{K}^{(\rho)}$), and the restriction of $\mathcal{K}^{(\rho)}$ to $\mathcal{R}(P)$ is self-adjoint (because $\mathcal{K}^{(\rho)}$ is self-adjoint) and has spectrum $\{\lambda_{j}(\mathcal{K}^{(\rho)})\}$, then from the finite dimensional linear algebra, $\mathcal{R}(P)$ is spanned by the eigenfunctions of $\mathcal{K}^{(\rho)}$ associated with $\lambda_{j}(\mathcal{K}^{(\rho)})$. Therefore, recall that $f_{j}$ is a normalized eigenfunction of $\mathcal{K}^{(\rho)}$ associated with $\lambda_{j}(\mathcal{K}^{(\rho)})$, we have $Pf_{j}=f_{j}$. The same argument shows that $P_{n}$ is a projection to the eigenspace of $\mathcal{K}_{n}^{(\gamma)}$ and thus $P_{n}f_{j}$ is an eigenfunction of $\mathcal{K}_{n}^{(\gamma)}$ associated with $\lambda_{j}(\mathcal{K}_{n}^{(\gamma)})$, in condition that $P_{n}f_{j}\neq 0$.

We prove that $\|P_{n}-P\|_{\infty}\to 0$. Indeed we have

[TABLE]

Thus the main task is to uniformly control $\|(z-\mathcal{K}_{n}^{(\gamma)})^{-1}-(z-\mathcal{K}^{(\rho)})^{-1}\|_{\infty}$ in term of $\|\mathcal{K}_{n}^{(\gamma)}-\mathcal{K}^{(\rho)}\|_{\infty}$ for $z\in\mathscr{C}$. As $(z-\mathcal{K}^{(\rho)})^{-1}$ is analytic outside of $\operatorname*{Spec}(\mathcal{K}^{(\rho)})$, there exists $K>0$ such that $\sup_{z\in\mathscr{C}}\|(z-\mathcal{K}^{(\rho)})^{-1}\|_{\infty}\leq K$. Let $n$ be sufficiently large such that $\|\mathcal{K}_{n}^{(\gamma)}-\mathcal{K}^{(\rho)}\|_{\infty}<1/(2K)$. Then we have

[TABLE]

Thus as $n\to\infty$ we have

[TABLE]

From this convergence we have

[TABLE]

Then from (3.13) we obtain

[TABLE]

Combining (3.13) and (3.14) we conclude

[TABLE]

Notice that the range of $\mathcal{K}^{(\gamma)}_{n}$ consists of step functions

[TABLE]

so the eigenfunctions of $\mathcal{K}^{(\gamma)}_{n}$ must also have this form. Notice also that a $n$-dimensional normalized vector $v=(v_{k})_{k=1}^{n}$ is an eigenvector of $T_{n}$ associated with $\lambda_{j}(T_{n})$ if and only if the normalized function

[TABLE]

is an eigenfunction of $\mathcal{K}_{n}^{(\gamma)}$ associated with $\lambda_{j}(\mathcal{K}_{n}^{(\gamma)})=\lambda_{j}(T_{n})/(n\gamma(n))$. Since $Pf_{j}/\|Pf_{j}\|_{2}$ is a normalized eigenfunction of $\mathcal{K}_{n}^{(\gamma)}$, by the relation (3.16), up to a change of sign, we have

[TABLE]

From (3.15) we get the desired result (2.4).

3.4 Proof of Theorem 2.3

Let $T_{n}=(\gamma(i-j))$ be a $n\times n$ Toeplitz matrix with spectral density $\varphi$ satisfying (1.2). Let $T^{\prime}_{n}=(\gamma^{\prime}(i-j))$ be a $n\times n$ Toeplitz matrix with spectral density $x\mapsto 1/|x|^{1-\rho},x\in[-\pi,\pi]$. Let $D_{n}(x):=\frac{\sin((n+1/2)x)}{\sin(x/2)}$ be the Dirichlet kernel. From the theory of Toeplitz matrices, we have

[TABLE]

By the inequality (2.2.1) of [31], for a certain $0<\delta<\min(\rho/2,(1-\rho)/2)$, there exists $0<\eta<1$ such that

[TABLE]

Then because $L_{2}$ is locally bounded, we have

[TABLE]

The same argument also gives

[TABLE]

Combining the last two inequalities, and using the triangle inequality, we get

[TABLE]

Thus in order to prove that (3.18) tends to [math], we only need to prove that

[TABLE]

By changing variables we write

[TABLE]

Let

[TABLE]

From the properties of Dirichlet kernels, there exists a constant $K>0$ such that for any $-n\pi\leq x\leq n\pi$ and $-n\eta\leq y\leq n\eta$,

[TABLE]

For any $\varepsilon>0$, let $A>1$ be a large enough positive number to be determined afterwards. Then

[TABLE]

Because $\Delta$ is bounded by $1$, and also by (3.19), we have

[TABLE]

Using (3.19) and Young’s convolution inequality, let $\frac{1}{p}=\frac{1-\rho-\delta}{2},\frac{1}{q}=\frac{1+\rho+\delta}{2}$, we have

[TABLE]

Let $A=\max(\varepsilon^{-p},\varepsilon^{-\frac{1}{\rho-\delta}})$, then there exists $K>0$ such that

[TABLE]

By the uniform convergence theorem for slowly varying functions (see for example (2.2.4) of [31]), for large enough $n$, we have

[TABLE]

Then using Young’s inequality again, we have

[TABLE]

Therefore, there exists $K>0$ such that, for any $\varepsilon>0$, for large enough $n$,

[TABLE]

and the proof of (2.6) is complete.

The convergence (2.7) is an immediate consequence of (2.6), (2.2) and (2.5). To prove (2.8), using the spectral projections and repeat the same procedure as in the proof of Proposition 2.2 with $L^{2}\to L^{2}$ norm, the result then follows.

3.5 Proof of Proposition 2.4

First we prove Item 1. Let $T_{n}$ satisfies (1.1) or (1.2) and assume $\rho\in(0,1/2)$. From (2.2) or (2.7), there exists a constant $K>0$ such that

[TABLE]

with $i=1,2$. Since $n^{1/2-\rho}L_{i}(n)\to\infty$, we have

[TABLE]

Then we prove Item 2. Let $T_{n}$ satisfy (1.1) with $\rho\in[1/2,3/4)$. Note that

[TABLE]

Also from (2.2), we have $\lambda_{1}^{2}(T_{n})\sim\lambda_{1}^{2}(\mathcal{K}^{(\rho)})n^{2+2\rho}L_{1}^{2}(n)\gg n^{1/2+\epsilon}$ for some $\epsilon>0$, where for two sequences of positive numbers $(x_{n})_{n}$ and $(y_{n})_{n}$, the notation $x_{n}\gg y_{n}$ means that $y_{n}/x_{n}\to 0$. We then have

[TABLE]

Since $|\gamma(k)|^{2}\sim L_{1}^{2}(k)k^{2\rho}\leq L_{1}^{2}(k)k^{-1}$, it follows that

[TABLE]

Hence

[TABLE]

Now let $T_{n}$ satisfy (1.2) with $\rho\in[1/2,3/4)$. From Theorem 2.3, there exists $K>0$ such that $\|T_{n}\|\sim Kn^{1-\rho}L_{2}(n)$. From the formula

[TABLE]

we have

[TABLE]

where $F_{n}(x)=\frac{\sin^{2}(\frac{nx}{2})}{n\sin^{2}(\frac{x}{2})}$ is the Féjer kernel. Let $1<r<\frac{2}{4\rho-1}$, and $p$ be such that $2/p+1/r=2$. Then we have $1<p<2$. From the form of $\varphi$, we have $\varphi\in L^{p}(-\pi,\pi)$. By Young’s inequality,

[TABLE]

Note that

[TABLE]

Then

[TABLE]

4 Proofs of Proposition 2.5 and 2.9

4.1 Proof of Proposition 2.5

Let $j\geq 1$ be a fixed integer. We first prove the convergence of $\lambda_{j}(S_{N}(\Gamma))$ in probability. Suppose that $\Gamma_{n}=U\operatorname*{diag}(t_{1},\dots,t_{n})U^{*}$ where $U=(u_{1},\dots,u_{n})$ is a unitary matrix whose columns are $u_{1},\dots,u_{n}$. Recall that A3 holds. For any $\varepsilon>0$ sufficiently small, let $\ell>j$ be the smallest integer such that $a_{\ell}<\varepsilon/(2K)$, where $K=\sup\|C_{N}\|(1+\sqrt{r})^{2}(1+\varepsilon)$. let $n$ be large enough such that $|t_{k}-a_{k}|<\varepsilon/(2K)$ for $k=1,\dots,\ell+1$.

Let $\tilde{\Gamma}_{n}=U\operatorname*{diag}(t_{1},\dots,t_{\ell},0,\dots,0)U^{*}$. Then we have

[TABLE]

As $\|\Gamma_{n}-\tilde{\Gamma}_{n}\|=t_{\ell+1}<\varepsilon/K$, from [41] we know that for large $N,n$, with high probability,

[TABLE]

Thus from the stability of spectrum of Hermitian matrices, with high probability, we have

[TABLE]

So we only need to prove that

[TABLE]

The matrix $S_{N}(\tilde{\Gamma})$ has the same non-zero eigenvalues with the $\ell\times\ell$ matrix

[TABLE]

Then because $A_{\ell}$ is a fixed-dimensional matrix, it suffices to prove that each of its entries converges in probability. Note that

[TABLE]

where $C_{p,p}$ are the diagonal entries of $C_{N}$. Because $\|C_{N}\|$ is uniformly bounded, we have

[TABLE]

Thus we have

[TABLE]

Combine with the equality

[TABLE]

we obtain the convergence in probability.

Assume that $Z_{i,j}$ are standard real or complex Gaussian and prove the almost sure convergence. We argue similarly as the proof of Proposition 4.1 of [28]. Precisely we will prove that for any $\epsilon>0$,

[TABLE]

where $K$ is a constant. Indeed using [10, Theorem 5.6] we only need to prove that the function

[TABLE]

is $\frac{1}{\sqrt{KN}}$-Lipschitz with respect to the Frobenius norm $\|Z\|_{F}$. Let $Z,\hat{Z}$ be two $N\times n$ matrices, and let

[TABLE]

Then by Wielandt-Hoffmann inequality for singular values, we have

[TABLE]

Thus we have

[TABLE]

This proves the Lipschitz property and the concentration inequality (4.1) holds. Then by Borel-Cantelli’s lemma, we have

[TABLE]

Together with the convergence in probability

[TABLE]

the almost sure convergence in the Gaussian case follows.

We now assume that the bound condition A6 holds and prove the following lemma which will be useful in Section 5. As a byproduct, this lemma implies the almost sure convergence of $\lambda_{j}(S_{N}(\Gamma))$.

Lemma 4.1**.**

Under A1, A2, A3, A4, A5 and A6, for any $j\geq 1$ and any $\epsilon>0$, with overwhelming probability,

[TABLE]

For the definition of "overwhelming probability" or "tiny probability", refer to Definition 1.

Proof.

We can repeat the first part of the proof of Proposition 2.5 and we just need to verify that each "high probability" can be replaced by "overwhelming probability" under the assumptions of this lemma. Let $\tilde{\Gamma}_{n},A_{\ell}$ be defined as above. From Theorem 3.1 of [40], we know that with overwhelming probability,

[TABLE]

Then we only need to prove that under the assumptions of this lemma, for any $\epsilon>0$, with overwhelming probability,

[TABLE]

As $C_{N},\Gamma_{n}$ are diagonal, the above inequality is actually

[TABLE]

Let

[TABLE]

Note that $\operatorname*{Var}(\overline{Z_{1,i}}Z_{1,k})=1$ if $i\neq k$, and $\operatorname*{Var}(\overline{Z_{1,i}}Z_{1,k})=\mathbb{E}|Z_{1,1}|^{4}-1$ if $i=k$. We assume that $\operatorname*{Var}(\overline{Z_{1,i}}Z_{1,k})\neq 0$, because otherwise we have $i=k$ and $|Z_{p,i}|^{2}=1$ almost surely, then (4.2) holds almost surely, and there is nothing to prove.

Using Bennett’s inequality (8b) of [9], for any $t>0$, one has

[TABLE]

As $\sigma^{2}/N\to\operatorname*{Var}(\overline{Z_{1,i}}Z_{1,k})\int x^{2}\,\mathrm{d}\nu_{C}(x)\neq 0$, let $t$ be such that $t\sigma=N\epsilon$, then we have

[TABLE]

where $a,b,c,d$ are some positive constants. Because $\varepsilon_{n}$ is an almost sure upper bound of $|Z_{i,k}|/\sqrt{n}$, we can assume that $\sqrt{n}\varepsilon_{n}^{2}\to\infty$. Then for any fixed $M>0$, and for large enough $N,n$,

[TABLE]

Then the result follows. ∎

5 Proof of Theorem 2.7

In this section and Section 6, in order to simplify the notation, we omit the subscription $N$ and $n$ of $C_{N}$ and $\Gamma_{n}$, so they are just denoted as $C$ and $\Gamma$. We also simplify the notation $S_{N}(\Gamma)$ as $S$, and denote the eigenvalues of $S$ by

[TABLE]

We prove the CLT for largest eigenvalues of $S$ in the following steps. First we truncate, recenter and rescale the entries of $Z$ so that $|Z_{i,j}|\leq\varepsilon_{n}\sqrt{n}$ where $\varepsilon_{n}$ is a sequence of positive numbers tending to [math]. The truncation step is identical to the approach used in the proof of Theorem 1.1 of [3], from where we know that this does not affect the result. So from now on we assume that A6 holds.

Then in order to prove the weak convergence of $\Lambda_{m}(\Gamma)$, it suffices to prove that for any fixed vector $(b_{1},\dots,b_{m})^{\top}\in\mathbb{R}^{m}$, we have

[TABLE]

where the random vector $(\mathbf{G}_{1},\dots,\mathbf{G}_{m})^{\top}$ follows the limiting distribution $\mathcal{N}(0,(\mathbb{E}|Z_{1,1}|^{4}-1)\int x^{2}\,\mathrm{d}\nu_{C}(x)I_{m})$. For each $1\leq j\leq m$, we prove that the inequality

[TABLE]

is equivalent to

[TABLE]

for some random variable $\mathcal{Y}_{j}$ which is expressed by the entries of $Z$. Then we determine the limiting distribution of $(\mathcal{Y}_{1},\dots,\mathcal{Y}_{m})^{\top}$. Then the result follows by using Slutsky’s Theorem.

Reformulation of the eigenvalue inequality (5.2).

We begin to rewrite the inequality (5.2). For further use we temporarily do not suppose that $C$ and $\Gamma$ are diagonal. So this part is shared with Section 6. We suppose that $\Gamma=UDU^{*}$ where $D=\operatorname*{diag}(t_{1},\dots,t_{n})$ and $U=(u_{1},\dots,u_{n})$ is unitary. By normalizing $C,\Gamma$ and $S$, we assume without loss of generality that $\int x\,\mathrm{d}\nu_{C}(x)=1$ and $a_{j}=t_{j}=1$. We set

[TABLE]

Then $\tilde{\lambda}_{j}$ satisfies the equation

[TABLE]

Under A8, applying Proposition 2.5 to both $S$ and $S_{(j)}$, for a small enough $\epsilon>0$, with high probability, for $i=1,\dots,j-1$ we have

[TABLE]

and since the $j$th largest eigenvalue of $\Gamma_{(j)}$ is $\lambda_{j+1}(\Gamma_{n})$, which tends to $a_{j+1}$, we have

[TABLE]

We denote the above evenement by $\Omega$. Suppose that $\Omega$ happens. Then $\tilde{\lambda}_{j}$ is not the eigenvalue of $S_{(j)}$, and the equation (5.3) is equivalent to

[TABLE]

Note that the matrix $N^{-1}C^{1/2}Zu_{j}u_{j}^{*}Z^{*}C^{1/2}(\tilde{\lambda}_{j}I-S_{(j)})^{-1}$ is of rank one, so the equation is in fact

[TABLE]

Moreover, note that $\Upsilon(\lambda)>0$ for $\lambda$ large enough. Note also that

[TABLE]

for $\lambda\in\mathbb{R}$, and with $\Omega$ holds, the denominator and the numerator of $\Upsilon(\lambda)$ change sign $(j-1)$ times respectively on $[1+\epsilon,\infty)$. So we deduce that

[TABLE]

and $\Upsilon(\lambda)$ changes sign in $[1-\epsilon,1+\epsilon]$ exactly at $\tilde{\lambda}_{j}$. Thus for large enough $N,n$ such that

[TABLE]

the inequality

[TABLE]

is equivalent to

[TABLE]

If it happens that $\Gamma_{(j)}=0$, then $S_{(j)}=0$ and $\theta_{j}=1$, and the inequality (5.5) is in fact

[TABLE]

Let the LHS of the above inequality be $\mathcal{Y}_{j}$, then the procedure of rewriting (5.2) is complete.

We now assume that $\Gamma_{(j)}\neq 0$, then we have $\mu^{\Gamma_{(j)}}\neq\delta_{0}$. We recall some results from [12]. By Proposition 1.1 of [12], for any $z\in\mathbb{C}_{+}$, the system of equations

[TABLE]

has a unique solution $(g^{0}_{\Gamma_{(j)}}(z),g^{0}_{C}(z))$ such that $\Im g^{0}_{\Gamma_{(j)}}(z)<0,\Im g^{0}_{C}(z)<0$. Define

[TABLE]

By plugging the first equation of (5.7) into the second one, and replacing the discrete integrals by sums, we can see that $g^{0}_{C}(z)$ is the unique solution of the equation

[TABLE]

such that $\Im g_{C}(z)<0$. Let

[TABLE]

Then $s_{C}^{0}$ (resp. $s_{\Gamma}^{0}$) is the Cauchy-Stieltjes transform of the probability measure $\mu^{0}_{C}$ (resp. $\mu^{0}_{\Gamma_{(j)}}$), which is the asymptotic equivalent of $\mu^{S_{(j)}}$ (resp. $\mu^{\frac{1}{N}\Gamma_{(j)}^{1/2}Z^{*}CZ\Gamma_{(j)}^{1/2}}$). See also (1.6)-(1.10) of [1]. Moreover by Lemma 3.3 of [12], for any $x\in\mathbb{R}\backslash\{0\}$, the limit $\lim_{z\in\mathbb{C}_{+}\to x}g^{0}_{C}(z)$ exists. Let $\gamma,\tilde{\gamma}$ be defined as

[TABLE]

[TABLE]

Then a main result of [12] says that a non-zero real number $x$ is outside the support of $\mu^{0}_{C},\mu^{0}_{\Gamma_{(j)}}$, if and only if

[TABLE]

Now we relate the function $g^{0}_{C}$ with $\theta_{j}$. By the definition of $\theta_{j}$, we have

[TABLE]

Because of the assumptions A2, A3, also because the distance between $1$ and $\operatorname*{Spec}(\Gamma_{(j)})$ is bounded away from [math], there is a complex neighborhood $B(1,\epsilon)$ of $1$ such that

[TABLE]

uniformly for $g\in B(1,\epsilon)$ and $i=1,\dots,N$. So there is a neighborhood $\mathcal{U}=B((1,1),\epsilon)\subset\mathbb{C}^{2}$ of $(1,1)$ ($\epsilon$ may take different values from one place to another) such that for $N,n$ large enough, the function $F$ is holomorphic in $\mathcal{U}$. Some calculation shows that

[TABLE]

which is also holomorphic in $\mathcal{U}$ for $N,n$ large enough. Moreover we have

[TABLE]

uniformly for $(g,z)\in\mathcal{U}$ as $N,n\to\infty$. So for $N,n$ large enough and for $(g,z)\in\mathcal{U}$, we have

[TABLE]

From Remark 2.3, we have $\theta_{j}\to 1$. Whenever $(1,\theta_{j})\in\mathcal{U}$, by holomorphic implicit function theorem [16, Ch. 1, Th 7.6], there exists a holomorphic function $g:z\mapsto g(z)$, defined in a complex neighborhood $B(\theta_{j},\epsilon)$ of $\theta_{j}$, such that

[TABLE]

Some calculations similar to those between (6)-(8) of [12] gives

[TABLE]

for $z\in\mathbb{C}_{+}$. Then by the unicity of solution of the function $F$ for $z\in\mathbb{C}_{+}$, we have $g(z)=g^{0}_{C}(z)$ for $z\in\mathbb{C}_{+}\cup B(\theta_{j},\epsilon)$. This proves that $g^{0}_{C}(\theta_{j})=1$.

Moreover from (5.9) and Proposition 3.2 of [12] we deduce that for $N,n$ large enough, the point $\theta_{j}$ is in an interval $[1-\epsilon,1+\epsilon]$ who lies outside the support of $\mu^{0}_{C},\mu^{0}_{\Gamma_{(j)}}$ for $N,n$ large enough. For any $z\in\mathbb{C}\backslash\operatorname*{supp}\mu^{0}_{C}$, we have

[TABLE]

Note that $g^{0}_{C}(\theta_{j})=1$, we rewrite the inequality (5.5) as

[TABLE]

By Lemma 3.4 of [12], $g^{0}_{C}(x)\in\mathbb{R}$ for any $x\in\mathbb{R}\backslash\operatorname*{supp}\mu^{0}_{C}$. Then we can apply Lagrange’s Mean Value Theorem and get

[TABLE]

where $\xi$ is a number between $\theta_{j}$ and $\eta_{j}$. As $\theta_{j}$ and $\eta_{j}$ both tend to $1$, we also have $\xi\to 1$. Then from the formula (5.10), as $N,n\to\infty$,

[TABLE]

Plugging (5.12), (5.13) into (5.11), and multiplying the two sides by $\sqrt{N}\eta_{j}$, also note that $\eta_{j}\to 1$, the inequality (5.11) can be written as

[TABLE]

Using the formula $A^{-1}-B^{-1}=A^{-1}(B-A)B^{-1}$, and letting

[TABLE]

we can rewrite (5.14) as

[TABLE]

Let

[TABLE]

and

[TABLE]

In the following, we prove the CLT for $(\mathcal{Y}_{1},\dots,\mathcal{Y}_{m})^{\top}$, and prove that $R_{n}=o_{P}(1)$, under the diagonal condition A4, A5 in this section, and under A9 or A10 in Section 6, respectively.

CLT for $(\mathcal{Y}_{1},\dots,\mathcal{Y}_{m})^{\top}$ and estimation of the remainder $R_{n}$.

We now assume that $C,\Gamma$ are diagonal. Then we have

[TABLE]

and by the CLT for independent random variables, we have

[TABLE]

Now we prove that $R_{n}=o_{P}(1)$. Let $\mathbf{z}_{j}$ denote the $j$th column of $Z$. As $C,\Gamma$ are diagonal, we have

[TABLE]

where $Q$ is defined in (5.15). Note that $\mathbf{z}_{j}$ and $Q$ are independent, then by Lemma B.1 in [1], we have

[TABLE]

By Proposition 2.2, for any $\varepsilon>0$, with high probability, there is only a finite number of eigenvalues of $S_{(j)}$ larger than $\varepsilon$, and the distance between $\eta_{j}$ and the spectrum of $S_{(j)}$ is bounded away from [math]. So one has

[TABLE]

where $\|\cdot\|_{F}$ denotes the Frobenius norm, and recall that $\|AB\|_{F}\leq\|A\|\|B\|_{F}$ for any matrices $A,B$. This proves that $\mathbb{E}_{Q}|P_{1}|^{2}=o_{P}(1)$. We also conclude that $P_{1}=o_{P}(1)$, because for any $\varepsilon>0$,

[TABLE]

and by Dominated Convergence Theorem,

[TABLE]

Then we prove that $P_{2}=o_{P}(1)$. We set

[TABLE]

Let $0<\epsilon<\min(1-a_{j+1},a_{j-1}-1)/3$. We note that for $N,n$ large enough, $g^{0}_{C}(z)$ is analytic in the complex disc $B(\eta_{j},\epsilon)$ of $\eta_{j}$, and with high probability, $g_{C}(z)$ is also analytic in this disc. If we can prove that for $w:=\eta_{j}+\mathbf{i}/n$,

[TABLE]

Then, because

[TABLE]

with high probability, and note that $\eta_{j}\to 1$, the result follows. The proof of (5.17) uses some techniques of [1, Section 3], and is postponed to Appendix A.

6 Proof of Theorem 2.10

In this section we extend our result to some non-diagonal $\Gamma$’s. We prove the CLT (2.15) under the condition A9 or A10.

We can prove that in any subsequence of the sequence

[TABLE]

there is still a subsequence converging to [math]. Note that the entries of $\Sigma_{m}^{(N)}$ are bounded, we can assume that they converge, i.e.

[TABLE]

Then we are led to proving that

[TABLE]

Let $U=(u_{1},\dots,u_{n})$ and $Y=ZU$. Let $\mathbf{y}_{k}=Zu_{k}$ be the $k$th column of $Y$. From the last section, the proof of the theorem can be done by proving that

[TABLE]

where

[TABLE]

and that for any $j=1,\dots,m$,

[TABLE]

where $Q$ is defined in (5.15).

We use Cramér-Wold device to prove the CLT of the $m$-dimensional vector $(\mathcal{Y}_{1},\dots,\mathcal{Y}_{m})^{\top}$. By a direct calculation, the covariance matrix of $(\mathcal{Y}_{1},\dots,\mathcal{Y}_{m})^{\top}$ is exactly $\Sigma_{m}^{(N)}$ which tends to $\Sigma_{m}$ as we have assumed. Then for any fixed vector $\alpha:=(\alpha_{1},\dots,\alpha_{m})^{\top}$, we prove that

[TABLE]

If $\alpha^{\top}\Sigma_{m}\alpha=0$, it means that $\operatorname*{Var}(\sum_{j=1}^{m}\alpha_{j}\mathcal{Y}_{j})=\alpha^{\top}\Sigma_{m}^{(N)}\alpha\to 0$. Then $\sum_{j=1}^{m}\alpha_{j}\mathcal{Y}_{j}=o_{P}(1)$ and hence (6.1) holds. Now we assume that $\alpha^{\top}\Sigma_{m}\alpha\neq 0$ and prove that

[TABLE]

Note that the rows of $Y$ are i.i.d., then

[TABLE]

is a weighted sum of $N$ i.i.d. random variables. We can use Lindeberg’s CLT to prove (6.2). To do so, we need to verify the Lindeberg condition

[TABLE]

as $N,n\to\infty$ for any $\varepsilon>0$. Since the quantities in the expectations are identically distributed for different $i$, and since

[TABLE]

we only need to prove

[TABLE]

Since $\mathbb{E}|Y_{1,j}|^{2}=1$ and since $\operatorname*{Var}|Y_{1,j}|^{2}$ is uniformly bounded, for any $\varepsilon>0$, the events

[TABLE]

occur with low probability. By Minkowski’s inequality, we have

[TABLE]

Since $m$ is a fixed number, we only need to prove that for each $j=1,\dots,m$,

[TABLE]

Since $\mathbb{P}(E_{N})\to 0$ and from the uniform boundedness of $\mathbb{E}|Y_{1,j}|^{4}$ we have $\mathbb{E}|Y_{1,j}|^{2}\mathbbm{1}_{E_{N}}\to 0$, then (6.3) is equivalent to

[TABLE]

This is a corollary of the following lemma.

Lemma 6.1**.**

Let $v=(v_{1},\dots,v_{n})\in\mathbb{C}^{n}$ such that $\|v\|\leq 1$. Let $(Z_{k})_{k\geq 1}$ be a sequence of i.i.d. random variables satisfying $\mathbb{E}Z_{k}=0$, $\mathbb{E}|Z_{k}|^{2}=1$, $\mathbb{E}|Z_{k}|^{4}<\infty$. Let $E_{n}$ be a sequence of events such that $\mathbb{P}(E_{n})\to 0$, then

[TABLE]

Proof.

We only prove the case where $v$ and $Z_{k}$ are real. For the complex case, it can be easily proved from the real case by separating real and complex parts, and then using Minkowski’s inequality.

As all the random variables $|Z_{k}|^{4}$ are identically distributed and integrable, they are uniformly integrable. Thus we have

[TABLE]

Let $(e_{n})$ be a sequence of positive numbers tending to [math] such that

[TABLE]

Note that from $\|v\|\leq 1$ we have

[TABLE]

We write

[TABLE]

By Minkowski’s inequality, we have

[TABLE]

For the first part, using again Minkowski’s inequality and noting that $|v_{k}|\leq 1$, we get

[TABLE]

For the second part, it suffices to prove that from any subsequence of $\left(\mathbb{E}|P_{2}|^{4}\mathbbm{1}_{E_{n}}\right)_{n}$ we can extract a subsequence tending to [math]. From any subsequence of $(\sum_{k=1}^{N}|v_{k}|^{2}\mathbbm{1}_{|v_{k}|<e_{n}})_{n}$, there exists a convergent subsequence. So we can assume that

[TABLE]

Then if we can prove that

[TABLE]

the proof of the lemma will be complete due to the inequality

[TABLE]

To prove (6.4), by the equality $x^{4}=\min(x^{4},M^{4})+x^{4}\mathbbm{1}_{|x|>M}-M^{4}\mathbbm{1}_{|x|>M}$ for any $x\in\mathbb{R}$ and $M>0$, we have

[TABLE]

Let $N\to\infty$, since $\max_{k}\{|v_{k}|\mathbbm{1}_{|v_{k}|\leq e_{n}}\}\leq e_{n}\to 0$, by Lindeberg’s Theorem we have $P_{2}\xrightarrow{\mathcal{D}}\mathcal{N}(0,\sigma^{2})$. Let $\mathbf{G}\sim\mathcal{N}(0,\sigma^{2})$, then we have

[TABLE]

where the first convergence is from direct calculation, the second and the third are from the fact that $P_{2}\xrightarrow{\mathcal{D}}G$ and that the function $x\mapsto\min(x^{4},M^{4})$ is continuous and bounded. So we have

[TABLE]

Finally we take $M\to\infty$ and see that (6.4) holds. ∎

Next we prove that

[TABLE]

under the conditions A4 and A9 or A10, where $Q$ is defined in (5.15).

We assume that A4 and A9 hold. From the equation satisfied by $g^{0}_{C}$, we have

[TABLE]

Now it suffices to prove that

[TABLE]

Note that with high probability, we have

[TABLE]

for some $K>0$. Then the matrices

[TABLE]

are both positive semi-definite. Then we have

[TABLE]

To prove that this is $o_{P}(1)$, it suffices to prove that

[TABLE]

Denote the entries of $\Gamma_{(j)}$ by $\Gamma_{i,k}$. By a simple calculation, one has

[TABLE]

Note that

[TABLE]

so the second term of the above sum is zero. Also, if we use $\overline{u}_{j}$ to denote the conjugate of the vector $u_{j}$, we have

[TABLE]

Therefore by A2 and A9, the limit (6.5) is proved.

We now assume that A10 and A4 hold. Using the formula $A^{-1}-B^{-1}=A^{-1}(B-A)B^{-1}$, and the inequality

[TABLE]

we can write

[TABLE]

The calculation (6.6) also shows that

[TABLE]

To prove $P^{\prime}_{1}=o_{P}(1)$, it suffices to prove that

[TABLE]

Recall that $Y=Z(u_{1},\dots,u_{n})$ and $\mathbf{y}_{k}=Zu_{k}$. Note that the rows of $Y$ are independent, and that $\mathbf{y}_{1},\dots,\mathbf{y}_{n}$ are decorrelated. Let $D_{(j)}=\operatorname*{diag}(t_{1},\dots,t_{j-1},0,t_{j+1},\dots,t_{n})$. Then by some calculation, we have

[TABLE]

Using Lemma 2.1 in [2] and Holder’s inequality, $\mathbb{E}|Y_{1,j}|^{4}|Y_{1,k_{1}}|^{2}|Y_{1,k_{2}}|^{2}$ is uniformly bounded. Also we have $\frac{1}{N^{2}}\sum_{k_{1},k_{2}\neq j}t_{k_{1}}t_{k_{2}}=\left(\frac{1}{N}\sum_{k\neq j}t_{k}\right)^{2}\to 0$, so the first term of the above sum is negligible. We only need to prove that

[TABLE]

We now prove the first convergence. By simple algebra, we have

[TABLE]

By the elementary inequality $|a+b|^{2}\leq 2|a|^{2}+2|b|^{2}$, we only need to prove

[TABLE]

To prove the first convergence in (6.10), we have

[TABLE]

To prove the second convergence in (6.10), we take an arbitrary small $\varepsilon>0$. Then by the assumption A3, the number of $t_{k}$ larger than $\varepsilon$ is finite. So we have

[TABLE]

Thus we have proved (6.10), implying the first convergence of (6.9).

We then prove the second part of (6.9). We have

[TABLE]

where $\delta_{k_{1},k_{2}}$ is the Kronecker symbol. Using the inequality $|a+b+c|^{2}\leq 3(|a|^{2}+|b|^{2}+|c|^{2})$ we only need to prove

[TABLE]

and

[TABLE]

The first of (6.12) can be proved similarly as (6.11), the second of (6.12) is a consequnce of A10. To prove (6.13), we have

[TABLE]

Then $P^{\prime}_{1}=o_{P}(1)$ is proved.

To prove that $P^{\prime}_{2}=o_{P}(1)$, using the same argument leading to (6.5), we only need to prove that

[TABLE]

Using the same notations as proving (6.8), by simple algebra, we have

[TABLE]

Then we have $P^{\prime}_{2}=o_{P}(1)$.

Remark 6.1*.*

As we have seen, the main difficulty of this proof is the convergence to [math] in probability of a quadratic form $\mathbf{y}^{*}Q\mathbf{y}$ where $\mathbf{y}$ and $Q$ are not independent. In the expansion e.g. (6.7) or other expansions afterwards, we can see that the orthogonal relation between $u_{j}$ and $\Gamma_{(j)}$ is crucial for the result. Up to the date of submission of this article and to our best knowledge, we have not found any method other than moment expansions which can achieve the same or stronger result. The method used in [11, Theorem 2.4] could be a possible option. But instead of the orthogonality between $u_{j}$ and $\Gamma_{(j)}$, the proof in [11] relies on the clear separation between spiked and non-spiked population eigenvalues and the quicker speed of the non-spiked eigenvalues converging to [math], which is not satisfied by our model.

Potentially our method works also in the case of non-diagonal $C_{N}$, and more general $\Gamma_{n}$, especially for the Toeplitz matrices $T_{n}$ with parameter $\rho\in(-1,-3/4]$. But restricted to the complexity of proof and the length of the paper we may proceed in this direction in the forthcoming works.

Appendix A Proof of (5.17)

We write

[TABLE]

To prove that $P_{21}=o_{P}(1)$, we use the martingale decomposition. By reassigning $t_{j}=0$, we continue to denote the eigenvalues of $\Gamma_{(j)}$ by $t_{1},\dots,t_{n}$. Let $\mathbf{y}_{k}$ be the $k$th column of $C^{1/2}Z$. Let $\mathbb{E}_{k}$ denote $\mathbb{E}_{\mathbf{y}_{1},\dots,\mathbf{y}_{k}}$ and $\mathbb{E}_{0}=\mathbb{E}$. We denote

[TABLE]

[TABLE]

and

[TABLE]

[TABLE]

[TABLE]

Lemma A.1**.**

Under the conditions of Theorem 2.7 with $Z_{i,j}$ satisfying A6, for any $p\geq 1$, there exists a constant $K_{p}$ such that for large enough $N,n$, we have

[TABLE]

[TABLE]

Proof.

The first two inequalities are due to the fact that with overwhelming probability the distance from $w$ to the spectra of $S$ and $S_{k}$ are bounded away from zero uniformly in $n,k$ (Lemma 4.1), so $\|D^{-1}(w)\|$ and $\|D_{k}^{-1}(w)\|$ are uniformly bounded with overwhelming probability; and with tiny probability, we use the general bound

[TABLE]

Therefore the expectations $\mathbb{E}\|D^{-1}(w)\|^{p}$, $\mathbb{E}\|D_{k}^{-1}(w)\|^{p}$ are uniformly bounded.

For the third one, the proof is identical to the proof of Lemma A.3 in [1], up to some adaptions. The first steps of the proof of Lemma A.3 in [1] give

[TABLE]

Then we have

[TABLE]

To prove that $\mathbb{E}|\frac{1}{N}t_{k}\mathbf{y}_{k}^{*}\mathbf{y}_{k}|^{p}$ is uniformly bounded for any fixed $p\geq 1$, with overwhelming probability, we can use the bound

[TABLE]

and with tiny probability we use the general bound

[TABLE]

Then we have

[TABLE]

For the fourth one, we write

[TABLE]

Note that for large enough $N,n$, $(Nw)^{-1}t_{k}\operatorname*{tr}C$ has a positive distance to $1$. We now prove that with overwhelming probability, the term $(Nw)^{-1}t_{k}\operatorname*{tr}D_{k}^{-1}(w)S_{(k)}C$ is small enough so that the denominator of $\tilde{\beta}_{k}(w)$ is bounded away from [math].

Let $\lambda_{1},\dots,\lambda_{N}$ be the eigenvalues of $S_{(k)}$, with eigenvectors $v_{1},\dots,v_{N}$ where $v_{i}=(v_{1,i},\dots,v_{N,i})^{\top}$. Recall that $C=\operatorname*{diag}(c_{1},\dots,c_{N})$. Then

[TABLE]

Then by Lemma 4.1, with overwhelming probability, the above quantity is smaller than any fixed $\varepsilon>0$.

With tiny probability, the above estimation does not hold. For the estimation of expectation, we should find a new estimation of $\tilde{\beta}_{k}(w)$. Note that if $|N^{-1}t_{k}\operatorname*{tr}D_{k}^{-1}(w)C|\leq 1/2$, then $|\tilde{\beta}_{k}(w)|\leq 2$; else if $|N^{-1}t_{k}\operatorname*{tr}D_{k}^{-1}(w)C|>1/2$ we have

[TABLE]

where we estimate $\max_{i}|\lambda_{i}|^{2}$ as in the proof of Lemma A.1 in [1]. Thus finally we have

[TABLE]

By the same arguments, one can also verify the boundedness of $\phi_{k}(w)$ and $\psi_{k}(w)$. ∎

Then

[TABLE]

Then we have

[TABLE]

By Lemma A.1, the expectation $\mathbb{E}|\beta_{k}(w)|^{4}$ is uniformly bounded; by Lemma B.1 in [1], we have, for any $p\geq 1,q\geq 1$,

[TABLE]

So we have

[TABLE]

For $J_{2}$, note that

[TABLE]

then we have

[TABLE]

Using the same arguments, we get

[TABLE]

Therefore in fact $P_{21}=o_{P}(1/\sqrt{N})$.

To prove that $P_{22}=o_{P}(1)$, we consider two cases:

$\operatorname*{tr}\Gamma/\sqrt{n}\to 0$; 2. 2.

$\operatorname*{tr}\Gamma/\sqrt{n}\to a\neq 0$, where $a$ is a positive number or $\infty$.

By extracting subsequences we can assume that one of these two cases holds. Indeed, we want to prove that for any $\varepsilon>0$, the following limit holds:

[TABLE]

We can prove that from any subsequence of

[TABLE]

there exists a subsequence converging to [math]. We know that from any subsequence of $(\Gamma_{n})_{n}$, one can extract a subsequence $(\Gamma_{n_{k}})_{k\geq 1}$ satisfying one of the above two cases.

Suppose that Condition 1 holds. Then we have

[TABLE]

One can see that

[TABLE]

is bounded, and

[TABLE]

thus $P_{222}=o(1)$. For $P_{221}$, by (A.1) and the arguments afterwards, we have, for an arbitrary $p>1$,

[TABLE]

where $\lambda_{k}$ are the eigenvalues of $S$, and we have used the equality

[TABLE]

Now we suppose that Condition 2 holds. We define

[TABLE]

Then

[TABLE]

where $d_{n}:=\mathbb{E}g_{C}(w)-\sum_{i=1}^{N}\frac{1}{w-\frac{1}{N}\sum_{k=1}^{n}t_{k}\psi_{k}(w)c_{i}}$. On the other hand, following the calculation for (3.41) of [1], we have

[TABLE]

By the definition of $s_{C}$ and the system of equations (5.7), we have

[TABLE]

Taking the difference of the last two equalities, we obtain

[TABLE]

Combining the equations (A.4) and (A.5), we obtain

[TABLE]

Next we prove that $d_{n}=\mathbb{E}g_{C}(w)-\sum_{i=1}^{N}\frac{1}{w-\frac{1}{N}\sum_{k=1}^{n}t_{k}\psi_{k}(w)c_{i}}=o(1)$, $\sum_{k=1}^{n}(\mathbb{E}\beta_{k}(w)-\psi_{k}(w))=o(1)$, and that the multiplier

[TABLE]

is bounded away from [math].

Repeating the calculations in Section 3.3 of [1], one can check that $d_{n}=o(1)$ and $\sum_{k=1}^{n}(\mathbb{E}\beta_{k}(w)-\psi_{k}(w))=o(1)$. The proof is similar so we omit the details. We just point out some adaptions due to the differences between the models. We define

[TABLE]

Because $\|\frac{1}{N}\sum_{k=1}^{n}t_{k}\psi_{k}(w)C\|\leq\frac{1}{N}\sum_{k=1}^{n}t_{k}|\psi_{k}(w)|\|C\|\to 0$, the matrix $W(w)$ is invertible for $N,n$ large enough, and $\|W^{-1}(w)\|$ is uniformly bounded. According to (3.33) of [1] and the estimations afterwards, the result can be similarly deduced. We also remind that in Section 3.3 of [1] the proof is made for Gaussian entries. Here we only assume that the entries $Z_{i,j}$ have finite sixth moment, so according to (A.2), we have for example the following estimation of $\mathcal{H}_{2}$ which is correspondingly defined in the equation next to (3.36) of [1]:

[TABLE]

thus by the formula next to (3.36) and the formula (3.38) of [1], we have

[TABLE]

To prove that the multiplier (A.6) is bounded from below, we recall that $w\to 1$, $g^{0}_{C}(w)\to 1$, $\mathbb{E}g_{C}(w)\to 1$, $\frac{1}{N}\sum_{\ell=1}^{n}t_{\ell}\psi_{\ell}(w)=o(1)$, and $\frac{1}{N}\sum_{\ell=1}^{n}\frac{t_{\ell}}{1-g^{0}_{C}(w)t_{\ell}}=o(1)$, thus for any $k=1,\dots,n$, as $N,n\to\infty$,

[TABLE]

[TABLE]

The above two limits are uniform in $k$, so we have, for any $\varepsilon>0$, for $N,n$ large enough,

[TABLE]

Therefore

[TABLE]

which is lower bounded because we are just in the case where $\frac{1}{\sqrt{n}}\sum_{k=1}^{n}t_{k}\to a\neq 0$.

The proof of (5.17) is complete.