Spiked multiplicative random matrices and principal components

TL;DR

This paper analyzes the spectral properties of spiked multiplicative random matrices with Haar randomness, providing precise limits for outlier eigenvalues and eigenvectors, and demonstrating eigenvalue sticking and eigenvector delocalization.

Contribution

It offers new results on eigenvalue limits and eigenvector behavior in spiked Haar-invariant models, extending previous work and covering near the BBP transition and degenerate spikes.

Findings

Outlier eigenvalues have established limits with optimal convergence.

Eigenvectors associated with outliers are characterized with generalized components.

Non-outlier eigenvalues stick to those of unspiked matrices, and eigenvectors are delocalized.

Abstract

In this paper, we study the eigenvalues and eigenvectors of the spiked invariant multiplicative models when the randomness is from Haar matrices. We establish the limits of the outlier eigenvalues $\widehat{\lambda}_i$ and the generalized components ($\langle \mathbf{v}, \widehat{\mathbf{u}}_i \rangle$ for any deterministic vector $\mathbb{v}$) of the outlier eigenvectors $\widehat{\mathbf{u}}_i$ with optimal convergence rates. Moreover, we prove that the non-outlier eigenvalues stick with those of the unspiked matrices and the non-outlier eigenvectors are delocalized. The results also hold near the so-called BBP transition and for degenerate spikes. On one hand, our results can be regarded as a refinement of the counterparts of [12] under additional regularity conditions. On the other hand, they can be viewed as an analog of [34] by replacing the random matrix with i.i.d. entries with Haar random matrix.

Equations

$$\hat{Y}={\hat{A}}^{1/2}X.$$

$$\hat{Y}={\hat{A}}^{1/2}U{\hat{B}}^{1/2}.$$

$${\mu }_W=\frac{1}{N}\sum _{i=1}^N{\delta }_{{\lambda }_i(W)}.$$

$$m_{\mu }(z):=\int \frac{1}{x-z}\mathrm{d}\mu (x),\text{for }z\in \mathbb{C}\setminus {\mathbb{R}}_{+}.$$

$$M_{\mu }(z)$$

$$A\equiv A_N=\mathrm{diag}(a_1,\cdots ,a_N)\text{and}B\equiv B_N=\mathrm{diag}(b_1,\cdots ,b_N).$$

$$C^{-1}\le \frac{{\rho }_{\alpha }(x)}{(x-E_{-}^{\alpha }{)}^{t_{-}^{\alpha }}(E_{+}^{\alpha }-x{)}^{t_{+}^{\alpha }}}\le C,\forall x\in [E_{-}^{\alpha },E_{+}^{\alpha }],$$

$$C^{-1}\le \frac{{\rho }_{\beta }(x)}{(x-E_{-}^{\beta }{)}^{t_{-}^{\beta }}(E_{+}^{\beta }-x{)}^{t_{+}^{\beta }}}\le C,\forall x\in [E_{-}^{\beta },E_{+}^{\beta }].$$

$$\mathbf{d}:=\mathcal{L}({\mu }_{\alpha },{\mu }_A)+\mathcal{L}({\mu }_{\beta },{\mu }_B)\le N^{-1+ϵ},$$

$$\mathrm{supp}{\mu }_A\subset [E_{-}^{\alpha }-\delta ,E_{+}^{\alpha }+\delta ]\text{and}\mathrm{supp}{\mu }_B\subset [E_{-}^{\beta }-\delta ,E_{+}^{\beta }+\delta ].$$

$$\arg {\Omega }_{\alpha }(z)\ge \arg z,\text{and}\arg {\Omega }_{\beta }(z)\ge \arg z.$$

$$\underset{x\searrow -\infty }{\lim }{\Omega }_{\alpha }(x)=\underset{x\searrow -\infty }{\lim }{\Omega }_{\beta }(x)=-\infty .$$

$$z{M}_{{\mu }_{\alpha }}({\Omega }_{\beta }(z))=z{M}_{{\mu }_{\beta }}({\Omega }_{\alpha }(z))={\Omega }_{\alpha }(z){\Omega }_{\beta }(z).$$

$$M(z):=M_{{\mu }_{\alpha }}({\Omega }_{\beta }(z))=M_{{\mu }_{\beta }}({\Omega }_{\alpha }(z)).$$

$$\rho (x)\sim \sqrt{(E_{+}-x)(x-E_{-})},x\in [E_{-},E_{+}].$$

$${\Omega }_{\alpha (\beta )}(z)={\Omega }_{\alpha (\beta )}(E_{+})+C_{\alpha (\beta )}\sqrt{z-E_{+}}+\mathrm{O}(|z-E_{+}{|}^{3/2}),z\in U.$$

$$\mathrm{dist}({\Omega }_{\alpha }(\mathbb{C}\setminus {\mathbb{R}}_{+}),\mathrm{supp}{\mu }_{\beta })>0,\mathrm{dist}({\Omega }_{\beta }(\mathbb{C}\setminus {\mathbb{R}}_{+}),\mathrm{supp}{\mu }_{\alpha })>0.$$

$$\underset{x\to \infty ,x\in \mathbb{R}}{\lim }{\Omega }_{\alpha }(x)=\infty =\underset{x\to \infty ,x\in \mathbb{R}}{\lim }{\Omega }_{\beta }(x).$$

$${\hat{E}}_{+}:=\sup \mathrm{supp}{\mu }_A⊠{\mu }_B.$$

$$E_{+}-N^{-2/3+ϵ}\le {\hat{E}}_{+}\le E_{+}+N^{-1+ϵ}.$$

$${\hat{a}}_k=\{\begin{array}{ll}{a}_k(1+d_k^a),& 1\le k\le r\\ a_k,& k\ge r+1\end{array},{\hat{b}}_k=\{\begin{array}{ll}{b}_k(1+d_k^b),& 1\le k\le s\\ b_k,& k\ge s+1\end{array}.$$

$${\hat{a}}_i>{\Omega }_{\beta }(E_{+}),\text{or}{\hat{b}}_j>{\Omega }_{\alpha }(E_{+}).$$

$$r^{+}:=\max \{1\le i\le r:{\hat{a}}_i\ge {\Omega }_{\beta }(E_{+})+N^{-1/3}\},s^{+}:=\max \{1\le j\le s:{\hat{b}}_j\ge {\Omega }_{\alpha }(E_{+})+N^{-1/3}\},$$

$${\mathcal{Q}}_1\equiv \tilde{H}:=A^{1/2}UB{U}^{\ast }{A}^{1/2},$$

$${\hat{\mathcal{Q}}}_1:=\hat{Y}{\hat{Y}}^{\ast }\equiv {\hat{A}}^{1/2}U\hat{B}{U}^{\ast }{\hat{A}}^{1/2},{\hat{\mathcal{Q}}}_2:={\hat{Y}}^{\ast }\hat{Y}\equiv {\hat{B}}^{1/2}{U}^{\ast }\hat{A}U{\hat{B}}^{1/2}.$$

$${\Omega }_B^c(\hat{z}):=\frac{\hat{z}\mathrm{tr}A\tilde{G}(\hat{z})}{1+\hat{z}\mathrm{tr}\tilde{G}(\hat{z})},\text{where}\hat{z}={\lambda }_1({\mathcal{Q}}_1)+\mathrm{i}{N}^{-2/3+ϵ}\text{and}\tilde{G}(z):=(\tilde{H}-z{)}^{-1}.$$

$$\mathbb{P}\left(|X_N|\ge N^{ϵ}|Y_N|\right)\le N^{-D},$$

$$\mathcal{O}:=\{{\pi }_a(i):1\le i\le r\}\cup \{{\pi }_b(j):1\le j\le s\},$$

$${\mathcal{O}}^{+}:=\{{\pi }_a(i):1\le i\le r^{+}\}\cup \{{\pi }_b(j):1\le j\le s^{+}\},$$

$$\left|{\hat{\lambda }}_{{\pi }_a(i)}-{\Omega }_B^{-1}({\hat{a}}_i)\right|\prec N^{-1/2}({\hat{a}}_i-{\Omega }_{\beta }(E_{+}){)}^{1/2},1\le i\le r^{+},$$

$$\left|{\hat{\lambda }}_{{\pi }_b(j)}-{\Omega }_A^{-1}({\hat{b}}_j)\right|\prec N^{-1/2}({\hat{b}}_j-{\Omega }_{\alpha }(E_{+}){)}^{1/2},1\le j\le s^{+}.$$

Full text

1 Introduction

Finite rank deformed random matrices have found applications in many scientific endeavors. In these contexts, the low-rank part is usually regarded as the signal whereas the random matrix part can be viewed as the high dimensional noise. From an application viewpoint, researchers are interested in understanding the signal part from the noisy matrix especially from the first few largest eigenvalues and eigenvectors, which are closely related to the principal component analysis (PCA) [42].

In the literature, a popular and sophisticated model is the spiked covariance matrix model proposed by Johnstone [41]. In such a model, a finite number of spikes (eigenvalues detached from the bulk of the spectrum) are added to the spectrum of the population covariance matrix. Specifically, consider that

[TABLE]

Here $\widehat{A}$ is the spiked covaraince matrix constructed by adding a finite rank perturbation to some non-spiked positive definite matrix $A,$ and $X=(x_{ij})$ is the main random source where $x_{ij}$’s are i.i.d. centered random variables. An extension is the spiked separable covariance matrix [34], where the data matrix is $Y=\widehat{A}^{1/2}X\widehat{B}^{1/2},$ with another spiked matrix $\widehat{B}$. In spatiotemporal data analysis, $\widehat{A}$ and $\widehat{B}$ are respectively the spatial and temporal covariance matrices [52].

While the assumption that $X$ has i.i.d. entries has been useful in many instances, other types of random matrices also appear naturally in certain applications. An important example is the Haar distributed random matrices which have been used in statistical learning theory, for instance, see [36, 37, 46, 48, 58]. In the current paper, we aim to study the spiked random matrices where the main randomness is Haar random matrices. Especially, we consider that $X=U$ is either an $N\times N$ random Haar unitary or orthogonal matrix, so that

[TABLE]

We point out that the data matrix (1.1) has also appeared in the study of high dimensional data analysis, for instance, see [21, 22, 33].

1.1 Some related results on finite rank deformation of random matrices

In this section, we first pause to give a brief review of the literature on the spectra of fixed-rank deformation of random matrices, a category of random matrix models including signal-plus-noise and spiked covariance matrices as typical examples. There exists rich literature in understanding the limiting behavior of the eigenvalues and eigenvectors of such deformed models. Since the seminal work of Baik, Ben Arous, and Péché [3], it is now well-understood that the extreme eigenvalues undergo the so-called BBP transition as the magnitude of the deformation changes. Roughly speaking, the extreme eigenvalues of the deformed matrix detaches from spectrum of the undeformed random matrix if and only if the strength of the deformation exceeds a certain threshold. In this case, we call the extreme eigenvalue as an outlier, and the associated eigenvector as an outlier eigenvector. In parallel to the outlier eigenvalues, an outlier eigenvector is concentrated on a cone with the axis parallel to the true eigenvectors (of the deformation) and the aperture explicitly determined by the deformations. Moreover, the remaining eigenvalues are close to those of the undeformed random matrices and the associated eigenvectors are delocalized.

The results in the same spirit of the aforementioned arguments have been established for various deformed random matrix models under different settings when the random matrix part contains i.i.d. entries. On one hand, when the deformation is additive, the eigenvalues and eigenvectors have been studied for deformed Wigner matrices in [9, 14, 26, 27, 43, 44], for signal-plus-noise matrices in [6, 15, 23, 24, 30] and for deformed non-Hermtian matrices in [10, 16, 20, 29, 54]. On the other hand, when the deformation is multiplicative, the eigenvalues and eigenvectors have been investigated for spiked covariance matrices in [1, 2, 3, 4, 5, 14, 18, 19, 31, 51], for spiked separable covariance matrices in [34], for spiked CCA matrices in [8, 49], for spiked MANOVA matrices in [39, 57] and for spiked correlation matrices in [50].

When the randomness comes from Haar invariant random matrices, there are relatively fewer related works [12, 14, 25]. All the existing works focus on finding the limits of the outlier eigenvalues and eigenvectors under stronger assumptions that the spikes are far away from the critical values by a distance of constant order. Consequently, they leave the convergent rates and the non-outlier eigenvalues and eigenvectors undiscussed. The aim of this paper is to fill this gap by establishing the first order limits and precise rates of convergence for the outlier eigenvalues and eigenvectors and concentration bounds for the non-outlier eigenvalues and eigenvectors for the model (1.1).

1.2 An overview of our results

In this subsection, we provide a rough overview of our results. Our theoretical findings are in the same spirit as those discussed in Section 1.1. We pause to introduce some notations. It is well-known that for the unspiked model $A^{1/2}UBU^{*}A^{1/2},$ its empirical spectral distribution (ESD) is given by the free multiplicative convolution of the ESDs $\mu_{A}$ and $\mu_{B}$ of $A$ and $B$ respectively, denoted as $\mu_{A}\boxtimes\mu_{B}$ [56]; see Definition 2.4 below for a precise definition. More recently, in our previous works [32, 40], we investigated the behavior of $\mu_{A}\boxtimes\mu_{B}$ by analyzing a pair of analytic functions, known as subordination functions, $\Omega_{A}$ and $\Omega_{B}$ that define the free convolution; see (2.6) for details.

We briefly describe our results, firstly on eigenvalues of $\widehat{Y}\widehat{Y}^{*}$. Due to invariance, we only need to consider diagonal $\widehat{A}$ and $\widehat{B}$ whose entries are denoted in the decreasing order as $\{\widehat{a}_{i}\}$ and $\{\widehat{b}_{j}\}.$ We further assume that $\widehat{A}$ and $\widehat{B}$ contain spikes $\{\widehat{a}_{i}:1\leq i\leq r\}$ and $\{\widehat{b}_{j}:1\leq j\leq s\}$ with finite $r$ and $s$, respectively. A spike $\widehat{a}_{i}$ (or $\widehat{b}_{j}$) gives rise to an outlier of $\widehat{Y}\widehat{Y}^{*}$ if and only if $\widehat{a}_{i}>\Omega_{\beta}(E_{+})$ (or $\widehat{b}_{j}>\Omega_{\alpha}(E_{+})$), where $E_{+}$ is the rightmost edge of the support of $\mu_{\alpha}\boxtimes\mu_{\beta}.$ In this case, the outlier concentrates around a fixed location, namely $\Omega_{B}^{-1}(\widehat{a}_{i})$ (or $\Omega_{A}^{-1}(\widehat{b}_{j})$), where $\Omega_{A(B)}^{-1}(\cdot)$ are the inverse functions of the subordination functions; see Theorem 3.11 for more details. Our result also shows that this transition occurs on the scale $N^{-1/3}$, as in a typical BBP transition [3]. More precisely, if $\widehat{a}_{i}-\Omega_{\beta}(E_{+})\gg N^{-1/3}$ or $\widehat{b}_{j}-\Omega_{\alpha}(E_{+})\gg N^{-1/3}$, that is, if the spike is supercritical, then the outlier will be well-separated from the support of $\mu_{A}\boxtimes\mu_{B}$ and can be detected readily. For $0<\widehat{a}_{i}-\Omega_{B}(E_{+})\ll N^{-1/3}$ or $0<\widehat{b}_{j}-\Omega_{\alpha}(E_{+})\ll N^{-1/3}$, that is, when the spike is subcritical, the corresponding ”outlier” cannot be distinguished from the non-deformed spectrum and will instead stick to the right-most edge $E_{+}$ up to some random fluctuation of order $\mathrm{O}(N^{-2/3})$. The rest of the non-outlier eigenvalues will stick to the eigenvalues of $A^{1/2}UBU^{*}A^{1/2}$; see Theorem 3.14 for more details. We also remark that the convergence rates in Theorems 3.11 and 3.14 are consistent with [3], indicating that the second-order transition therein might also be true for our model. That is, under some mild assumptions, the outlier eigenvalues are asymptotically Gaussian whereas the extremal non-outlier eigenvalues follow Tracy-Widom distribution. We will pursue this direction in future works.

Next for the sample eigenvector of $\widehat{Y}\widehat{Y}^{*}$ associated with the outlier caused by a supercritical spike $\widehat{a}_{i}$, we show that it is concentrated on a cone with axis parallel to the true eigenvector with an explicit aperture determined by $\widehat{a}_{i}$ and $\Omega_{B}$; see Theorem 3.15 for more details. On the other hand, the sample eigenvector of $\widehat{Y}\widehat{Y}^{*}$ that is associated with a subcritical spike $\widehat{b}_{j}$ is delocalized. Moreover, the non-outlier eigenvectors are also delocalized; see Theorem 3.17 for more details. Similar results hold for the eigenvectors of $\widehat{Y}^{*}\widehat{Y}$ by switching the role of $\widehat{A}$ and $\widehat{B}.$ We remark that in [12], the authors have studied the convergent limits of the outlying eigenvalues and eigenvectors on a macroscopic scale; see Remarks 3.12 and 3.16 for comparison. Our results extend the counterparts of [12] to full strength, that is, with the optimal scale and rate.

Our proof adapts the same proof strategy as in [18, 34, 43]. The key input is to establish the local laws both near and far away from the edge of the undeformed random matrices. That is to say, we need precise concentration estimates for the resolvent of $A^{1/2}UBU^{*}A^{1/2},$ denoted as $\widetilde{G}(z)=(A^{1/2}UBU^{*}A^{1/2}-z)^{-1},\ z=E+\mathrm{i}\eta,\eta\gg N^{-1}.$ Especially, we need the entry-wise local law, i.e., concentration estimates for $\widetilde{G}_{ij}(z),1\leq i,j\leq N,$ and the averaged local law, i.e., concentration estimate for $N^{-1}\sum_{i=1}^{N}\widetilde{G}_{ii}(z).$ In fact, all these concentration estimates depend on the subordination functions and the Stieltjes transform of $\mu_{A}\boxtimes\mu_{B};$ see Theorem 4.19 for more details. Once we have the local laws, we can reduce our tasks to studying the undeformed models using some perturbative approach; see Section 4.3 for more details on the proof strategies. We remark that we can also generalize the results for the invariant additive models of [12] in a similar fashion by using and modifying the local laws for the additive model $A+UBU^{*}$ as in [7].

This paper is organized as follows. In Section 2, we formally introduce our model and some necessary assumptions. In Section 3, we provide the main results of the paper. In Section 4, we collect and prove some results which will be used for our proofs. We also provide a description of our proof strategy. The technical proofs are provided in Sections 5 and 6.

Conventions. Throughout the rest of the paper, $N$ always denotes the size of our matrix model and we often omit the dependence on $N$. For $m,n\in\mathbb{N}$, we denote the set $\{k\in\mathbb{N}:m\leq k\leq n\}$ by $\llbracket m,n\rrbracket$. For $i\in\llbracket 1,N\rrbracket$, we denote by $\mathbf{e}_{i}$ the $(N\times 1)$ column vector with $(\mathbf{e}_{i})_{j}=\delta_{ij}$. We use $I$ for the identity matrix of any dimension without causing any confusion, and we abbreviate $\operatorname{tr}=N^{-1}\operatorname{Tr}$ for matrices with any dimension. For a matrix $A,$ we denote its operator norm by $\left\lVert A\right\rVert.$ Throughout the paper, we use $\mathbf{1}$ for the indicator function.

Finally, we use the standard big-$\mathrm{O}$ and little-$\mathrm{o}$ notations; for $N$-dependent nonnegative numbers $a_{N}$ and $b_{N}$, we write $a_{N}=\mathrm{O}(b_{N})$ or $a_{N}\lesssim b_{N}$ if $a_{N}/b_{N}$ remains bounded, $a_{N}\sim b_{N}$ if $a_{N}/b_{N}$ and $b_{N}/a_{N}$ are both bounded, and $a_{N}=\mathrm{o}(b_{N})$ (or equivalently $a_{N}\ll b_{N}$) if $a_{N}/b_{N}$ converges to zero, with the convention $\cdot/0\mathrel{\mathop{:}}=0$.

2 Definition of spiked multiplicative model

In this section, we introduce the model and some necessary assumptions.

2.1 Some notations and assumptions

We first introduce some notations and technical assumptions. Throughout the paper, we will consistently use the following notations. For an $N\times N$ matrix $W$, its empirical spectral distribution (ESD) is denoted as

[TABLE]

The following transforms will be used frequently.

Definition 2.1**.**

For a probability measure $\mu$ defined on $\mathbb{R}_{+},$ its Stieltjes transform $m_{\mu}$ is defined as

[TABLE]

Moreover, we define the $\mathtt{M}$-transform $M_{\mu}$ and $\mathtt{L}$-transform $L_{\mu}$ on $\mathbb{C}\setminus\mathbb{R}_{+}$ as

[TABLE]

We first introduce the non-spiked model. Consider two $N\times N$ real, deterministic, positive definite matrices

[TABLE]

We next introduce our assumptions on the undeformed model.

Assumption 2.2**.**

Throughout the paper, we assume that there exist two $N$-independent absolutely continuous probability measures $\mu_{\alpha}$ and $\mu_{\beta}$ on $(0,\infty)$ with densities respectively $\rho_{\alpha}$ and $\rho_{\beta}$ satisfying the following.

• (i).

For simplicity, we assume that both of them have means $1$, that is, $\int x\mathrm{d}\mu_{\alpha}(x)=\int x\rho_{\alpha}(x)\mathrm{d}x=1.$

• (ii).

Both $\rho_{\alpha}$ and $\rho_{\beta}$ have single non-empty intervals as supports, denoted as $[E_{-}^{\alpha},E_{+}^{\alpha}]$ and $[E_{-}^{\beta},E_{+}^{\beta}],$ respectively. Here $E_{-}^{\alpha},E_{+}^{\beta},E_{-}^{\alpha}$ and $E_{+}^{\beta}$ are all positive numbers. Moreover, both of $\rho_{\alpha}$ and $\rho_{\beta}$ are strictly positive in the interior of their supports.

• (iii).

There exist constants $-1<t^{\alpha}_{\pm},t^{\beta}_{\pm}<1$ and $C>1$ such that

[TABLE]

Moreover, for the ESDs of $A$ and $B,$ denoted as $\mu_{A}$ and $\mu_{B},$ we assume the following:

• (iv).

For the Levy distance $\mathcal{L}(\cdot,\cdot),$ we have that for any small constant $\epsilon>0,$ when $N$ is sufficiently large

[TABLE]

• (v).

For the supports of $\mu_{A}$ and $\mu_{B}$, we have that for any constant $\delta>0,$ when $N$ is sufficiently large

[TABLE]

The first assumption is introduced for technical simplicity and it can be removed easily via scaling; see Remark 3.2 of [40] for details. The single-cut assumption in (ii) guarantees that there are only two edges so that $m_{\mu_{H}}(z)$ is always monotone outside the spectrum; this is a technicality and one can still prove the same result around the uppermost edge in a multi-cut setting. Moreover, assumption (iii) is introduced to guarantee the square root behavior near the edges of the free multiplicative convolution of $\mu_{\alpha}$ and $\mu_{\beta}.$ When this condition fails, a different behavior of $\mu_{\alpha}\boxtimes\mu_{\beta}$ from our current discussion may arise; for example, see [45, 47]. Assumption (iv) ensures that $\mu_{A}$ and $\mu_{B}$ converge respectively to $\mu_{\alpha}$ and $\mu_{\beta}$ sufficiently fast down to a local scale. Finally, we may localize all of Assumption 2.2 to the upper edge and weaken (2.2) as far as eigenvalues and eigenvectors around the upper edge are concerned; see Remark 4.23 for details.

The central results of Voiculescu in [55, 56] implies that $\mu_{H}$ converges weakly to a deterministic measure under Assumption 2.2, denoted as $\mu_{\alpha}\boxtimes\mu_{\beta}$. It is called the free multiplicative convolution of $\mu_{\alpha}$ and $\mu_{\beta}$. In the present paper, we use the $\mathsf{M}$-transform in (2.1) and associated subordination functions to define the free multiplicative convolution.

Lemma 2.3** (Proposition 2.5 of [40]).**

*For Borel probability measures $\mu_{\alpha}$ and $\mu_{\beta}$ on $\mathbb{R}_{+}$, there exist unique analytic functions $\Omega_{\alpha},\Omega_{\beta}:\mathbb{C}\setminus\mathbb{R}_{+}\to\mathbb{C}\setminus\mathbb{R}_{+}$ satisfying the following:

(1). For all $z\in\mathbb{C}_{+}$, we have $\Omega_{\alpha}(z),\Omega_{\beta}(z)\in\mathbb{C}_{+}$, $\Omega_{\alpha}(\overline{z})=\overline{\Omega_{\alpha}(z)}$, $\Omega_{\beta}(\overline{z})=\overline{\Omega_{\beta}(z)}$,*

[TABLE]

(2).We have

[TABLE]

(3). For all $z\in\mathbb{C}\setminus\mathbb{R}_{+}$, we have

[TABLE]

We remark that Lemma 2.3 is originally due to [11], with a slightly different notation using $\eta$-transform defined by $\eta(z)=1/M(1/z)$. Later in [28], exactly the same result as Lemma 2.3 was proved with different notations. The aforementioned papers also considered analytic subordination functions for free additive and multiplicative convolutions on $\mathbb{R}$ and the circle, respectively.

Definition 2.4**.**

Denote the analytic function $M:\mathbb{C}\backslash\mathbb{R}_{+}\rightarrow\mathbb{C}\backslash\mathbb{R}_{+}$ by

[TABLE]

The free multiplicative convolution of $\mu_{\alpha}$ and $\mu_{\beta}$ is defined as the unique probability measure $\mu,$ denoted as $\mu\equiv\mu_{\alpha}\boxtimes\mu_{\beta}$ such that (2.6) holds for all $z\in\mathbb{C}\setminus\mathbb{R}_{+}$. In this sense, $M(z)\equiv M_{\mu_{\alpha}\boxtimes\mu_{\beta}}(z)$ is the $\mathsf{M}$-transform of $\mu_{\alpha}\boxtimes\mu_{\beta}.$ Furthermore, the analytic functions $\Omega_{\alpha}$ and $\Omega_{\beta}$ are referred to as the subordination functions. Similarly we define $\Omega_{A}$ and $\Omega_{B}$ by replacing $(\alpha,\beta)$ with $(A,B)$ in Lemma 2.3, and define $\mu_{A}\boxtimes\mu_{B}$ so that $M_{\mu_{A}}(\Omega_{B}(z))=M_{\mu_{B}}(\Omega_{A}(z))=M_{\mu_{A}\boxtimes\mu_{B}}(z)$ for all $z\in\mathbb{C}\setminus\mathbb{R}_{+}$.

We conclude this subsection with preliminary facts on the free convolutions $\mu_{\alpha}\boxtimes\mu_{\beta},\mu_{A}\boxtimes\mu_{B}$ and their associated subordination functions, that will be required in the precise statements of our main results. In actual proofs, we also use more detailed description from Lemma 4.18 below.

Lemma 2.5**.**

Suppose that $\mu_{\alpha},\mu_{\beta},\mu_{A},$ and $\mu_{B}$ satisfy Assumption 2.2.

• (i).

The restrictions $\Omega_{\alpha}|_{\mathbb{C}_{+}},\Omega_{\beta}|_{\mathbb{C}_{+}},\Omega_{A}|_{\mathbb{C}_{+}},\Omega_{B}|_{\mathbb{C}_{+}}$ extend continuously to $[0,\infty)$, mapping into the Riemann sphere $\mathbb{C}\cup\{\infty\}$; we consistently use the same notations $\Omega_{\alpha},\Omega_{\beta}$, etc., to denote these extensions. Furthermore, $\Omega_{\alpha}$ and $\Omega_{\beta}$ are bounded on each compact subset of $\mathbb{C}_{+}\cup\mathbb{R}$.

• (ii).

The free convolution $\mu_{\alpha}\boxtimes\mu_{\beta}$ has a continuous and bounded density $\rho$. The density $\rho$ is supported on a compact interval $[E_{-},E_{+}]$ in $(0,\infty)$ and satisfies

[TABLE]

Here $E_{+}:=\sup\operatorname{supp}\mu_{\alpha}\boxtimes\mu_{\beta}.$

• (iii).

There exist positive constants $C_{\alpha}$ and $C_{\beta}$ such that, for each fixed neighborhood $U$ of $E_{+}$,

[TABLE]

• (iv).

We have

[TABLE]

• (v).

The maps $\Omega_{\alpha},\Omega_{\beta}$ are real analytic and strictly increasing on $(E_{+},\infty)$, and satisfy

[TABLE]

The same results hold true for $\Omega_{A},\Omega_{B}$ if we replace $E_{+}$ by

[TABLE]

• (vi).

For any fixed $\epsilon>0$, we have

[TABLE]

Proof.

The first statement is due to Proposition 4.3 and Lemma 5.2 of [40]. The second to fourth statements are proved in Theorem 3.3 and Propositions 5.10 and 5.6 of [40], in that order. The fifth and sixth statements are immediate consequences of Lemmas A.8 and A.7 of [32]. ∎

2.2 The model

With the above preparation, we introduce the spiked model following the setup of [31, 34]. Throughout the paper, we consider the spikes associated with the upper edge $E_{+}.$ Similar discussion applies to the lower edge $E_{-}.$ To add a few spikes, we assume that there exist some fixed integers $r$ and $s$ with two sequences of positive numbers $\{d_{i}^{a}\}_{i\leq r}$ and $\{d_{j}^{b}\}_{j\leq s}$ such that $\widehat{A}=\operatorname{diag}\{\widehat{a}_{1},\cdots,\widehat{a}_{N}\}$ and $\widehat{B}=\operatorname{diag}\{\widehat{b}_{1},\cdots,\widehat{b}_{N}\},$ where

[TABLE]

Without loss of generality, we assume that $\widehat{a}_{1}\geq\widehat{a}_{2}\cdots\geq\widehat{a}_{N}$ and $\widehat{b}_{1}\geq\widehat{b}_{2}\cdots\geq\widehat{b}_{N}.$ In the current paper, we assume that all the $d_{k}^{a}$’s and $d_{k}^{b}$’s are bounded.

Let $\Omega_{A}(\cdot)$ and $\Omega_{B}(\cdot)$ be the subordination functions associated with $\mu_{A}$ and $\mu_{B}$. We will see that a spike $\widehat{a}_{i},1\leq i\leq r$ or $\widehat{b}_{j},1\leq j\leq s,$ causes an outlier eigenvalue, if

[TABLE]

More precisely, we will use the following assumption.

Assumption 2.6**.**

We assume that (2.10) holds for all $1\leq i\leq r$ and $1\leq j\leq s.$ Moreover, we define the integers $0\leq r^{+}\leq r$ and $0\leq s^{+}\leq s$ by

[TABLE]

and take their value to be zero when the index sets are empty. The lower bound $N^{-1/3}$ is chosen for definiteness, and it can be replaced with any $N$-dependent parameter that is of the same order.

Remark 2.7*.*

A spike $\widehat{a}_{i}$ or $\widehat{b}_{j}$ that does not satisfy the conditions in Assumption 2.6 will cause an outlier eigenvalue that lies within an $\mathrm{O}(N^{-2/3})$ neighborhood of the edge $E_{+}.$ In this sense, it will be hard to detect such a spike as $\mathrm{O}(N^{-2/3})$ exactly matches the scale of eigenvalue spacings around the edge in the non-spiked model as demonstrated in Lemma 4.20. In this sense, Assumption 2.6 simply chooses the actual spikes. In the statistical literature, this is referred to as the supercritical regime and a reliable detection of the spikes is only available in this regime. We refer the readers to [5, 18, 34, 53] for more detailed discussion.

Throughout the paper, for the ease of discussion, we will consistently use the following notations

[TABLE]

and for $\widehat{Y}$ is defined in (1.1)

[TABLE]

Remark 2.8*.*

We provide a remark on how to estimate the quantities in (2.10). We use $E_{+}$ and $\Omega_{\beta}(E_{+})$ for examples. First, according to (3.1) and (3.6) below, $E_{+}$ can be efficiently estimated using the largest eigenvalue of $\mathcal{Q}_{1},$ denoted as $\lambda_{1}(\mathcal{Q}_{1}),$ or the first non-outlier eigenvalue of $\widehat{\mathcal{Q}}_{1}.$ Second, according Lemmas A.3 and A.7 and (C.34) of [32], we find that $|\Omega_{B}^{c}(\widehat{z})-\Omega_{\beta}(E_{+})|\prec N^{-1/3},$ where for some sufficiently small constant $\epsilon>0,$ we denote

[TABLE]

This shows that $\operatorname{Re}\Omega_{B}^{c}(\widehat{z})$ can serve as a consistent estimator for $\Omega_{\beta}(E_{+}).$

3 Main results

In this section, we state the main results. Throughout the paper, we will consistently use the notion of stochastic domination, which was firstly introduced in [38]. It simplifies the presentation by systematizing the statements of the form ”$X_{N}$ is bounded by $Y_{N}$ with high probability up to a small power of $N$.”

Definition 3.9**.**

For two sequences of random variables $\{X_{N}\}_{N\in\mathbb{N}}$ and $\{Y_{N}\}_{N\in\mathbb{N}},$ we say that $X_{N}$ is stochastically dominated by $Y_{N},$ written as $X_{N}\prec Y_{N}$ or $X_{N}=\mathrm{O}_{\prec}(Y_{N}),$ if for all (small) $\epsilon>0$ and (large) $D>0,$ we have

[TABLE]

for sufficiently large $N\geq N_{0}(\epsilon,D).$ If $X_{N}(\upsilon)$ and $Y_{N}(\upsilon)$ depend on a common parameter $\upsilon,$ we say $X_{N}\prec Y_{N}$ uniformly in $\upsilon$ if the threshold $N_{0}(\epsilon,D)$ can be chosen independently of the parameter $\upsilon.$ Moreover, we say an event $\Xi$ holds with high probability if for any constant $D>0,$ $\mathbb{P}(\Xi)\geq 1-N^{-D}$ for large enough $N.$

3.1 Eigenvalue statistics

In this subsection, we state our results regarding the eigenvalue statistics. The statements often involve the inverse functions $\Omega_{A}^{-1}$ and $\Omega_{B}^{-1}$; since $\Omega_{A}$ and $\Omega_{B}$ are monotone increasing and real analytic on $(\widehat{E}_{+},\infty)$ by Lemma 2.5, we denote by $\Omega_{A}^{-1}(\cdot)$ and $\Omega_{B}^{-1}(\cdot)$ as the inverse functions of $\Omega_{A}$ and $\Omega_{B}$ on complex neighborhoods of $(\Omega_{A}(\widehat{E}_{+}),\infty)$ and $(\Omega_{B}(\widehat{E}_{+}),\infty),$ respectively. When $x<\Omega_{A}(\widehat{E}_{+})$ and $y<\Omega_{B}(\widehat{E}_{+})$, we use the conventions $\Omega_{A}^{-1}(x)=E_{+}$ and $\Omega_{B}^{-1}(y)=E_{+}$. For the ease of statements, we introduce the following re-labeling for the eigenvalues of $\widehat{\mathcal{Q}}_{i}$’s as in [34, Definition 3.5].

Definition 3.10**.**

We define the labeling functions $\pi_{a},\pi_{b}:\llbracket 1,N\rrbracket\to\llbracket 1,N\rrbracket$ as follows. For any $1\leq i\leq r$, we assign to it a label $\pi_{a}(i)\in\{1,\cdots,r+s\}$ if $\Omega_{B}^{-1}(\widehat{a}_{i})$ is the $\pi_{a}(i)$-th largest element in $\{\Omega_{B}^{-1}(\widehat{a}_{i})\}_{i=1}^{r}\cup\{\Omega^{-1}_{A}(\widehat{b}_{j})\}_{j=1}^{s}$. We also assign to any $1\leq j\leq s$ a label $\pi_{b}(j)\in\{1,\cdots,r+s\}$ in a similar way. Moreover, we define $\pi_{a}(i)=\min(i+s,N)$ if $i>r$ and similarly for $\pi_{b}(j)$. We define the following sets of outlier indices:

[TABLE]

and

[TABLE]

where we recall the definitions of $r^{+}$ and $s^{+}$ in (2.11).

First, we state the results on the convergence limits and rates for the outliers and the first few non-outlier eigenvalues. Recall (2.13). Denote the eigenvalues of $\widehat{\mathcal{Q}}_{1}$ and $\widehat{\mathcal{Q}}_{2}$ as $\widehat{\lambda}_{1}\geq\widehat{\lambda}_{2}\geq\cdots\geq\widehat{\lambda}_{N}.$

Theorem 3.11** (Outlier and extremal non-outlier eigenvalues).**

Suppose Assumptions 2.2 and 2.6 hold. Then we have that

[TABLE]

and

[TABLE]

Moreover, for any fixed integer $\varpi>r+s,$ we have

[TABLE]

Theorem 3.11 offers the concentration bounds for the locations of the outlier and first few extremal non-outlier eigenvalues. It shows that the convergence rates of the outliers change from the order of $N^{-1/2}(\widehat{a}_{i}-\Omega_{\beta}(E_{+}))^{1/2}$ or $N^{-1/2}(\widehat{b}_{j}-\Omega_{\alpha}(E_{+}))^{1/2}$ to $N^{-2/3}$ once $\widehat{a}_{i}-\Omega_{\beta}(E_{+})$ or $\widehat{b}_{j}-\Omega_{\alpha}(E_{+})$ cross the scale $N^{-1/3},$ as opposed to Assumption 2.6. This indicates the occurrence of BBP transition [3]. Moreover, we believe that under some spectral gap assumptions as in [1, 2, 3, 5, 18, 35], the distributions of the outlier eigenvalues can also be studied. We will pursue this direction in future works.

Remark 3.12*.*

We remark that the locations, in other words, convergence limits of the outlier eigenvalues have been obtained without Assumption 2.2 in [12, Section 2.2] for the spiked unitarily invariant model. However, in [12], the conditions in Assumption 2.6 are stronger in the sense that

[TABLE]

and

[TABLE]

where $\varsigma>0$ is some fixed constant. Compared to [12], we extend the results on the outlier eigenvalues in the following aspects: We consider spikes on a finer scale in Assumption 2.6, establish their convergent rates, and provide the results for the extremal non-outlier eigenvalues. We believe that Assumption 2.6 is the most general assumption possible for the existence of the outliers, and the convergent rates obtained here are optimal up to an $N^{\epsilon}$ factor, where $\epsilon>0$ can be any (small) constant.

Remark 3.13*.*

On macroscopic level, [12] covers more general settings assuming (3.2) and (3.3). Firstly, the limiting measures $\mu_{\alpha}$ and $\mu_{\beta}$ are only assumed to be non-trivial and compactly supported. In this general setting the resulting set of outliers is (see [12, Theorem 2.2])

[TABLE]

which is consistent with our Theorem 3.11. It should be noted that, under Assumption 2.2, each set $\Omega_{B}^{-1}(\{a_{i}\})$ necessarily has at most one element. In other words, each $\widehat{a}_{i}$ or $\widehat{b}_{j}$ gives rise to at most one outlier. To see this, recall the following three facts from Lemma 2.5; (i) $\Omega_{\beta}$ is an increasing real analytic function on $\mathbb{R}\setminus\operatorname{supp}(\mu_{\alpha}\boxtimes\mu_{\beta})$ with $\Omega_{\beta}(\pm\infty)=\pm\infty$, (ii) $\operatorname{\mathrm{Im}}\Omega_{\beta}>0$ in the interior of $\operatorname{supp}(\mu_{\alpha}\boxtimes\mu_{\beta})$, and (iii) $\Omega_{\beta}(E_{+})>E_{+}^{\beta}$ and $\Omega_{\beta}(E_{-})<E_{-}^{\beta}$. From these facts and that $\mu_{\alpha}\boxtimes\mu_{\beta}$ is supported on a single interval, we easily find that each of $\Omega_{\beta}^{-1}(\widehat{a}_{i})$ has at most one element.

However, for general $\mu_{\alpha}$ and $\mu_{\beta}$, a single spike $\widehat{a}_{i}$ or $\widehat{b}_{j}$ may result in several outliers; see [12, Examples 2.3 and 2.4] for concrete examples. We believe that the three facts above extend to fairly general $\mu_{\alpha}$ and $\mu_{\beta}$, but their free convolution may have disconnected support, in particular if either $\mu_{\alpha}$ or $\mu_{\beta}$ does. In this case, each interval between connected components of $\operatorname{supp}(\mu_{\alpha}\boxtimes\mu_{\beta})$ may contain a solution $x$ of the equation $\Omega_{B}(x)=\widehat{a}_{i}$, in contrast to our case where the support is connected.

Finally, we point out that [12] also studied the additive and circular models, namely $A+UBU^{*}$ and $AUBU^{*}$ for Hermitian and unitary matrices $A,B$, respectively. We believe the counterpart of our results for the additive model can be proved analogously, since the two major inputs, the square-root decay of $\mu_{\alpha}\boxplus\mu_{\beta}$ near the edges and the optimal edge local laws, have already been proved in [7]. As for the circular model, neither of the two inputs is known to our knowledge; the latest result on the free multiplicative convolution on the circle is [13], which shows absolute continuity. We will pursue this direction in the future works.

Second, we study the non-outlier eigenvalues of $\widehat{\mathcal{Q}}_{1}$ for $i>r^{+}+s^{+}.$ These eigenvalues are governed by the eigenvalue sticking property, which states that the non-outlier eigenvalues of $\widehat{\mathcal{Q}}_{1}$ ”stick” with those of $\mathcal{Q}_{1}$ defined in (2.13) with high probability. Recall that we denote the eigenvalues of $\mathcal{Q}_{1}$ as $\lambda_{1}\geq\lambda_{2}\geq\cdots\geq\lambda_{N}.$

Theorem 3.14** (Eigenvalue sticking).**

Suppose Assumptions 2.2 and 2.6 hold. Define

[TABLE]

Fix any small constant $\tau>0.$ We have that

[TABLE]

Theorem 3.14 provides the convergence limits and rates for the non-outlier eigenvalues of $\widehat{\mathcal{Q}}_{1}$ in terms of those of $\mathcal{Q}_{1}.$ On one hand, if $\gamma\sim 1,$ together with the rigidity of $\mathcal{Q}_{1}$ established in [32] (or see Lemma 4.20), we conclude that the non-outlier eigenvalues of $\widehat{\mathcal{Q}}_{1}$ will converge to the quantiles of $\mu_{A}\boxtimes\mu_{B}$ with optimal rates. On the other hand, once $\gamma\gg N^{-1/3}$ (which is slightly stronger than the condition in Assumption 2.6), for $i=\mathrm{O}(1),$ we find that the right-hand side of (3.6) is much smaller than $N^{-2/3}$ obtained for the edge eigenvalues of $\mathcal{Q}_{1}$ as in Lemma 4.20. In future works, we will show that the edge eigenvalues of $\mathcal{Q}_{1}$ follow Tracy-Widom distribution, which immediately implies the Tracy-Widom asymptotics of the largest non-outlier eigenvalue of $\widehat{\mathcal{Q}}_{1}.$

3.2 Eigenvector statistics

In this subsection, we introduce the results regarding the singular vectors, that is, the eigenvectors of $\widehat{\mathcal{Q}}_{1}$ and $\widehat{\mathcal{Q}}_{2}$ in (2.13). In what follows, we consider an index set $S$ such that

[TABLE]

For convenience, we use the following notations. For $1\leq i_{1}\leq r^{+}$, $1\leq i_{2}\leq N$ and $1\leq j\leq N$, we define

[TABLE]

Similarly, for $1\leq j_{1}\leq s^{+}$, $1\leq j_{2}\leq N$ and $1\leq i\leq N$, we define

[TABLE]

Further if $\mathfrak{a}\in S$, we define

[TABLE]

if $\mathfrak{a}\notin S$, then we define

[TABLE]

With the above preparation, we proceed to state our main results on the outlier singular vectors. Denote the projections

[TABLE]

where $\{\widehat{\mathbf{u}}_{k}\}$ and $\{\widehat{\mathbf{v}}_{k}\}$ are the eigenvectors of $\widehat{\mathcal{Q}}_{1}$ and $\widehat{\mathcal{Q}}_{2}$ in (2.13), respectively.

Theorem 3.15** (Outlier eigenvectors).**

Suppose that Assumptions 2.2 and 2.6 hold. For the set $S$ in (3.7) and any given deterministic vector $\mathbf{v}=(v_{1},\cdots,v_{N})^{*}\in\mathbb{C}^{N},$ we have that for the left singular vectors,

[TABLE]

where $g_{a}(\mathbf{v},S)$ is defined as

[TABLE]

Similarly, for the right singular vectors, we have

[TABLE]

where $g_{b}(\mathbf{v},S)$ is defined as

[TABLE]

Theorem 3.15 establishes the concentration bounds for the generalized components of the outlier singular vectors. It demonstrates that the singular vectors are concentrated on a cone with axis parallel to the true singular vectors with an explicit aperture depending on the spikes and subordination functions. We consider an example for illustration. For simplicity, we consider the non-degenerate case such that all the outliers are well-separated in the sense that we can simply choose $S=\{\pi_{a}(i)\}$ or $S=\{\pi_{b}(j)\}.$ Let $S=\{\pi_{a}(i)\}$ and $\mathbf{v}=\mathbf{e}_{i}.$ Then we obtain from Theorem 3.15 that

[TABLE]

It is easy to see from (6.2), (6.12) and (6.14) that the error term is much smaller than the first term of the right-hand side of (3.12). In this sense, $\widehat{\mathbf{u}}_{i}$ is concentrated on a cone with axis parallel to $\mathbf{e}_{i}.$

Remark 3.16*.*

We mention that some partial results of Theorem 3.15 have been obtained in (4) of Theorem 2.5 in [12] for the spiked unitarily invariant model. More specifically, by assuming that $r=0$ or $s=0,$ (3.2) and (3.3), they obtained the concentration limit (3.11). We extend the counterparts in [12] by, on one hand, stating the results in a less restrictive setting, and on the other hand, establishing their convergent rates.

Finally, we state the main results regarding the non-outlier singular vectors. Denote

[TABLE]

Theorem 3.17** (Non-outlier eigenvectors).**

Suppose that Assumptions 2.2 and 2.6 hold. Fix any small constant $\tau>0.$ For $\pi_{a}(i)\notin\mathcal{O}_{+},\ i\leq\tau N$ and any deterministic vector $\mathbf{v}=(v_{1},\cdots,v_{N})^{*}\in\mathbb{R}^{N},$ we have that

[TABLE]

Similarly, for the right singular vectors, we have

[TABLE]

Theorem 3.17 establishes the results for the non-outlier eigenvectors. Especially, if the spikes are well separated from the critical values at a distance of constant order, (3.13) implies that the non-outlier eigenvectors are completely delocalized in the direction of the associated spiked eigenvectors. More specifically, assuming that for the spike $\widehat{a}_{l}\geq\Omega_{\beta}(E_{+})+\varsigma,$ for some fixed constant $\varsigma>0$ and $1\leq l\leq r,$ we consider the direction of the associated spiked eigenvector, i.e., $\mathbf{v}=\mathbf{e}_{l}.$ In such a setting, for $\pi_{a}(i)\notin\mathcal{O}_{+},$ the result of (3.13) reads

[TABLE]

which indicates that the eigenvector $\widehat{\mathbf{u}}_{\pi_{a}(i)}$ is completely delocalized when projected on the eigenvector associated with $\widehat{a}_{l}.$ Similar results hold for the right singular vectors.

4 Preliminary results and proof strategy

In this section, we collect and prove some preliminary results which will be used in our technical proof. For any spectral parameter $z=E+\mathrm{i}\eta\in\mathbb{C}_{+}$, we define

[TABLE]

where $E_{+}$ is the rightmost edge of $\mu_{\alpha}\boxtimes\mu_{\beta}$ given in (2.7). For each $0\leq a<b$ and $0<\tau<\min\{\frac{E_{+}-E_{-}}{2},1\}$, we define the following set of spectral parameter $z$ by

[TABLE]

Furthermore, for $0<\xi<1$, we let

[TABLE]

and let $\eta_{U}>1$ be a large $N$-independent constant. Finally, we define another domain of spectral parameters

[TABLE]

4.1 Properties of free multiplicative convolution and subordination functions

In this subsection, we collect some important properties regarding $\mu_{\alpha}\boxtimes\mu_{\beta},\mu_{A}\boxtimes\mu_{B}$ and their associated subordination functions. In the following lemma, we summarize the results regarding the properties of $\mu_{A}\boxtimes\mu_{B}$ on the spectral domains as in (4.2) and (4.4).

Lemma 4.18**.**

Suppose Assumption 2.2 holds. Fix some small constant $\tau>0,$ denote

[TABLE]

Then for sufficiently large $N$, the following hold uniformly over $z\in{{\mathcal{T}}}$:

• (1)

We have

[TABLE]

• (2)

For $\kappa$ defined in (4.1), we have

[TABLE]

• (3)

For the derivatives of $\Omega_{A}$ and $\Omega_{B}$, we have

[TABLE]

• (4)

For all fixed $\epsilon>0$, we have

[TABLE]

Proof.

See Proposition 2.11 and its proof in [32]. ∎

4.2 Local laws for free multiplication of random matrices

In this subsection, we prove the local laws which are the key ingredients for our technical proof. We start with introducing some notations. Let $U\equiv U_{N}$ be a random unitary or orthogonal matrix, Haar distributed on the $N$-dimensional unitary group $U(N)$ or orthogonal group $O(N)$. Define $\widetilde{A}\mathrel{\mathop{:}}=U^{*}AU$, $\widetilde{B}\mathrel{\mathop{:}}=UBU^{*}$, and

[TABLE]

Note that we only need to consider diagonal matrices $A$ and $B$ since $U$ is a Haar random unitary or orthogonal matrix. Moreover, $\widetilde{H}$ and $\widetilde{{\mathcal{H}}}$ are Hermitian random matrices.

Since $H$, ${\mathcal{H}}$, $\widetilde{H}$ and $\widetilde{{\mathcal{H}}}$ have the same eigenvalues, we denote the common eigenvalues as $\lambda_{1}\geq\lambda_{2}\geq\cdots\geq\lambda_{N}$ in what follows. For $z\in\mathbb{C}_{+}:=\{z\in\mathbb{C}:\operatorname{\mathrm{Im}}z>0\}$, we define the resolvents of the above random matrices as follows

[TABLE]

In the rest of the paper, we usually omit the dependence of $z$ and simply write $G,{\mathcal{G}},\widetilde{G}$ and $\widetilde{{\mathcal{G}}}.$ Let $\mu_{H}$ be the ESD of $H$ and $m_{H}(z)$ be the associated Stieltjes transform. Since $H,{\mathcal{H}},\widetilde{H}$ and $\widetilde{{\mathcal{H}}}$ are similar to each other, we have that $m_{H}(z)=\operatorname{tr}G=\operatorname{tr}{\mathcal{G}}=\operatorname{tr}\widetilde{G}=\operatorname{tr}\widetilde{{\mathcal{G}}}.$

Our proof relies on the following linearization crucially. For $z\in\mathbb{C}_{+}$, denote $Y=A^{1/2}UB^{1/2}$ and $\mathbf{H}\equiv\mathbf{H}(z)$ as

[TABLE]

and $\mathbf{G}(z)=(\mathbf{H}-z)^{-1}.$ By Schur’s complement, it is easy to see that

[TABLE]

where we recall the definitions in (4.9). To simplify notations, we define the index sets

[TABLE]

Then we relabel the indices of the matrices according to

[TABLE]

In the proof of the paper, we will consistently use the latin letters $i,j\in\mathcal{I}_{1}$ and greek letters $\mu,\nu\in\mathcal{I}_{2}$. We denote the $2N\times 2N$ diagonal matrix $\Theta\equiv\Theta(z)$ by letting

[TABLE]

We have the following controls for the resolvent $\mathbf{G}(z)$ uniformly in $z\in{\mathcal{T}_{\tau}(\eta_{L},\eta_{U})}.$ Denote $\gamma_{j}$ as the $j$-th $N$-quantile (or typical location) of $\mu_{\alpha}\boxtimes\mu_{\beta}$ such that

[TABLE]

Similarly, we denote $\gamma_{j}^{*}$ to be the $j$-th $N$-quantile of $\mu_{A}\boxtimes\mu_{B}.$

Theorem 4.19** (Local laws).**

Suppose Assumption 2.2 holds. Let $\tau$ and $\xi$ be fixed small positive constants. We have that

[TABLE]

and

[TABLE]

hold uniformly in $z\in{\mathcal{T}_{\tau}(\eta_{L},\eta_{U})}.$ Moreover, far away from the spectrum, for $z\in{\mathcal{T}_{\tau}(\eta_{U})}$ uniformly, we have that

[TABLE]

and

[TABLE]

Proof.

When $z\in\mathcal{T}_{\tau}(\eta_{L},\eta_{U}),$ (4.15) has been proved in Theorem 2.13 of [32]. Moreover, (4.14) has also been established therein with a slightly different form and it can be proved following the lines of the proof of [32, Theorem 2.13]; see Remark 2.14 therein for more details.

When $z\in\mathcal{T}_{\tau}(\eta_{U}),$ (4.16) follows from (2.8), Lemma 4.18, and (4.14). The calculation is standard in the random matrix theory literature; for instance, see Theorem 3.12 and its proof in [17]. We omit the details here. Finally, we prove (4.17). The discussion is similar to that of equation (2.19) of [7] and we only sketch the key points. We fix an arbitrary chosen $\epsilon\in(0,\tau/100)$ and consider the event $\Xi$ on which we have

[TABLE]

where in the proof we choose $\eta_{L}=N^{-1+\epsilon}$. By (4.14), (4.15) and Lemma 4.20 below, we have $\mathbb{P}\left[\Xi\right]\geq 1-N^{-D}$ for any large $D>0$. For all $z_{0}=E_{0}+\mathrm{i}\eta_{0}\in{{\mathcal{T}}_{\tau}(\eta_{U})}$ with $4\eta_{0}\leq\kappa_{0}=|E_{0}-E_{+}|$, we consider a counter-clockwise square contour ${\mathcal{C}}(z_{0})$ with side length $\kappa_{0}$ and (bary)center $z_{0}$. Then, on the event $\Xi,$ by Cauchy’s theorem, we have

[TABLE]

where ${\mathcal{C}}_{>}(z_{0})={\mathcal{C}}(z_{0})\cap\{z:|\operatorname{\mathrm{Im}}z|>\eta_{L}\}$ and ${\mathcal{C}}_{\leq}(z_{0})={\mathcal{C}}(z_{0})\cap\{z:|\operatorname{\mathrm{Im}}z|\leq\eta_{L}\}$. On the contour ${\mathcal{C}}_{>}(z_{0}),$ we use the first bound in (4.18) to get that for some constant $C>0$

[TABLE]

On the other hand, for $z$ on the other contour ${\mathcal{C}}_{\leq}(z_{0}),$ we use

[TABLE]

where in the second step we used the second bound in (4.18) and in the third step we used the fact that $E_{+}-\gamma_{i}\sim i^{2/3}N^{-2/3}$. Following the same argument and using Lemma 4.20 below, we get $|m_{\mu_{A}\boxtimes\mu_{B}}(z)|\leq C$. Using the above bounds, we get

[TABLE]

Combining (4.19) and (4.20), we conclude our proof.

∎

Finally, we collect two important consequences of (4.14) and (4.15). Recall (4.13) and $\lambda_{1}\geq\lambda_{2}\geq\cdots\geq\lambda_{N}$ are the eigenvalues of $AUBU^{*}.$

Lemma 4.20** (Spectral rigidity near the upper edge).**

Suppose Assumption 2.2 holds true. For any small constant $0<c<1/2,$ we have that for all $1\leq i\leq cN,$

[TABLE]

Moreover, the same conclusion holds if $\gamma_{i}^{*}$ is replaced with $\gamma_{i}.$

Proof.

See Theorem 2.15 of [32]. ∎

Denote the singular value decomposition (SVD) of $Y=A^{1/2}UB^{1/2}$ as

[TABLE]

where $\{\mathbf{u}\}_{k}$ and $\{\mathbf{v}_{k}\}$ are the left and right singular vectors of $Y,$ respectively.

Lemma 4.21** (Delocalization of the singular vectors).**

Suppose Assumption 2.2 holds true. For any small constant $0<c<1/2,$ we have that for all $1\leq k\leq cN,$

[TABLE]

Proof.

See Theorem 2.16 of [32]. ∎

4.3 Proof strategy

In this subsection, we summarize the proof strategies. We focus on explaining how to adapt the techniques of the spiked covariance matrix model [18] and separable covariance matrix model [34] to obtain the results for our spiked multiplicative model. Analogous strategies have also been used to study the deformed Wigner matrix in [43, 44], general spiked sample covariance matrix in [31] and signal-plus-noise matrix in [30].

First, we discuss how to prove Theorem 3.11. For the outlier eigenvalues, their locations satisfy a master equation in terms of the resolvent as summarized in Lemma 4.22 below. Recall (2.9). Denote

[TABLE]

where $\mathbf{E}_{r}=(\mathbf{e}_{1},\cdots,\mathbf{e}_{r})$,  $\mathbf{E}_{s}=(\mathbf{e}_{1},\cdots,\mathbf{e}_{s})$, $D^{a}=\operatorname{diag}(d_{1}^{a},\cdots,d_{r}^{a})$ and $D^{b}=\operatorname{diag}(d_{1}^{b},\cdots,d_{s}^{b}).$ Recall (2.13) and (4.11). Recall $\mathbf{H}(z)$ in (4.10). Define $\widehat{\mathbf{H}}\equiv\widehat{\mathbf{H}}(z)$ as

[TABLE]

Correspondingly, we denote $\widehat{\mathbf{G}}(z)=(\widehat{\mathbf{H}}(z)-z)^{-1}.$

Lemma 4.22**.**

If $x\neq 0$ is not an eigenvalue of $\mathcal{Q}_{1},$ then it is an eigenvalue of $\widehat{\mathcal{Q}}_{1}$ if and only if

[TABLE]

Proof.

By the linearization construction in (4.22), we find that the eigenvalues of $z^{-1/2}\widehat{\mathbf{H}}$ are given by

[TABLE]

Hence, it is clear that $x>0$ is an eigenvalue of $\widehat{\mathcal{Q}}_{1}$ if and only if

[TABLE]

Since $\mathbf{P}$ and $\bm{\mathcal{D}}$ are always invertible, $x\neq 0$ is an eigenvalue of $\widehat{\mathbf{H}}=\mathbf{P}\mathbf{H}\mathbf{P}$ if and only if

[TABLE]

where in the second step we used $\det(1+AB)=\det(1+BA)$. We can then conclude that proof as $x$ is not an eigenvalue of $\mathcal{Q}_{1}.$

∎

Heuristically, by Theorem 4.19 and Lemma 4.22, an outlier location $x>E_{+}$ should satisfy the condition that

[TABLE]

Therefore, solving (4.23) will yield the locations of the outlier eigenvalues. Moreover, according to (2.8) and (4.7), we find that

[TABLE]

which in turn is equivalent to $\widehat{a}_{i}>\Omega_{\beta}(E_{+}).$ Similar calculation holds for $\widehat{b}_{j},1\leq j\leq s.$ Furthermore, to obtain the convergence rates for the outlier eigenvalues, it suffices to apply the strategy developed in [18]. The proof consists of the following three steps: (1). Construct the permissible regions which contain all the eigenvalues of $\widehat{\mathcal{Q}}_{1}$ with high probability (c.f. Lemma 5.24). This step enables us to find the a union of sets in which all the eigenvalues including the outliers lie. (2). Apply a counting argument using Rouché’s theorem to a special case where the algebraic multiplicities of the spikes are all unity to show that each connected component of the permissible region contains the correct number of the eigenvalues of $\widehat{\mathcal{Q}}_{1}.$ (3). Use a continuous interpolation argument to extend Step (2) to the general case with a careful discussion on the gaps in the permissible region for the general setting. We point out that Steps (2) and (3) are quite standard, for example see [43, 18, 34], and we will focus on explaining the differences lying in Step (1) in the proof. For the extremal non-outlier eigenvalues, the proof makes use of the results in Steps (2) and (3). The details will be provided in Section 5.1.

Second, we explain how to justify Theorem 3.14. The proof again consists of three steps similar to those described earlier. In the first step, we construct the permissible regions for the eigenvalues; see Lemma 5.27 below for more details. In the second step, we prove our results to a special case (c.f. Lemma 5.28) and in the third step, we prove Theorem 3.14 using a continuity argument. Compared to the outlier eigenvalues, the second step is more complicated since Rouché’s theorem is not applicable. Instead, we need to employ a perturbation argument. The details can be found in Section 5.2.

Third, we discuss how to prove Theorem 3.15. Recall (4.22) and $\widehat{\mathbf{G}}(z)$ defined around it. We first provide some useful expressions. By a discussion similar to (4.11) and the singular value decomposition (SVD) of $Y$, we have that

[TABLE]

[TABLE]

Following the strategy of [18, 34], the starting point is an integral representation. Specifically, by (4.24), Lemma 6.31 below and Cauchy’s integral formula, for some properly chosen contour $\Gamma$ around the outliers, we have that

[TABLE]

where $\bm{e}_{i}$ and $\bm{e}_{j}$ are the natural embeddings of $\mathbf{e}_{i}$ and $\mathbf{e}_{j}$ in $\mathbb{C}^{2N}.$ Next, we provide an identity for $\mathbf{U}^{*}\widehat{\mathbf{G}}(z)\mathbf{U}$ in terms of $\mathbf{U}^{*}\mathbf{G}(z)\mathbf{U}$ (recall (4.21)). For the matrices $\mathcal{A},\mathcal{S},\mathcal{B}$, and $\mathcal{E}$ of conformable dimensions, by Woodbury matrix identity, we have

[TABLE]

as long as all the operations are legitimate. Moreover, when $\mathcal{A}+\mathcal{B}$ is non-singular, we have that

[TABLE]

By (4.22), (4.24) and the matrix identities (4.26) and (4.27), we have that

[TABLE]

where we used $(I-\bm{{\mathcal{D}}})=\mathbf{P}^{-2}$ and $\mathbf{P}$ is defined in (4.22). On one hand, we can expand (4.25) using (4.3). On the other hand, since (4.3) can be well-estimated using the local laws Theorem 4.19, the limit of (4.25) can be calculated using the residue theorem. To obtain the convergence rates, we need to apply a high order resolvent expansion for (4.3) as in (6.1). In the actual proof, we split it into two steps due to some technicalities. In the first step, we prove the results under a slightly stronger assumption than Assumption 2.6 that

[TABLE]

where $\tau_{1}>0$ is some small positive constant. The details are provided in Proposition 6.29 and proved in Section 6.1. Then in the second step, we remove (4.29) using a finer estimate to conclude the proof of Theorem 3.15. The details are provided in Section 6.3.

Finally, we explain how to handle the non-outlier eigenvectors in Theorem 3.17. In this setting, the residual calculus fails since the contour representation in (4.25) is invalid. Instead, we utilize the inequality

[TABLE]

where $\widehat{G}(z)=(\widehat{\mathcal{Q}}_{1}-z)^{-1}$ and $z_{k}=\widehat{\lambda}_{k}+\mathrm{i}\eta.$ Consequently, it suffices to obtain an accurate estimate for $\operatorname{\mathrm{Im}}\mathbf{v}^{*}\widehat{G}(z_{k})\mathbf{v}.$ We again use the local law Theorem 4.19 combining with a resolvent expansion to establish the delocalization bounds. The details can be found in Section 6.2.

Remark 4.23*.*

We remark that the results regarding outlier eigenvalues and eigenvectors in Theorems 3.11 and 3.15 remain true even if we loosen Assumption 2.2 in the following sense: (1). (iii) and (v) can be replaced by

[TABLE]

for a small constant $c>0$, and (2). the error in (2.2) can be replaced by $N^{-2/3-\epsilon}.$ However, under such weaker assumptions, the results in Theorems 3.14 and 3.17 will be weakened, especially (3.6) and (3.13) only hold up to $i\leq N^{\tau}$ for some small $0<\tau<1.$ Technically, this is because the proof of Theorems 3.11 and 3.15 only rely on the square root behavior of $\mu_{A}\boxtimes\mu_{B}$ and the local laws near the edges and outside the bulk of the spectrum, i.e., Theorem 4.19 holds for $\mathcal{T}_{\tau}(n^{-2/3+\epsilon},\eta_{U})$ and $\mathcal{T}_{\tau}(\eta_{U}).$ For the square root decay behavior, according to [32, Equation (A.17)],

[TABLE]

In order to ensure that the error is dominated near the edge by $\operatorname{Im}m_{\mu_{\alpha}\boxtimes\mu_{\beta}}(z)\sim\sqrt{|z-E_{+}|}$ for all $\operatorname{Im}z\gg N^{-2/3}$, we need $\bm{d}\ll N^{-2/3}$. Armed with the square root decay behavior, the proof only needs the weakened conditions mentioned above.

In contrast, in order to fully prove Theorems 3.14 and 3.17, we need (2.2) so that (4.31) implies $\operatorname{Im}m_{\mu_{\alpha}\boxtimes\mu_{\beta}}(z)$ is the main term for all $\operatorname{Im}z\gg\eta_{L}$. We also need the current Theorem 4.19 which relies on Assumption 2.2; see [18, 43] for more details.

5 Proof of Theorems 3.11 and 3.14: the eigenvalue statistics

In this section, we prove the main results of Section 3.1 following the strategies outlined in Section 4.3. Due to similarity, we focus on explaining the main differences from the counterparts in [18, 34] and how to adapt their proof strategies.

5.1 Proof of Theorem 3.11

In this subsection, we prove the results for the outlier and extremal non-outlier eigenvalues. By Theorem 4.19 and Lemma 4.20, for any fixed small constant $\epsilon>0,$ we can choose a high probability event $\Xi\equiv\Xi(\epsilon)$ where the following estimates hold:

[TABLE]

[TABLE]

[TABLE]

We will restrict our proof to $\Xi$ in what follows and hence the discussion below will be entirely deterministic. We first prepare some notations following the proof of [34, Theorem 3.6]. For any fixed constant $\epsilon>0,$ we denote in index sets

[TABLE]

Here and after, we use $\widehat{b}_{\mu}:=\widehat{b}_{\mu-N}$ for $\mu\in\mathcal{I}_{2}$. Recall that the eigenvalues of $\widehat{A}$ and $\widehat{B}$ are ordered in the decreasing fashion. By definition, it is easy to see that

[TABLE]

Moreover, since we are mainly interested in the outlier and extremal non-outlier eigenvalues, we use the convention that $\Omega_{B}^{-1}(\widehat{a}_{i})=E_{+},i\geq r$ and $\Omega_{A}^{-1}(\widehat{b}_{\mu})=E_{+},\mu\geq N+s.$ Throughout the proof, we will need the following estimate following from (2.8) and (4.7), for $i\in{\mathcal{O}}_{\epsilon}^{(a)},$

[TABLE]

Indeed, when $\Omega_{B}^{-1}(\widehat{a}_{i})-E_{+}\leq\varsigma_{1}$ for some sufficiently small constant $0<\varsigma_{1}<1,$ using (2.8), (4.7) and the fact $\Omega_{B}(\cdot)$ is monotone increasing, we readily see that

[TABLE]

where $\gamma>0$ is some constant. This immediately implies (5.5) using Assumption 2.6. On the other hand, when $\Omega^{-1}_{B}(\widehat{a}_{i})-E_{+}\geq\varsigma_{1},$ since $\Omega_{B}(\cdot)$ is increasing, we obtain that

[TABLE]

This proves the claim (5.5). A consequence of (5.5) is that

[TABLE]

Similarly, we have that

[TABLE]

An advantage of the above labeling is that the largest outliers of $\widehat{\mathcal{Q}}_{1}$ can be labelled according to $i\in\mathcal{O}_{\epsilon}^{(a)}$ and $\mu\in\mathcal{O}_{\epsilon}^{(b)}.$ Analogously to (D.9) and (D.10) of [34], we find that to prove Theorem 3.11, it suffices to prove that for arbitrarily small constant $\epsilon>0,$ there exists some constant $C>0$ so that

[TABLE]

for all $i\in\mathcal{O}_{4\epsilon}^{(a)}$ and $\mu\in\mathcal{O}_{4\epsilon}^{(b)},$ and

[TABLE]

for all $i\in\left\{1,2,\cdots,r\right\}\backslash\mathcal{O}_{4\epsilon}^{(a)}$ and $\mu\in\left\{N+1,\cdots,N+s\right\}\backslash\mathcal{O}_{4\epsilon}^{(b)}.$ Here we used the short-hand notations

[TABLE]

and $\pi_{b}(\mu)\equiv\pi_{b}(\mu-N)$ for $\mu\in{\mathcal{I}}_{2}$.

We now conclude the proof following the four steps outlined in Section 4.3. We will focus on explaining the first part, Step 1, since it differs the most from its counterpart of the proof of [34, Theorem 3.6], and briefly sketch Steps 2–4.

Step 1: For each $1\leq i\leq r^{+},$ we define the permissible intervals

[TABLE]

Similarly, for each $1\leq\mu-N\leq s^{+},$ we denote

[TABLE]

Then we define

[TABLE]

The main task of this step is to prove the following lemma.

Lemma 5.24**.**

On the event $\Xi$, the complement of $\mathrm{I}$ contains no eigenvalues of $\widehat{\mathcal{Q}}_{1}.$

Proof.

By Lemma 4.22, (5.3) and (5.2), we find that $x\notin\mathrm{I}_{0}$ is an eigenvalue of $\widehat{\mathcal{Q}}_{1}$ if and only if

[TABLE]

is singular. In light of (4.23), it suffices to show that if $x\notin\mathrm{I},$ then

[TABLE]

Indeed, when (5.12) holds, then the matrices on the left-hand side of (5.11) is non-singular. Note that

[TABLE]

where in the last equality we used (i) of Lemma 4.18. The rest of the proof is devoted to controlling (5.13) via mean value theorem.

First, we have that

[TABLE]

In fact, when $i\in\mathcal{O}_{\epsilon}^{(a)},$ (5.14) holds by definition. When $i\notin\mathcal{O}_{\epsilon}^{(a)},$ by (5.5) and the fact that $\Omega_{B}(x)$ is monotone increasing when $x>E_{+}$, we have

[TABLE]

Now we return to the proof of (5.12). We divide our proof into two cases. If there exists a constant $c>0$ such that $\Omega_{B}^{-1}(\widehat{a}_{i})\notin[x-c\kappa,x+c\kappa].$ Since $\Omega_{B}(\cdot)$ is monotonically increasing on $(E_{+},\infty),$ we have that

[TABLE]

where in the second step we used (4.6) and (4.7) with Cauchy’s integral formula when $x>E_{+}.$ On the other hand, if $\Omega_{B}^{-1}(\widehat{a}_{i})\in[x-c\kappa,x+c\kappa]$ such that $\Omega_{B}^{-1}(\widetilde{a}_{i})-E_{+}\sim\kappa,$ here $c<1$ is some small constant. By (5.5) and the fact $\widehat{a}_{i}-\Omega_{\beta}(E_{+})\geq N^{-1/3+\epsilon},$ we have that

[TABLE]

Moreover, by (4.6) and (4.7), we conclude that

[TABLE]

where we used (5.5) in the second step. Since $\Omega_{B}$ is monotonically increasing on $(E_{+},\infty),$ for $x\notin\mathrm{I}_{i}^{(a)},$ by (5.14) and (5.5), we conclude that

[TABLE]

The $d_{\mu}^{b}$ term can be dealt with in the same way, and this completes our proof. ∎

Step 2: In this step we will show that each $\mathrm{I}_{i}^{(a)}$, $i\in\mathcal{O}_{\epsilon}^{(a)}$, or $\mathrm{I}_{\mu}^{(b)}$, $\mu\in\mathcal{O}_{\epsilon}^{(b)}$, contains the right number of eigenvalues of $\widehat{\mathcal{Q}}_{1}$, under a special case; see (5.15) below. For simplicity, we relabel the indices in $\mathcal{O}_{\epsilon}^{(a)}\cup\mathcal{O}_{\epsilon}^{(b)}$ as $\widetilde{\sigma}_{1},\cdots,\widetilde{\sigma}_{r_{\epsilon}}$, and call them $\epsilon$-spikes. Moreover, we assume that they correspond to classical locations of outliers as $x_{1},\cdots,x_{r_{\epsilon}}$ (some of them are determined by $\Omega_{B}^{-1}(\widehat{a}_{i})$, while others are given by $\Omega_{A}^{-1}(\widehat{b}_{\mu})$), such that

[TABLE]

The corresponding permissible intervals $\mathrm{I}_{i}^{(a)}$ and $\mathrm{I}_{\mu}^{(b)}$ are relabelled as $\mathrm{I}_{i}$, $1\leq i\leq r_{\epsilon}$. In this step, we consider a special configuration $\mathbf{x}\equiv\mathbf{x}(0):=(x_{1},x_{2},\cdots,x_{r_{\epsilon}})$ of the outliers that is independent of $N$ and satisfies

[TABLE]

In this step, we claim that each $\mathbf{\mathrm{I}}_{i}(\mathbf{x})$, $1\leq i\leq r_{\epsilon}$, contains precisely one eigenvalue of $\widehat{\mathcal{Q}}_{1}$. Fix any $1\leq i\leq r_{\epsilon}$ and pick up a small $n$-independent positively oriented closed contour $\mathcal{C}\subset\mathbb{C}/[0,E_{+}]$ that encloses $x_{i}$ but no other point of the set $\{x_{i}\}_{i=1}^{r_{\epsilon}}.$ Define two functions

[TABLE]

The functions $h,l$ are holomorphic on and inside $\mathcal{C}$ when $n$ is sufficiently large by (5.3). Moreover, by the construction of $\mathcal{C},$ the function $l$ has precisely one zero inside $\mathcal{C}$ at $x_{i}.$ By (5.2), we have

[TABLE]

The claim then follows from Rouché’s theorem.

Step 3: In order to extend the results in Step 2 to arbitrary $N$-dependent configuration $\mathbf{x}_{N}$, we shall employ a continuity argument as in [43, Section 6.5]. We first choose an $N$-independent $\mathbf{x}(0)$ that satisfies (5.15). We then choose a continuous ($N$-dependent) path of the eigenvalues of $D^{a}$ and $D^{b}$, which gives a continuous path of the configurations $(\mathbf{x}(t):0\leq t\leq 1)$ that connects $\mathbf{x}(0)$ and $\mathbf{x}(1)=\mathbf{x}_{N}$. Correspondingly, we have a continuous path of eigenvalues $\{\widehat{\lambda}_{i}(t)\}_{i=1}^{N}$. We require that $\mathbf{x}(t)$ satisfies the following properties.

• (i)

For all $t\in[0,1]$, the eigenvalues of $D^{a}(t)$ and $D^{b}(t)$ are all non-negative.

• (ii)

For all $t\in[0,1]$, the number $r_{\epsilon}$ of $\epsilon$-spikes is unchanged and we denote them by $\widehat{\sigma}_{1}(t),\cdots,\widehat{\sigma}_{r_{\epsilon}}(t)$. Moreover, we always have the following order of the outliers: $x_{1}(t)\geq x_{2}(t)\geq\cdots\geq x_{r_{\epsilon}}(t)$.

• (iii)

For all $t\in[0,1]$, we denote the permissible intervals as $\mathrm{I}_{i}(t)$. If $\mathrm{I}_{i}(1)\cap\mathrm{I}_{j}(1)=\emptyset$ for $1\leq i<j\leq r_{\epsilon}$, then $\mathrm{I}_{i}(t)\cap\mathrm{I}_{j}(t)=\emptyset$ for all $t\in[0,1]$. The interval $\mathrm{I}_{0}$ in (5.10) is unchanged along the path.

It is easy to see that such a path $\mathbf{x}(t)$ exists. With a bootstrap argument along the path $\mathbf{x}(t)$, we can prove the following lemma and complete Step 3.

Lemma 5.25**.**

On the event $\Xi$, the estimate (5.7) holds for the configuration $\mathbf{x}(1)$.

Proof.

See [34, Lemma D.3]. ∎

Step 4: In this step, we consider the extremal non-outlier eigenvalues when $i\notin\left(\mathcal{O}_{\epsilon}^{(a)}\cup\mathcal{O}_{\epsilon}^{(b)}\right)$ and prove (5.8). The discussion will use the following eigenvalue interlacing result.

Lemma 5.26**.**

Recall the eigenvalues of $\widehat{\mathcal{Q}}_{1}$ and $\mathcal{Q}_{1}$ are denoted as $\{\widehat{\lambda}_{i}\}$ and $\{\lambda_{i}\},$ respectively. Then we have that

[TABLE]

where we adopt the convention that $\lambda_{i}=\infty$ if $i<1$ and $\lambda_{i}=0$ if $i>N.$

Proof.

See [34, Lemma C.3]. ∎

We first fix a configuration $\mathbf{x}(0)$ as mentioned earlier. Then by the discussion of Step 2, (5.3) and Lemma 5.26, we can prove (5.8) under the configuration $\mathbf{x}(0).$ For arbitrary $N$-dependent configuration, we again use a continuity argument as mentioned in Step 3. We refer the readers to Step 4 of the proof of Theorem 3.6 of [34]. This finishes the proof of Theorem 3.11.

5.2 Proof of Theorem 3.14

In this subsection, we prove the results for the bulk eigenvalues. According to Theorems 3.11 and 4.19 and Lemma 4.20, for any fixed $\epsilon>0,$ we can choose a high probability $\Xi_{1}$ in which (5.1)–(5.3) and the following estimates hold;

[TABLE]

for some integer $\varpi\geq r+s;$ and for $i\leq\tau N,$

[TABLE]

In what follows, we focus on our discussion on $\Xi_{1}$ and hence it will be purely deterministic. We follow the three steps discussed in Section 4.3 to conclude Theorem 3.14. The proof closely follows that of [34, Theorem 3.7], and we only focus on pointing out the differences due to similarity. In the first step, we find the permissible region for the eigenvalues and record the results in Lemma 5.27 below. For any $i$ and and $\gamma$ defined in (3.5), we define the set

[TABLE]

where $\mathrm{spec}(\mathcal{Q}_{1})$ stands for the spectrum of $\mathcal{Q}_{1}$.

Lemma 5.27**.**

For $\gamma\geq N^{-1/3+\epsilon}$ and $i\leq N^{1-2\epsilon}\gamma^{3},$ there exits a constant $c_{0}>0$ so that the set $\Omega_{i}$ contains no eigenvalue of $\widehat{\mathcal{Q}}_{1}.$

Proof.

The proof is similar to that of Lemma D.4 of [34] and we only sketch the key points here. Denote

[TABLE]

Using a discussion similar to (5.12), we find that $x$ is not an eigenvalues of $\widehat{\mathcal{Q}}_{1}$ if

[TABLE]

On one hand, using the definition of $\gamma,$ it is easy to see that for $x\in\Omega_{i}$ and some constant $C>0$

[TABLE]

On the other hand, using (2.8) and Lemma 4.18, we readily obtain that for $x\in\Omega_{i}$

[TABLE]

This completes the proof using (5.18). ∎

In the second step, we perform the counting argument for a special case as in Lemma 5.28 below.

Lemma 5.28**.**

We fix a configuration $\mathbf{x}\equiv\mathbf{x}(0):=(x_{1},x_{2},\cdots,x_{r+s})$ of the outliers that is independent of $N$ and satisfies

[TABLE]

For $\gamma\geq N^{-1/3+2\epsilon}$ and $i\leq N^{1-4\epsilon}\gamma^{3},$ we have that for some constant $C>0$

[TABLE]

Proof.

Since the proof is similar to the counterpart Lemma D.5 of [34], we only sketch it. The key idea is to group together the eigenvalues that are close to each other. Let $\mathcal{A}=\{A_{k}\}$ be the finest partition of $\{1,2,\cdots,N\}$ such that $i<j$ belong to the same block of $\mathcal{A}$ if

[TABLE]

Note that each block $A_{k}$ of $\mathcal{A}$ consists of a sequence of consecutive integers. Denote $k^{*}$ such that $N^{1-4\epsilon}\gamma^{3}\in A_{k^{*}}.$ Following a discussion similar to (D.47) and (D.48) of [34], we can conclude that

[TABLE]

and for any given $i_{k}\in A_{k}$,

[TABLE]

For any $1\leq k\leq k^{*},$ we denote

[TABLE]

With the above notations, we introduce the continuous path as

[TABLE]

Note that $x_{0}^{k}=a^{k}-\delta/3$ and $x_{1}^{k}=b^{k}+\delta/3$. The interval $[x_{0}^{k},x_{1}^{k}]$ contains precisely the eigenvalues of $\mathcal{Q}_{1}$ that are in $A_{k}$, and the endpoint $x_{0}^{k}$ (or $x_{1}^{k}$) is at a distance at least of the orders $\delta/3$ from any eigenvalue of $\mathcal{Q}_{1}$.

Moreover, on one hand, using a standard perturbation approach as in Proposition D.6 of [34], we can show that $\widehat{\mathcal{Q}}_{1}$ has at least $|A_{k}|$ eigenvalues in $[x_{0}^{k},x_{1}^{k}]$ for $1\leq k\leq k^{*}$. On the other hand, by (5.19), (5.20) and Lemma 5.26, we conclude that $\widehat{\mathcal{Q}}_{1}$ has at most $|A_{k}|$ eigenvalues in $[x_{0}^{k},x_{1}^{k}].$ This concludes the proof. ∎

In the third step, we generalize Lemma 5.28 using a continuity argument as in the proof of Theorem 3.11. We omit the details and conclude the proof of Theorem 3.14.

6 Proofs of Theorems 3.15 and 3.17: the eigenvector statistics

In this section, we prove the main results of Section 3.2 following the proof strategy outlined in Section 4.3. We again focus on explaining the main differences from the counterparts in [18, 34] and how to adapt their proof strategies. As mentioned before, the proof of Theorem 3.15 contains two parts. In Section 6.1 we prove the results under the assumption of (4.29) and then remove this assumption to complete the proof in Section 6.3.

6.1 Proof of Theorem 3.15 under (4.29)

In this subsection, we prove Theorem 3.15 assuming (4.29). Due to similarity, we only focus on the left singular vectors. The main technical task of this section is to prove Proposition 6.29, which implies the results. Recall the definitions in (5.9) and (3.8).

Proposition 6.29**.**

Suppose the assumptions of Theorem 3.15 and (4.29) hold. Then for all $i,j=1,2,\cdots,N,$ we have that

[TABLE]

where $(i\leftrightarrow j)$ denotes the same terms but with $i$ and $j$ interchanged.

Proof of Theorem 3.15 under (4.29).

For the left singular vectors, using that ${\mathcal{O}}^{+}$ is finite and

[TABLE]

the results simply follow from Proposition 6.29. The proof for the right singular vectors is analogous. ∎

The rest of the subsection is devoted to the proof of Proposition 6.29. Its proof consists of two steps. In the first step, we prove the results under the following non-overlapping condition, which guarantees a phenomenon of cone concentration.

Assumption 6.30**.**

For some fixed small constant $\tau_{2}>0,$ we assume that for all $\pi_{a}(i)\in S$ and $\pi_{b}(\mu)\in S$

[TABLE]

Let $\omega<\tau_{1}/2$ and $0<\epsilon<\min\{\tau_{1},\tau_{2}\}/10$ be some small positive constants to be chosen later. By Theorems 4.19 and 3.11 and Lemma 4.20, we can choose a high probability event $\Xi_{2}\equiv\Xi_{2}(\epsilon,\omega,\tau_{1},\tau_{2})$ where the following statements hold.

1. (i)

For all

[TABLE]

we have that

[TABLE]

where we recall the definition of $\Theta(z)$ in (4.12). 2. (ii)

Recall the notations in (5.9). For all $1\leq i\leq r^{+}$ and $1\leq\mu-N\leq s^{+}$, we have

[TABLE] 3. (iii)

For any fixed integer $\varpi>r+s$ and all $r^{+}+s^{+}<i\leq\varpi,$ we have that

[TABLE]

From now on, we will focus our discussion on the high probability event $\Xi_{2}$ and hence all the discussion will be purely deterministic.

Here we briefly pause to discuss consequences of the assumption (4.29). First of all, note that for any fixed $\epsilon>0$

[TABLE]

where we used Lemma 2.5 (v) and (vi) in the first inequality, Lemma 4.18 (4) in the second inequality, and Lemma 2.5 (iii) in the last inequality. Then, recalling (4.29) and $S\subset{\mathcal{O}}_{+}$, for any $i$ with $\pi_{a}(i)\in S$ we have

[TABLE]

where we took $\epsilon<\tau_{1}$ in (6.7). Hence $\widehat{a}_{i}$ is contained in the analytic domain of $\Omega_{B}^{-1}$ and in particular $\Omega_{B}^{-1}(\widehat{a}_{i})$ is well-defined. By the exact same reasoning we have $\widehat{b}_{\mu}>\Omega_{A}(\widehat{E}_{+})$ and $\Omega_{A}^{-1}(\widehat{b}_{\mu})$ is well-defined.

We next define a contour that will be used in the rest of the proof. Recall (3.9) and (3.10). Denote

[TABLE]

and

[TABLE]

for some sufficiently small constants $0<c_{i},c_{\mu}<1.$ Define the contour $\Gamma:=\partial\mathsf{C}$ as the boundary of the union of the open discs

[TABLE]

where $B_{\mathsf{r}}(x)$ denotes an open disc of radius $\mathsf{r}$ around $x.$ It is easy to see that the contour $\mathsf{C}$ is in the analytic domain of $\Omega_{B}^{-1}.$ In the following lemma, we will show that by choosing sufficiently small $c_{i},c_{\mu},$ we have that (1) $\overline{\Omega_{B}^{-1}(\mathsf{C})}$ is a subset of (6.3) and hence (6.4) holds; (2) $\partial\Omega_{B}^{-1}(\mathsf{C})=\Omega_{B}^{-1}(\Gamma)$ only encloses the outliers with indices in $S.$ Its proof is similar to [18, Lemmas 5.4 and 5.5] or [34, Lemma E.6] utilizing the results in Section 4.1. We omit the details here.

Lemma 6.31**.**

Suppose that the assumptions of Theorem 3.15 and (4.29) hold true. Then the set $\overline{\Omega_{B}^{-1}(\mathsf{C})}$ lies in the parameter set (6.3) as long as $c_{i}$’s and $c_{\mu}$’s are sufficiently small. Moreover, we have that $\{\widehat{\lambda}_{\mathfrak{a}}\}_{\mathfrak{a}\in S}\subset\Omega_{B}^{-1}(\mathsf{C})$ and all the other eigenvalues lie in the complement of $\overline{\Omega_{B}^{-1}(\mathsf{C})}.$

Then we prove Proposition 6.29. The proof follows from the same strategy as [34, Proposition E.5].

Proof of Proposition 6.29.

The proof consists of two steps. In the first step, we prove the results under Assumption 6.30. In the second step, we prove the results by removing Assumption 6.30 from the first step. Since the second step is rather standard, we focus on the first step and only briefly discuss the second step.

Step 1: In the first step, we prove the results assuming that the non-overlapping condition Assumption 6.30 holds. For convenience, we set $d_{i}^{a}=0$ when $i>r$ and deal with the general case at the end of this step. Denote

[TABLE]

Using the resolvent expansion, we obtain that

[TABLE]

Together with (4.25) and (4.3), using the fact that $\Gamma$ does not enclose any pole of $\mathbf{G}$ (due to (6.6)), we have the following decomposition

[TABLE]

where $s_{0},s_{1}$ and $s_{2}$ are defined as

[TABLE]

[TABLE]

[TABLE]

First, we we deal with the term containing $s_{0}.$ Using Cauchy’s integral formula, we readily see that

[TABLE]

Second, we control the term containing $s_{1}.$ We use the change of variables $\zeta=\Omega_{B}(z)$ to obtain that

[TABLE]

To bound $\xi_{ij}(\zeta),$ we first prepare some useful estimates. When $c_{i}$’s and $c_{\mu}$’s are sufficiently small, by a discussion similar to (5.5), we have that for $\zeta\in\Gamma,$

[TABLE]

Moreover, for any $z_{b}\in\Omega_{B}^{-1}(\Gamma)$, by Cauchy’s differentiation formula we obtain that

[TABLE]

where $\mathcal{C}_{b}$ is the disc of radius $|z_{b}-E_{+}|/2$ centered at $z_{b}.$ Here we used the facts that both $\Omega_{B}$ and $\Omega_{\beta}$ are holomorphic on $\Omega_{B}^{-1}(\Gamma).$ Together with (4.6), using the residue theorem, we readily see that for some constant $C>0$

[TABLE]

where in the last step we used (6.12) such that $|z_{b}-E_{+}|\geq CN^{-2/3+\epsilon}.$ Consequently, using implicit differentiation, we conclude that

[TABLE]

where in the last step we used (6.12). With the above preparation, we now proceed to control $\xi_{ij}(\zeta)$ and $s_{1}.$ By (6.4), (6.12) and (6.14), it is easy to see that for $\zeta\in\Gamma$

[TABLE]

Together with a discussion similar to (6.13), we see that

[TABLE]

In order to control $s_{1},$ we will consider different cases. In the first case when both $\pi_{a}(i)\in S$ and $\pi_{b}(j)\in S,$ if $\widehat{a}_{i}\neq\widehat{a}_{j},$ by (6.11) and the residue theorem, we have that

[TABLE]

where we used (6.16) in the last step. The same argument applies to the case $\widehat{a}_{i}=\widehat{a}_{j}$. In the second case when $\pi_{a}(i)\in S$ and $\pi_{a}(j)\notin S,$ we conclude from (6.15) that

[TABLE]

Similarly, we can estimate $s_{1}$ when $\pi_{a}(i)\notin S$ and $\pi_{b}(j)\in S.$ Finally, when both $\pi_{a}(i)\notin S$ and $\pi_{b}(j)\notin S,$ we have $s_{1}=0$ by the residue theorem. This completes the estimation regarding $s_{1}.$

We then estimate $s_{2},$ which relies on some crucial estimates on the contour. We decompose $\Gamma$ as

[TABLE]

The following lemma is the key input for $s_{2}.$ Its proof is similar to Lemma E.7 of [34] and omitted here.

Lemma 6.32**.**

For any $\pi_{a}(i)\in S,1\leq j\leq r,1\leq\nu-N\leq s$ and $\zeta\in\partial B_{\rho_{i}^{a}}(\widehat{a}_{i}),$ we have that

[TABLE]

and

[TABLE]

For any $\pi_{b}(\mu)\in S,1\leq j\leq r,1\leq\mu-N\leq s$ and $\zeta\in\partial B_{\rho_{\mu}^{b}}(\Omega_{B}(\Omega_{A}^{-1}(\widehat{b}_{\mu}))),$ we have

[TABLE]

and

[TABLE]

Then $s_{2}$ is estimated as follows, using (6.4), (6.15), (6.12), and (i) of Proposition 4.18.

[TABLE]

where $\mathfrak{d}(\zeta)$ is defined as

[TABLE]

Moreover, by (6.4) and (6.12), for some constant $C>0,$ we can bound

[TABLE]

Recall that both $r$ and $s$ are bounded. Together with Lemma 6.32 and the fact $\epsilon<\tau_{2}$, we obtain that

[TABLE]

where we used (6.20) and Assumption 6.30. Based on the above estimates, we arrive at

[TABLE]

Now we proceed to control $s_{2}.$ Decomposing the integral contour in (6.1) as in (6.17), using (6.21) and Lemma 6.32, and recalling that the length of $\Gamma_{i}$ (or $\Gamma_{\mu}$) is at most $2\pi\rho_{i}^{a}$ (or $2\pi\rho_{\mu}^{b}$), we get that for some constant $C>0,$

[TABLE]

Now we bound the right-hand side of (6.22) using Cauchy-Schwarz inequality. For $\pi_{a}(i)\notin S$, we have

[TABLE]

For $\pi_{a}(i)\in S$, we have $\rho_{k}^{a}+\delta^{a}_{\pi_{a}(k),\pi_{a}(i)}\gtrsim\rho_{i}^{a}$ for $\pi_{a}(k)\in S$, and $\rho_{\mu}^{b}+\delta^{b}_{\beta(\mu),\pi_{a}(i)}\gtrsim\rho_{i}^{a}$ for $\pi_{b}(\mu)\in S$. Then we have for some constant $C>0$

[TABLE]

Plugging the above two estimates into (6.22), we get that

[TABLE]

So far, we have proved Proposition 6.29 for $1\leq i,j\leq r$ since $\epsilon$ can be arbitrarily small.

Finally, the general case can be dealt with easily. For general $i,j\in\{1,\cdots,N\}$, we define $\mathcal{R}:=\{1,\cdots,r\}\cup\{i,j\}$. Then we define a perturbed model as

[TABLE]

where for some $\widetilde{\epsilon}>0,$

[TABLE]

Then all the previous proof goes through for the perturbed model as long as we replace the $\mathbf{U}$ and $\bm{\mathcal{D}}$ in (4.21) with

[TABLE]

Note that in the proof, only the upper bound on the $d_{k}^{a}$’s were used. Moreover, the proof does not depend on the fact that $\widehat{a}_{i}$ or $\widehat{a}_{j}$ satisfy (2.10) (we only need the indices in $S$ to satisfy Assumption 6.30). By taking $\widetilde{\epsilon}\downarrow 0$ and using continuity, we get that Proposition 6.29 holds for general $i,j\in\{1,\cdots,N\}$.

Step 2: In the second step, we complete our proof by removing Assumption 6.30. In fact, once we finish the proof of Step one, the second step is relatively standard and follows the same argument as in [34, Section E.2] and [18, Section 5.2]. The main idea behind is to split the discussion into several cases by considering two sets related to $S$; see Definition E.8 of [34]. We omit the details since our proof follows verbatim as [34, Section E.2]. This completes our proof.

∎

6.2 Proof of Theorem 3.17

In this subsection, we prove the results for the non-outlier eigenvectors. Our goal is to prove the following proposition, from which Theorem 3.17 immediately follows.

Proposition 6.33**.**

Fix a small constant $\widetilde{\tau}\in(0,1/3).$ For $\pi_{a}(i)\notin\mathcal{O}^{+}$ and $i\leq\tau N,$ where $\tau>0$ is given in Theorem 3.17, we have

[TABLE]

Moreover, if $\pi_{a}(i)\in\mathcal{O}^{+}$ satisfies

[TABLE]

we have that

[TABLE]

Similar results hold for the right singular vectors.

The rest of the subsection is devoted to the proof of Proposition 6.33. Its proof is similar to Proposition F.1 of [34]. We focus on explaining the main differences following the strategy outlined in Section 4.3.

Proof of Proposition 6.33.

Due to similarity, we only prove the first statement. As mentioned earlier, the control relies on the deterministic inequality (4.30). By Theorems 4.19, 3.11, 3.15, 3.14 and Lemmas 4.20 and 4.21, for any fixed $\epsilon>0,$ we can choose a high-probability event $\Xi_{3}$ where (5.1)–(5.3), (5.16)–(5.17) and the following hold:

[TABLE]

where we used the interlacing result Lemma 5.26.

In what follows, we again focus on $\Xi_{2}$ and the discussion will be purely deterministic. Recall $\bm{e}_{i}$ is the natural embedding of $\mathbf{e}_{i}$ in $\mathbb{R}^{2N}.$ As discussed in (6.23), for $1\leq j\leq\tau N,$ we may define a large set $\{1,2,3,\cdots,r\}\cup\{j\}$ to handle the general case. For simplicity, we assume that $1\leq j\leq r$ to simplify our notations. Let $\eta_{i}>0$ be the unique solution of

[TABLE]

which follows from the facts that $N^{-1+\epsilon}\operatorname{\mathrm{Im}}\Omega_{B}(\widehat{\lambda}_{i}+\mathrm{i}N^{-1+\epsilon})\lesssim N^{-1+\epsilon}$ due to Lemma 4.18 and the mapping $\eta\mapsto\eta\operatorname{\mathrm{Im}}\Omega_{B}(\widehat{\lambda}_{i}+\mathrm{i}\eta)$ is monotone increasing (see [32, Lemma A.8]). According to (4.30) and (4.24), fixing an $\pi_{a}(i)\notin\mathcal{O}^{+},$ we can write

[TABLE]

Recall (6.9) and (6.1). By (4.3), using the resolvent expansion (6.1), we find that

[TABLE]

and used the abbreviated notation that

[TABLE]

Similar to the discussion between (F.7) and (F.15) of [34], by (5.1), (2.8), Lemma 4.18 and the definition (6.25), we find that

[TABLE]

and consequently by (6.27), we have

[TABLE]

In order to bound the right-hand side of the above equation, we will need the following estimate. Its proof is the same as (6.10) of [18] or Lemma F.2 of [34] and we omit the details.

Lemma 6.34**.**

For any fixed $\delta\in[0,1/3-\epsilon),$ there exists a constant $c>0$ such that

[TABLE]

holds whenever $\widehat{\lambda}_{i}\in[0,\Omega_{B}^{-1}(\Omega_{\beta}(E_{+})+N^{-1/3+\delta+\epsilon})).$

By (6.26), (6.28), and fixing a suitable $\delta>0$ in Lemma 6.34, we find that

[TABLE]

It only remains to control the terms on the right-hand side of (6.29) one by one. For the first term, we find that

[TABLE]

where we used the definition of $\eta_{i}$ in (6.25), (2.8) and (4) of Lemma 4.18. The other two terms can be analyzed similarly and we refer the readers to Section F of [34]. For example, for the second term on the right-hand side of (6.29), we have that on the probability event $\Xi_{3}$

[TABLE]

According to Lemma 6.34, we find that $|\widehat{a}_{j}-\Omega_{B}(z_{i})|^{-2}=\mathrm{O}(|\widehat{a}_{j}-\Omega_{\beta}(E_{+})|^{-2})$ by setting $\delta=0.$ Moreover, by (6.25), we find that $\operatorname{Im}\Omega_{B}(z_{i})=N^{6\epsilon}(\eta_{i}N)^{-1}.$ Consequently, we see that for some constant $C>0,$

[TABLE]

Using these estimates and (6.29), we immediately arrive at

[TABLE]

Finally, to conclude the proof, we need to provide a lower bound for the denominator of the second term of the right-hand side of (6.30). The lower bound directly follows from Lemma 6.34 by choosing $\delta=0$. This concludes our proof. ∎

6.3 Proof of Theorem 3.15

In this subsection, we complete the proof of Theorem 3.15 by removing (4.29) from the proof in Section 6.1 using the estimate (6.24). The proof is similar to the discussion in Section F of [34] and we only provide the main points.

As discussed earlier, it suffices to prove Proposition 6.29 without imposing (4.29). Fix a constant $\epsilon>0,$ then it is easy to check that there exists some $x_{0}\in[1,r+s+1]$ so that there is no $\widehat{a}_{k}$ between $\Omega_{\beta}(E_{+})+x_{0}N^{-1/3+\epsilon}$ and $\Omega_{\beta}(E_{+})+(x_{0}+1)N^{-1/3+\epsilon}.$ Thus, following the ideas of [18, Section 6.2] or [34, Section F], we may split $S=S_{0}\cup S_{1}$ so that $\widehat{a}_{k}\leq\Omega_{\beta}(E_{+})+x_{0}N^{-1/3+\epsilon}$ for $\pi_{a}(k)\in S_{0},$ and $\widehat{a}_{k}>\Omega_{\beta}(E_{+})+(x_{0}+1)N^{-1/3+\epsilon}$ for $\pi_{a}(k)\in S_{1}.$ Without loss of generality, we assume that $S_{0}\neq\emptyset$, otherwise the proof is complete.

Based on the above discussion, the actual proof is divided into six cases according to which of the sets $S_{0},$ $S_{1}$ or $S^{c}$ the indices $\pi_{a}(i)$ and $\pi_{a}(j)$ belong to. Due to similarity, we restrict our attention to few specific cases. Other cases can be handled in a similar fashion. The proof relies on the decomposition

[TABLE]

When $\pi_{a}(i),\pi_{a}(j)\in S_{0},$ applying the Cauchy-Schwarz inequality and the estimate (6.24) to the first term of the right-hand side of (6.31), and Proposition 6.29 to the second term of it, we obtain that

[TABLE]

where in the second step we used the fact that $\Delta_{1}(\widehat{a}_{i(j)})^{2}=\mathrm{O}(N^{-1/3+\epsilon})=\mathrm{O}(\delta_{\pi_{a}(i(j))}(S_{1})).$ Similarly, we can obtain the results when $\pi_{a}(i),\pi_{a}(j)\in S_{1}$ using Propositions 6.33 and 6.29. Moreover, when $\pi_{a}(i)\in S_{0},\pi_{a}(j)\in S_{1},$ the proof can be divided into two steps. In the first step, we prove the results under Assumption 6.30 for some constant $0<\tau_{2}<\epsilon$. In the second step we remove this assumption as in the proof of Proposition 6.29. We only highlight the first step. Applying the Cauchy-Schwarz inequality and Proposition 6.33 to the first term of the RHS of (6.31) , and applying Proposition 6.29 to the second term, we get that

[TABLE]

where we used the facts that $\Delta_{1}(\widehat{a}_{i})^{2}=\mathrm{O}(\delta_{\pi_{a}(i)}(S_{1}))=\mathrm{O}(\Delta_{1}(\widehat{a}_{j})^{2})=\mathrm{O}(\delta^{a}_{\pi_{a}(i),\pi_{a}(j)})$ and $\delta_{\pi_{a}(j)}(S)\wedge|\widehat{a}_{i}-\Omega_{\beta}(E_{+})|=\mathrm{O}(\delta_{\pi_{a}(j)}(S_{1})).$ The other cases can be discussed similarly and we refer the readers to Section F of [34] for more details. This concludes our proof of Theorem 3.15.

Acknowledgment

The authors would like to thank the editor, the associated editor and two anonymous referees for their many critical suggestions which have significantly improved the paper. The authors are also grateful to Zhigang Bao and Ji Oon Lee for many helpful discussions. The first author also wants to thank Hari Bercovici for many useful comments. The first author is partially supported by NSF DMS-2113489 and the second author is supported by ERC Advanced Grant ”RMTBeyond” No. 101020331.