Networks of reinforced stochastic processes: asymptotics for the empirical means

TL;DR

This paper investigates the long-term behavior of empirical means in networks of interacting reinforced stochastic processes, establishing almost sure synchronization and central limit theorems, with applications to statistical inference on the network structure.

Contribution

It provides the first analysis of the asymptotic behavior of empirical means in such networks, including convergence results and statistical inference methods.

Findings

Almost sure synchronization of empirical means

Central limit theorems for empirical means

Statistical tools for inference on network structure

Abstract

This work deals with systems of interacting reinforced stochastic processes, where each process $X^j=(X_{n,j})_n$ is located at a vertex $j$ of a finite weighted direct graph, and it can be interpreted as the sequence of "actions" adopted by an agent $j$ of the network. The interaction among the evolving dynamics of these processes depends on the weighted adjacency matrix $W$ associated to the underlying graph: indeed, the probability that an agent $j$ chooses a certain action depends on its personal "inclination" $Z_{n,j}$ and on the inclinations $Z_{n,h}$, with $h\neq j$, of the other agents according to the elements of $W$. Asymptotic results for the stochastic processes of the personal inclinations $Z^j=(Z_{n,j})_n$ have been subject of studies in recent papers (e.g. Aletti, Crimaldi, and Ghiglietti [arXiv:1607.08514, Ann. Appl. Probab., 27(6):3787-3844, 2017]; Crimaldi, Dai Pra, Louis, and Minelli [arXiv:1602.06217, Forthcoming in Stochastic Process. Appl.]); while the asymptotic behavior of quantities based on the stochastic processes $X^j$ of the actions has never been studied yet. In this paper, we fill this gap by characterizing the asymptotic behavior of the empirical means $N_{n,j}=\sum_{k=1}^n X_{k,j}/n$, proving their almost sure synchronization and some central limit theorems in the sense of stable convergence. Moreover, we discuss some statistical applications of these convergence results concerning confidence intervals for the random limit toward which all the processes of the system converge and tools to make inference on the matrix $W$.

Equations

$$P(X_{n+1,j}=1|{\mathcal{F}}_n)=\sum _{h=1}^N{w}_{h,j}{Z}_{n,h},$$

$$Z_{n,h}=(1-r_{n-1})Z_{n-1,h}+r_{n-1}{X}_{n,h},$$

$$\underset{n}{\lim }{n}^{\gamma }{r}_n=c>0with\; 1/2<\gamma \le 1.$$

$$N_{n,j}:=\frac{1}{n}\sum _{k=1}^n{X}_{k,j}.$$

$$N_{n,j}=\left(1-\frac{1}{n}\right)N_{n-1,j}+\frac{1}{n}{X}_{n,j}.$$

$$E[{\mathbf{X}}_{n+1}|{\mathcal{F}}_n]=W^{\top }{\mathbf{Z}}_n,$$

$$\{\begin{array}{rl}& {\mathbf{Z}}_n=\left(1-r_{n-1}\right){\mathbf{Z}}_{n-1}+r_{n-1}{\mathbf{X}}_n,\\ & {\mathbf{N}}_n=\left(1-\frac{1}{n}\right){\mathbf{N}}_{n-1}+\frac{1}{n}{\mathbf{X}}_n.\end{array}$$

$$\{\begin{array}{rl}& {\mathbf{Z}}_n-{\mathbf{Z}}_{n-1}=r_{n-1}\left({\mathbf{X}}_n-{\mathbf{Z}}_{n-1}\right),\\ & {\mathbf{N}}_n-{\mathbf{N}}_{n-1}=\frac{1}{n}\left({\mathbf{X}}_n-{\mathbf{N}}_{n-1}\right),\end{array}$$

$$n^{\gamma }{r}_n=c+o(1).$$

$$n{r}_n=c+O\left(n^{-1}\right).$$

$${\mathbf{u}}_j^{\top }{\mathbf{v}}_j=1,\text{mbox}and{\mathbf{u}}_h^{\top }{\mathbf{v}}_j=0,\forall h\ne j.$$

$${\mathbf{u}}_1=N^{-1/2}1,N^{-1/2}{1}^{\top }{\mathbf{v}}_1=1\text{mbox}and{v}_{1,j}:=[{\mathbf{v}}_1{]}_j>0\forall 1\le j\le N.$$

$$\mathcal{R}e({\lambda }^{\ast })=\max \left\{\mathcal{R}e({\lambda }_j):{\lambda }_j\in Sp(W)\setminus \{1\}\right\}.$$

$${\mathbf{N}}_n\stackrel{a.s.}{⟶}{Z}_{\infty }1,$$

$$n^{\gamma -\frac{1}{2}}\left(\begin{array}{c}{\mathbf{Z}}_n-Z_{\infty }1\\ {\mathbf{N}}_n-Z_{\infty }1\end{array}\right)⟶\mathcal{N}\left(0,Z_{\infty }(1-Z_{\infty })\left(\begin{array}{cc}{\tilde{\Sigma }}_{\gamma }& {\tilde{\Sigma }}_{\gamma }\\ {\tilde{\Sigma }}_{\gamma }& {\tilde{\Sigma }}_{\gamma }+{\hat{\Gamma }}_{\gamma }\end{array}\right)\right)stably,$$

$${\tilde{\Sigma }}_{\gamma }:={\tilde{\sigma }}_{\gamma }^{2}11^{\top }\text{mbox}and{\tilde{\sigma }}_{\gamma }^2:=\frac{c^2\Vert {\mathbf{v}}_1{\Vert }^2}{N(2\gamma -1)}>0,$$

$${\hat{\Gamma }}_{\gamma }:={\hat{\sigma }}_{\gamma }^{2}11^{\top }\text{mbox}and{\hat{\sigma }}_{\gamma }^2:=\frac{c^2\Vert {\mathbf{v}}_1{\Vert }^2}{N(3-2\gamma )}>0.$$

$$1\le 1+\Vert {\mathbf{v}}_1-{\mathbf{u}}_1{\Vert }^2=\Vert {\mathbf{v}}_1{\Vert }^2\le N,$$

$$\frac{c^2}{N(2\gamma -1)}\le {\tilde{\sigma }}_{\gamma }^2\le \frac{c^2}{(2\gamma -1)}and\frac{c^2}{N(3-2\gamma )}\le {\hat{\sigma }}_{\gamma }^2\le \frac{c^2}{(3-2\gamma )}.$$

$$\sqrt{n}\left(\begin{array}{c}{Z}_n-Z_{\infty }\\ N_n-Z_{\infty }\end{array}\right)⟶\mathcal{N}\left(0,Z_{\infty }(1-Z_{\infty })\left(\begin{array}{cc}{c}^2& c^2\\ c^2& c^2+(c-1{)}^2\end{array}\right)\right)stably.$$

$$\sqrt{n}\left(\begin{array}{c}{\mathbf{Z}}_n-Z_{\infty }1\\ {\mathbf{N}}_n-Z_{\infty }1\end{array}\right)⟶\mathcal{N}\left(0,Z_{\infty }(1-Z_{\infty })\left(\begin{array}{cc}{\tilde{\Sigma }}_1+{\hat{\Sigma }}_{\mathbf{ZZ}}& {\tilde{\Sigma }}_1+{\hat{\Sigma }}_{\mathbf{ZN}}\\ {\tilde{\Sigma }}_1+{\hat{\Sigma }}_{\mathbf{ZN}}^{\top }& {\tilde{\Sigma }}_1+{\hat{\Sigma }}_{\mathbf{NN}}\end{array}\right)\right)stably,$$

$${\hat{\Sigma }}_{\mathbf{ZZ}}:=U{\hat{S}}_{\mathbf{ZZ}}{U}^{\top },\text{mbox}with$$

$$[{\hat{S}}_{\mathbf{ZZ}}{]}_{h,j}:=\frac{c^2}{c(2-{\lambda }_h-{\lambda }_j)-1}({\mathbf{v}}_h^{\top }{\mathbf{v}}_j),2\le h,j\le N;$$

$${\hat{\Sigma }}_{\mathbf{NN}}:=\tilde{U}{\hat{S}}_{\mathbf{NN}}{\tilde{U}}^{\top },\text{mbox}with$$

$$[{\hat{S}}_{\mathbf{NN}}{]}_{1,1}:=(c-1{)}^2\Vert {\mathbf{v}}_1{\Vert }^2,[{\hat{S}}_{\mathbf{NN}}{]}_{1,j}=[{\hat{S}}_{\mathbf{NN}}{]}_{j,1}:=\left(\frac{1-c}{1-{\lambda }_j}\right)({\mathbf{v}}_1^{\top }{\mathbf{v}}_j),2\le j\le N,$$

$$[{\hat{S}}_{\mathbf{NN}}{]}_{h,j}:=\frac{1+(c-1)[(1-{\lambda }_h{)}^{-1}+(1-{\lambda }_j{)}^{-1}]}{c(2-{\lambda }_h-{\lambda }_j)-1}({\mathbf{v}}_h^{\top }{\mathbf{v}}_j),2\le h,j\le N;$$

$${\hat{\Sigma }}_{\mathbf{ZN}}:=U{\hat{S}}_{\mathbf{ZN}}{\tilde{U}}^{\top },\text{mbox}with$$

$$[{\hat{S}}_{\mathbf{ZN}}{]}_{h,1}:=\left(\frac{1-c}{1-{\lambda }_h}\right)({\mathbf{v}}_h^{\top }{\mathbf{v}}_1),2\le h\le N,$$

$$[{\hat{S}}_{\mathbf{ZN}}{]}_{h,j}:=\frac{c+(c-1)(1-{\lambda }_h{)}^{-1}}{c(2-{\lambda }_h-{\lambda }_j)-1}({\mathbf{v}}_h^{\top }{\mathbf{v}}_j),2\le h,j\le N.$$

$$\sqrt{\frac{n}{\ln (n)}}\left(\begin{array}{c}{\mathbf{Z}}_n-Z_{\infty }1\\ {\mathbf{N}}_n-Z_{\infty }1\end{array}\right)⟶\mathcal{N}\left(0,Z_{\infty }(1-Z_{\infty })\left(\begin{array}{cc}{\hat{\Sigma }}_{\mathbf{ZZ}}^{\ast }& {\hat{\Sigma }}_{\mathbf{ZN}}^{\ast }\\ {\hat{\Sigma }}_{\mathbf{ZN}}^{\ast \top }& {\hat{\Sigma }}_{\mathbf{NN}}^{\ast }\end{array}\right)\right)stably,$$

Full text

Asymptotic results for the stochastic processes of the personal inclinations $Z^{j}=(Z_{n,j})_{n}$ have been subject of studies in recent papers (e.g. [2, 21]); while the asymptotic behavior of quantities based on the stochastic processes $X^{j}$ of the actions has never been studied yet. In this paper, we fill this gap by characterizing the asymptotic behavior of the empirical means $N_{n,j}=\sum_{k=1}^{n}X_{k,j}/n$, proving their almost sure synchronization and some central limit theorems in the sense of stable convergence. Moreover, we discuss some statistical applications of these convergence results concerning confidence intervals for the random limit toward which all the processes of the system converge and tools to make inference on the matrix $W$.

Keywords:

Interacting Systems; Reinforced Stochastic Processes; Urn Models; Complex Networks; Synchronization; Asymptotic Normality.

2010 AMS classification: 60F05, 60F15, 60K35; 62P35, 91D30.

1. Framework, model and main ideas

Real-world systems often consist of interacting agents that may develop a collective behavior (e.g. [1, 9, 39, 44]): in neuroscience the brain is an active network where billions of neurons interact in various ways in the cellular circuits; many studies in biology focus on the interactions between different sub-systems; social sciences and economics deal with individuals that take decisions under the influence of other individuals, and also in engineering and computer science “consensus problems”, understood as the ability of interacting dynamic agents to reach a common asymptotic stable state, play a crucial role. In all these frameworks, an usual phenomenon is the synchronization, that could be roughly defined as the tendency of different interacting agents to adopt a common behavior. Taking into account various features of these systems, several research works employed agent-based models in order to analyze how macro-level collective behaviors arise as products of the micro-level processes of interaction among the agents of the system (we refer to [8] for a detailed and well structured survey on this topic, rich of examples and references). The main goals of these researches are twofold: (i) to understand whether and when a (complete or partial) synchronization in a dynamical system of interacting agents can emerge and (ii) to analyze the interplay between the network topology of the interactions among the agents and the dynamics followed by the agents.

This work is placed in the stream of scientific literature that studies systems of interacting urn models (e.g. [3, 10, 14, 16, 22, 25, 32, 33, 34, 37, 40, 43]) and their variants and generalizations (e.g. [2, 21]). Specifically, our work deals with the class of the so-called interacting reinforced stochastic processes considered in [2, 21]. Generally speaking, by reinforcement in a stochastic dynamics we mean any mechanism for which the probability that a given event occurs has an increasing dependence on the number of times that events of the same type occurred in the past. This “self-reinforcing property”, also known as “preferential attachment rule”, is a key feature governing the dynamics of many biological, economic and social systems (see, e.g. [42]). The best known example of reinforced stochastic process is the standard Pòlya’s urn [26, 36], which has been widely studied and generalized (some recent variants can be found in [4, 5, 7, 12, 13, 15, 17, 19, 27, 28, 31]).

We consider a system of $N\geq 1$ interacting reinforced stochastic processes $\{X^{j}=(X_{n,j})_{n\geq 1}:\,1\leq j\leq N\}$ positioned at the vertices of a weighted directed graph $G=(V,\,E,\,W)$, where $V:=\{1,...,N\}$ denotes the set of vertices, $E\underline{\subset}V\times V$ the set of edges and $W=[w_{h,j}]_{h,j\in V\times V}$ the weighted adjacency matrix with $w_{h,j}\geq 0$ for each pair of vertices. The presence of the edge $(h,j)\in E$ indicates a “direct influence” that the vertex $h$ has on the vertex $j$ and it corresponds to a strictly positive element $w_{h,j}$ of $W$ that represents a weight quantifying this influence. We assume the weights to be normalized so that $\sum_{h=1}^{N}w_{h,j}=1$ for each $j\in V$. For any $n\geq 1$, we assume the random variables $\{X_{n,j}:\,j\in V\}$ to take values in $\{0,1\}$ and hence they can be interpreted as “two-modality actions” that the agents of the network can adopt at time $n$. Formally, the interaction between the processes $\{X^{j}:\,j\in V\}$ is modeled as follows: for any $n\geq 0$, the random variables $\{X_{n+1,j}:\,j\in V\}$ are conditionally independent given ${\mathcal{F}}_{n}$ with

[TABLE]

and, for each $h\in V$,

[TABLE]

where $Z_{0,h}$ are random variables with values in $[0,1]$, ${\mathcal{F}}_{n}:=\sigma(Z_{0,h}:h\in V)\vee\sigma(X_{k,j}:\,1\leq k\leq n,\,j\in V)$ and $0\leq r_{n}<1$ are real numbers such that

[TABLE]

(We refer to [21] for a discussion on the case $0<\gamma\leq 1/2$, for which we have a different asymptotic behavior of the model that is out of the scope of this research work.) For example, if at each vertex $j\in V$ we have a standard Pólya’s urn, with initial composition given by the pair $(a,b)$, then we have $r_{n}=(a+b+n+1)^{-1}$ and so $\gamma=c=1$. Each random variable $Z_{n,h}$ takes values in $[0,1]$ and it can be interpreted as the “personal inclination” of the agent $h$ of adopting “action 1”, so that the probability that the agent $j$ adopts “action 1” at time $(n+1)$ depends on its personal inclination $Z_{n,j}$ and on the inclinations $Z_{n,h}$, with $h\neq j$, of the other agents at time $n$ according to the “influence-weights” $w_{h,j}$.

The previous quoted papers [2, 21, 22, 25] are all focused on the asymptotic behavior of the stochastic processes of the “personal inclinations” $\{Z^{j}=(Z_{n,j})_{n}:\,j\in V\}$ of the agents. On the contrary, in this work we focus on the average of times in which the agents adopt “action 1”, i.e. we study the stochastic processes of the empirical means $\{N^{j}=(N_{n,j})_{n}:\,j\in V\}$ defined, for each $j\in V$, as $N^{j}_{0}:=0$ and, for any $n\geq 1$,

[TABLE]

Since $(1/n)\sum_{k=1}^{n-1}X_{k,j}=(1-1/n)N_{n-1,j}$, the dynamics of each process $N^{j}$ can be written as follows:

[TABLE]

Furthermore, the above dynamics (1), (2) and (5) can be expressed in a compact form, using the random vectors $\mathbf{X}_{n}:=(X_{n,1},\dots,X_{n,N})^{\top}$ for $n\geq 1$, $\mathbf{N}_{n}:=(N_{n,1},\dots,N_{n,N})^{\top}$ and $\mathbf{Z}_{n}:=(Z_{n,1},\dots,Z_{n,N})^{\top}$ for $n\geq 0$, as:

[TABLE]

where $W^{\top}\mathbf{1}=\mathbf{1}$ by the normalization of the weights, and

[TABLE]

In the framework described above, under suitable assumptions, we prove that all the stochastic processes $N^{j}=(N_{n,j})_{n}$, with $j\in V$, converge almost surely to the same limit random variable (in other words, we prove their almost sure synchronization), which is also the common limit random variable of the stochastic processes $Z^{j}=(Z_{n,j})_{n}$, say $Z_{\infty}$ (see Theorem 3.1). From an applicative point of view, the almost sure synchronization of the stochastic processes $N^{j}$ means that, with probability $1$, the percentages of times that the agents of the system adopt the “action 1” tend to the same random value $Z_{\infty}$. Moreover, we provide some Central Limit Theorems (CLTs) in the sense of stable convergence, in which the asymptotic variances and covariances are expressed as functions of the eigen-structure of the weighted adjacency matrix $W$ and of the parameters $\gamma,\,c$ governing the asymptotic behavior of the sequence $(r_{n})_{n}$ (see Theorem 3.2, Theorem 3.3, Theorem 3.4 and Theorem 3.5). These convergence results are also discussed from the point of view of the statistical applications. In particular, they lead to the construction of asymptotic confidence intervals for the common limit random variable $Z_{\infty}$ based on the random variables $X_{n,j}$ through the empirical means (4), that specifically require neither the knowledge of the initial random variables $\{Z_{0,j}:\,j\in V\}$ nor of the exact expression of the sequence $(r_{n})_{n}$. For the case $\gamma=1$, that for instance includes the case of interacting standard Pólya’s urns, we also provide a statistical test, based on the random variables $X_{n,j}$ through the empirical means (4), to make inference on the weighted adjacency matrix $W$ of the network. The fact that the confidence intervals and the inferential procedures presented in this work are based on $X_{n,j}$, instead of $Z_{n,j}$ as done in [2], represents a great improvement in any area of application, since the “actions” $X_{n,j}$ adopted by the agents of the network are much more likely to be observed than their “personal inclinations” $Z_{n,j}$ of adopting these actions.

The proofs of the given CLTs are a substantial part of this work and we believe that it is worth spending some words on the main tools employed and technical issues faced. The essential idea is to decompose the stochastic process $(\mathbf{N}_{n})_{n}$ into the sum of two terms, where the first one converges, at the rate $n^{\gamma-1/2}$ for each $1/2<\gamma\leq 1$, stably in the strong sense with respect to the filtration $(\mathcal{F}_{n})_{n}$ toward a certain Gaussian kernel, and the second term is an $(\mathcal{F}_{n})$-adapted stochastic process that converges stably to a suitable Gaussian kernel, with the corresponding rate and argument required for the proof different according to the value of $\gamma$. Indeed, when $1/2<\gamma<1$, the second term converges stably at the same rate as above, i.e. $n^{\gamma-1/2}$, and in the proof we have a certain remainder term that tends to zero in probability (see Theorem 4.2). On the contrary, when $\gamma=1$ and $N\geq 2$ (the case $\gamma=1,\,N=1$ is similar to the previous case $1/2<\gamma<1$), we do not have the convergence to zero of that remainder term (see Remark 4.3) and so we develop a coupling technique based on the pair of random vectors $(\mathbf{Z}_{n},\mathbf{N}_{n})$. So doing, we determine two different rates for the convergence of the second term, depending on the second highest real part ${\mathcal{R}e}(\lambda^{*})$ of the eigenvalues of $W$ (see Theorem 4.3 where the rate is $\sqrt{n}$ and Theorem 4.4 where the rate is $\sqrt{n/\ln(n)}$). The contributions of the two terms are in particular reflected in the analytic expressions of the asymptotic covariance matrix of $\mathbf{N}_{n}$ (see Theorem 3.2, Theorem 3.4 and Theorem 3.5), where there is a component $\widetilde{\Sigma}_{\gamma}$ due to the first term (which is zero when the rate for the second term is $\sqrt{n/\ln(n)}$, because the contribution of the first term vanishes) and another component due to the second term that is different in the various cases: $\widehat{\Gamma}_{\gamma}$ when $1/2<\gamma<1$, and $\widehat{\Sigma}_{\mathbf{N}\mathbf{N}}$ or $\widehat{\Sigma}^{*}_{\mathbf{N}\mathbf{N}}$, according to the value of ${\mathcal{R}e}(\lambda^{*})$, when $\gamma=1$.

Summing up, the main focus here concerns the asymptotic behavior of the empirical means $({\mathbf{N}}_{n})_{n}$, that has not been subject of study yet. Furthermore, although we recover some results on $(\mathbf{Z}_{n})_{n}$ proved in [2], we point out that the existence of joint central limit theorems for the pair $(\mathbf{Z}_{n},\mathbf{N}_{n})$ is not obvious because the “discount factors” in the dynamics of the increments $(\mathbf{Z}_{n}-\mathbf{Z}_{n-1})_{n}$ and $(\mathbf{N}_{n}-\mathbf{N}_{n-1})_{n}$ are generally different. Indeed, as shown in (7), these two stochastic processes follow the dynamics

[TABLE]

and so, when we assume $1/2<\gamma<1$, it could be surprising that there exists a common convergence rate. In addition, we will show that, when $1/2<\gamma<1$, the stochastic processes $N^{j}=(N_{n,j})_{n}$ located at different vertices of the graph synchronize among each other faster than how they converge to the common random limit $Z_{\infty}$, i.e. for any pair of vertices $(j,h)$ with $j\neq h$, the velocity at which $(N_{n,j}-N_{n,h})_{n}$ converges almost surely to zero is higher than the one at which $N^{j}=(N_{n,j})_{n}$ and $N^{h}=(N_{h,n})_{n}$ converge almost surely to $Z_{\infty}$. At the contrary, when $\gamma=1$ the stochastic processes $N^{j}=(N_{n,j})_{n}$ synchronize and converge almost surely to $Z_{\infty}$ at the same velocity. The same asymptotic behaviors characterize the stochastic processes $Z^{j}=(Z_{n,j})_{n}$, as proved also in [2, 21]. However, while it is somehow guessable from (8) that the velocities of synchronization and convergence for the processes $Z^{j}=(Z_{n,j})_{n}$ depend on the parameter $\gamma$, it could be somehow unexpected that, although the discount factor of the increments $(\mathbf{N}_{n}-\mathbf{N}_{n-1})$ is always $n^{-1}$, the corresponding velocities for the processes $N^{j}=(N_{n,j})_{n}$ also depend on $\gamma$ and, in general, also these processes do not synchronize and converge to $Z_{\infty}$ at the same velocity. As we will see, this fact is essentially due to their dependence on the process $(\mathbf{Z}_{n})_{n}$, which is induced by the process $(\mathbf{X}_{n})_{n}$. It is worthwhile to note that dynamics similar to (8) have already been considered in the Stochastic Approximation literature. Specifically, in [38] the authors established a CLT for a pair of recursive procedures having two different step-sizes. However, this result does not apply to our situation. Indeed, the covariance matrices $\Sigma_{\mu}$ and $\Sigma_{\theta}$ in their main result (Theorem 1) are deterministic, while the asymptotic covariance matrices in our CLTs are random (as said before, they depend on the random variable $Z_{\infty}$). This is why we do not use the simple convergence in distribution, but we employ the notion of stable convergence, which is, among other things, essential for the considered statistical applications. Finally, in [38], the authors find two different convergence rates, depending on the two different step-sizes, while, as already said, we find a common convergence rate.

The rest of the paper is organized as follows. In Section 2 we describe the notation and the assumptions used along the paper. In Section 3 we illustrate our main results and we discuss some possible statistical applications. An interesting example of interacting system is also provided in order to clarify the statement of the theorems and the related comments. Section 4 contains the proofs or the main steps of the proofs (postponing some technical lemmas to Appendix A) of the presented results. For the reader’s convenience, Appendix B supplies a brief review on the notion of stable convergence and its variants.

2. Notation and assumptions

Throughout all the paper, we will adopt the same notation used in [2]. In particular, we denote by ${\mathcal{R}e}(z)$, ${\mathcal{I}m}(z)$, $\overline{z}$ and $|z|$ the real part, the imaginary part, the conjugate and the modulus of a complex number $z$. Then, for a matrix $A$ with complex elements, we let $\overline{A}$ and $A^{\top}$ be its conjugate and its transpose, while we indicate by $|A|$ the sum of the modulus of its elements. The identity matrix is denoted by $I$, independently of its dimension that will be clear from the context. The spectrum of $A$, i.e. the set of all the eigenvalues of $A$ repeated with their multiplicity, is denoted by $Sp(A)$, while its sub-set containing the eigenvalues with maximum real part is denoted by $\lambda_{\max}(A)$, i.e. $\lambda^{*}\in\lambda_{\max}(A)$ whenever ${\mathcal{R}e}(\lambda^{*})=\max\{{\mathcal{R}e}(\lambda):\,\lambda\in Sp(A)\}$. Finally, we consider any vector $\mathbf{v}$ as a matrix with only one column (so that all the above notations apply to $\mathbf{v}$) and we indicate by $\|\mathbf{v}\|$ its norm, i.e. $\|\mathbf{v}\|^{2}=\overline{\mathbf{v}}^{\top}\mathbf{v}$. The vectors whose elements are all ones or zeros are denoted by $\mathbf{1}$ and $\mathbf{0}$, respectively, independently of their dimension that will be clear from the context.

Throughout all the paper, we assume that the following conditions hold:

Assumption 2.1**.**

There exist real constants $c>0$ and $1/2<\gamma\leq 1$ such that condition (3) is satisfied, which can be rewritten as

[TABLE]

In some results for $\gamma=1$, we will require a slightly stricter condition than (9), that is:

[TABLE]

We will explicitly mention this assumption in the statement of the theorems when it is required.

Assumption 2.2**.**

The weighted adjacency matrix $W$ is irreducible and diagonalizable.

The irreducibility of $W$ reflects a situation in which all the vertices are connected among each others and hence there are no sub-systems with independent dynamics (see [2, 3] for further details). The diagonalizability of $W$ allows us to find a non-singular matrix $\widetilde{U}$ such that $\widetilde{U}^{\top}W(\widetilde{U}^{\top})^{-1}$ is diagonal with complex elements $\lambda_{j}\in Sp(W)$. Notice that each column $\mathbf{u}_{j}$ of $\widetilde{U}$ is a left eigenvector of $W$ associated to some eigenvalue $\lambda_{j}$. Without loss of generality, we set $\|\mathbf{u}_{j}\|=1$. Moreover, when the multiplicity of some $\lambda_{j}$ is bigger than one, we set the corresponding eigenvectors to be orthogonal. Then, if we define $\widetilde{V}=(\widetilde{U}^{\top})^{-1}$, we have that each column $\mathbf{v}_{j}$ of $\widetilde{V}$ is a right eigenvector of $W$ associated to $\lambda_{j}$ such that

[TABLE]

These constraints combined with the above assumptions on $W$ (precisely, $w_{h,j}\geq 0$, $W^{\top}\mathbf{1}=\mathbf{1}$ and the irreducibility) imply, by Frobenius-Perron Theorem, that $\lambda_{1}:=1$ is an eigenvalue of $W$ with multiplicity one, $\lambda_{\max}(W)=\{1\}$ and

[TABLE]

We use $U$ and $V$ to indicate the sub-matrices of $\widetilde{U}$ and $\widetilde{V}$, respectively, whose columns are the left and the right eigenvectors of $W$ associated to $Sp(W)\setminus\{1\}$, that is $\{\mathbf{u}_{2},\dots,\mathbf{u}_{N}\}$ and $\{\mathbf{v}_{2},\dots,\mathbf{v}_{N}\}$, respectively, and, finally, we denote by $\lambda^{*}$ an eigenvalue belonging to $Sp(W)\setminus\{1\}$ such that

[TABLE]

In other words, if we denote by $D$ the diagonal matrix whose elements are $\lambda_{j}\in Sp(W)\setminus\{1\}$, we have $\lambda^{*}\in\lambda_{\max}(D)$.

3. Main results and discussion

In this section, we present and discuss our main results concerning the asymptotic behavior of the joint process $(\mathbf{Z}_{n},\mathbf{N}_{n})_{n}$. We recall the assumptions stated in Section 2 and we refer to Appendix B for a brief review on the notion of stable convergence and its variants.

We start by providing a first-order asymptotic result concerning the almost sure convergence of the sequence of pairs $(\mathbf{Z}_{n},\mathbf{N}_{n})_{n}$.

Theorem 3.1**.**

For $N\geq 1$, we have

[TABLE]

*where $Z_{\infty}$ is the random variable with values in $[0,1]$ defined as the common almost sure limit of the stochastic processes $Z^{j}=(Z_{n,j})_{n}$.

Moreover, the following statements hold true:*

• (i)

$P(Z_{\infty}=z)=0$* for any $z\in(0,1)$.*

• (ii)

If we have $P(\bigcap_{j=1}^{N}\{Z_{0,j}=0\})+P(\bigcap_{j=1}^{N}\{Z_{0,j}=1\})<1$, then $P(0<Z_{\infty}<1)>0$.

In particular, this result states that, when $N\geq 2$, all the stochastic processes $N^{j}=(N_{n,j})_{n}$, located at the different vertices $j\in V$ of the graph, synchronize almost surely, i.e. all of them converge almost surely toward the same random variable $Z_{\infty}$. Moreover, this random variable is the same limit toward which all the stochastic processes $Z^{j}=(Z_{n,j})_{n}$ synchronize almost surely (see Theorem 3.1 in [2]). In addition, it is interesting to note that the synchronization holds true without any assumption on the initial configuration $\mathbf{Z}_{0}$ and for any choice of the weighted adjacency matrix $W$ with the required assumptions. Finally, note that the synchronization is induced along time independently of the fixed size $N$ of the network, and so it does not require a large-scale limit (i.e. the limit for $N\to+\infty$), which is usual in statistical mechanics for the study of interacting particle systems.

We now focus on the second-order asymptotic results. Specifically, we present joint central limit theorems for the sequence of pairs $(\mathbf{Z}_{n},\mathbf{N}_{n})_{n}$ in the sense of stable convergence, that establish the rate of convergence to the limit $Z_{\infty}\mathbf{1}$ given in Theorem 3.1 and the relative asymptotic random covariance matrices. First, we consider the case $1/2<\gamma<1$:

Theorem 3.2**.**

For $N\geq 1$ and $1/2<\gamma<1$, we have that

[TABLE]

where

[TABLE]

and

[TABLE]

Remark 3.1**.**

Some considerations can be drawn by looking at the analytic expressions of $\widetilde{\sigma}_{\gamma}^{2}$ and $\widehat{\sigma}_{\gamma}^{2}$ in (14) and (15), respectively. First, they are both decreasing in $N$, so that the asymptotic variances are small when the number of vertices in the graph is large. Second, they are both increasing in $c$ and decreasing in $\gamma$, which, recalling that $\lim_{n}n^{\gamma}r_{n}=c$, means that the faster is the convergence to zero of the sequence $(r_{n})_{n}$, the lower are the values of the asymptotic variances $\widetilde{\sigma}_{\gamma}^{2}$ and $\widehat{\sigma}_{\gamma}^{2}$. Third, when $\gamma$ is close to $1/2$, $\widetilde{\sigma}_{\gamma}^{2}$ becomes very large, while $\widehat{\sigma}_{\gamma}^{2}$ remains bounded, and hence the processes $(\mathbf{Z}_{n}-Z_{\infty}\mathbf{1})$ and $(\mathbf{N}_{n}-Z_{\infty}\mathbf{1})$ become highly correlated. Finally, since we have

[TABLE]

we can obtain the following lower and upper bounds for $\widetilde{\sigma}_{\gamma}^{2}$ and $\widehat{\sigma}_{\gamma}^{2}$ (not depending on $W$):

[TABLE]

Notice that the lower bound is achieved when $\mathbf{v}_{1}=\mathbf{u}_{1}=N^{-1/2}\mathbf{1}$, i.e. when $W$ is doubly stochastic.

Remark 3.2**.**

Note that from (13) of Theorem 3.2, we get in particular that, for any pair of vertices $(j,h)$ with $j\neq h$, $n^{\gamma-\frac{1}{2}}(N_{n,j}-N_{n,h})$ converges to zero in probability. Indeed, denoting by $\mathbf{e}_{j}$ the vector such that $e_{j,j}=1$ and $e_{j,i}=0$ for all $i\neq j$, we have $\mathbf{1}^{\top}(\mathbf{e}_{j}-\mathbf{e}_{h})=0$ and hence $(\mathbf{e}_{j}-\mathbf{e}_{h})^{\top}\widetilde{\Sigma}_{\gamma}(\mathbf{e}_{j}-\mathbf{e}_{h})=(\mathbf{e}_{j}-\mathbf{e}_{h})^{\top}\widehat{\Gamma}_{\gamma}(\mathbf{e}_{j}-\mathbf{e}_{h})=0$. Therefore, Theorem 3.2 implies that the velocity at which the stochastic processes $N^{j}=(N_{n,j})_{n}$, located at different vertices $j\in V$, synchronize among each other is higher than the one at which each of them converges almost surely to the common random limit $Z_{\infty}$. The same asymptotic behavior is shown also by the stochastic processes $Z^{j}=(Z_{n,j})_{n}$ as shown in [2, 21].

For $\gamma=1$ we need to distinguish the case $N=1$ and the case $N\geq 2$. Indeed, in the second case we can have different convergence rates according to the value of ${\mathcal{R}e}(\lambda^{*})$. More precisely, we have the following results:

Theorem 3.3**.**

For $N=1$ and $\gamma=1$, we have that

[TABLE]

Theorem 3.4**.**

For $N\geq 2$, $\gamma=1$ and ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$, under condition (10), we have that

[TABLE]

where $\widetilde{\Sigma}_{1}$ is defined as in (14) with $\gamma=1$, and

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

The condition ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$ in the above Theorem 3.4 is the analogous of the one typically required for the CLTs in the Stochastic Approximation framework (e.g. [30, 38, 41]). However, we deal with a random limit $Z_{\infty}$ and random asymptotic covariances and our proofs are not based on that results, but we employ different arguments. Moreover, in the next theorem, we analyze also the case ${\mathcal{R}e}(\lambda^{*})=1-(2c)^{-1}$.

Theorem 3.5**.**

For $N\geq 2$, $\gamma=1$ and ${\mathcal{R}e}(\lambda^{*})=1-(2c)^{-1}$, under condition (10), we have that

[TABLE]

where

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

Remark 3.3**.**

The central limit theorem only for the stochastic process $(\mathbf{Z}_{n})_{n}$ can be established in the case ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$ replacing condition (10) with the more general assumption (9) (see Theorem 3.2 in [2]). However, condition (10) is essential in our proof of the central limit theorem for the joint stochastic process $(\mathbf{Z}_{n},\mathbf{N}_{n})_{n}$ as stated in Theorem 3.4.

Remark 3.4**.**

From Theorem 3.4 and Theorem 3.5 we get that, when $N\geq 2$ and $\gamma=1$, for any pair of vertices $(j,h)$ with $j\neq h$, the difference $(N_{n,j}-N_{n,h})$ converges almost surely to zero with the same velocity at which each process $N^{j}=(N_{n,j})$ converges almost surely to $Z_{\infty}$. (The same asymptotic behavior is shown also by the stochastic processes $Z^{j}=(Z_{n,j})_{n}$ as provided in [2, 21].) Indeed, although $\widetilde{\Sigma}_{1}(\mathbf{e}_{j}-\mathbf{e}_{h})=\mathbf{0}$ and $\mathbf{u}_{1}^{\top}(\mathbf{e}_{j}-\mathbf{e}_{h})=0$, we have $U^{\top}(\mathbf{e}_{j}-\mathbf{e}_{h})\neq\mathbf{0}$ and hence, setting $\mathbf{u}_{j,h}:=U^{\top}(\mathbf{e}_{j}-\mathbf{e}_{h})$ and $\widetilde{\mathbf{u}}_{j,h}:=\widetilde{U}^{\top}(\mathbf{e}_{j}-\mathbf{e}_{h})=(0,\mathbf{u}_{j,h})^{\top}$, for ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$ by (16) we have

[TABLE]

while for ${\mathcal{R}e}(\lambda^{*})=1-(2c)^{-1}$ by (25) we have

[TABLE]

Notice that the only elements $[\widehat{S}_{\mathbf{NN}}]_{h,j}$ that count in the above limit relations are those with $2\leq h,j\leq N$. Then, from (19) we can see that these elements remain bounded for any value of $c$, while from (17) we can see that the elements of $\widehat{S}_{\mathbf{ZZ}}$ are increasing in $c$. (The same considerations can be made for the elements of the matrices $\widehat{S}^{*}_{\mathbf{NN}}$ and $\widehat{S}^{*}_{\mathbf{ZZ}}$, but in this case the value of $c$ is uniquely determined by ${\mathcal{R}e}(\lambda^{*})$). As a consequence, for large values of $c$, the asymptotic variance of $(N_{n,j}-N_{n,h})$ becomes negligible with respect to the one of $(Z_{n,j}-Z_{n,h})$. Therefore, when $N\geq 2$ and $\gamma=1$, the synchronization between the empirical means $N^{j}=(N_{n,j})_{n}$, located at different vertices $j\in V$, is more accurate than the synchronization between the stochastic processes $Z^{j}=(Z_{n,j})_{n}$.

An interesting example of interacting system is provided by the “mean-field interaction”, already considered in [2, 21, 22, 25]. Naturally, all the weighted adjacency matrices introduced and analyzed in [2] can be considered as well.

Example 3.1**.**

The mean-field interaction can be expressed in terms of a particular weighted adjacency matrix $W$ as follows: for any $1\leq h,j\leq N$ (here we consider only the true “interacting case”, that is $N\geq 2$)

[TABLE]

where $\delta_{h,j}$ is equal to $1$ when $h=j$ and to [math] otherwise. Note that $W$ in (32) is irreducible for $\alpha>0$ and so we are going to consider this case. Since $W$ is doubly stochastic, we have ${\mathbf{v}}_{1}={\mathbf{u}}_{1}=N^{-1/2}\mathbf{1}$. Thus, for $1/2<\gamma<1$, we have

[TABLE]

Furthermore, we have $\lambda_{j}=1-\alpha$ for all $\lambda_{j}\in Sp(W)\setminus\{1\}$ and, consequently, the conditions ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$ or ${\mathcal{R}e}(\lambda^{*})=1-(2c)^{-1}$ required in the previous results when $\gamma=1$ correspond to the conditions $2c\alpha>1$ or $2c\alpha=1$. Finally, since $W$ is also symmetric, we have $U=V$ and so $U^{\top}U=V^{\top}V=I$ and $UU^{\top}=VV^{\top}=(I-N^{-1}\mathbf{1}\mathbf{1}^{\top})$. Therefore, for the case $\gamma=1$ and $2c\alpha>1$, we obtain:

• (i)

$\widehat{S}_{\mathbf{ZZ}}=\frac{c^{2}}{2c\alpha-1}I$;

• (ii)

$[\widehat{S}_{\mathbf{NN}}]_{1,1}=(c-1)^{2}$ and $[\widehat{S}_{\mathbf{NN}}]_{j,j}=\frac{1+2(c-1)\alpha^{-1}}{2c\alpha-1}$ for $2\leq j\leq N$, while $[\widehat{S}_{\mathbf{NN}}]_{h,j}=0$ for any $h\neq j$, $1\leq h,j\leq N$;

• (iii)

$[\widehat{S}_{\mathbf{ZN}}]_{j,j}=\frac{c+(c-1)\alpha^{-1}}{2c\alpha-1}$ for $2\leq j\leq N$, while $[\widehat{S}_{\mathbf{ZN}}]_{h,j}=0$ for any $h\neq j$, $2\leq h\leq N$ and $1\leq j\leq N$.

Finally, when $\gamma=1$ and $2c\alpha=1$, we get:

• (i)

$\widehat{S}^{*}_{\mathbf{ZZ}}=c^{2}I$;

• (ii)

$\widehat{S}^{*}_{\mathbf{NN}}=\frac{(1-\alpha)^{2}}{\alpha^{2}}I$;

• (iii)

$\widehat{S}^{*}_{\mathbf{ZN}}=\frac{c(1-\alpha)}{\alpha}I$. ∎

3.1. Some comments on statistical applications

The first statistical tool that can be derived from the previous convergence results is the construction of asymptotic confidence intervals for the limit random variable $Z_{\infty}$. This issue has been already considered in [2], where from the central limit theorem for $\widetilde{Z}_{n}:=N^{-1/2}\,\mathbf{v}_{1}^{\top}\,\mathbf{Z}_{n}$ (recalled here in the following Theorem 4.1), a confidence interval with approximate level $(1-\theta)$ is obtained for any $1/2<\gamma\leq 1$ as:

[TABLE]

where $\widetilde{\sigma}_{\gamma}^{2}$ is defined as in (14) (also for $\gamma=1$) and $z_{\theta}$ is such that ${\mathcal{N}}(0,1)(z_{\theta},+\infty)=\theta/2$. We note that the construction of the above interval requires to know the following quantities:

• (i)

$N$: the number of vertices in the network;

• (ii)

$\mathbf{v}_{1}$: the right eigenvector of $W$ associated to $\lambda_{1}=1$ (note that it is not required to know the whole weighted adjacency matrix $W$, e.g. we have $\mathbf{v}_{1}=\mathbf{u}_{1}=N^{-1/2}\mathbf{1}$ for any doubly stochastic matrix);

• (iii)

$\gamma$ and $c$: the parameters that describe the first-order asymptotic approximation of the sequence $(r_{n})_{n}$ (see Assumption 2.1).

In addition, the asymptotic confidence interval in (33) requires the observation of $\widetilde{Z}_{n}$, and so of $Z_{n,j}$ for any $j\in V$. However, this requirement may not be feasible in practical applications since the initial random variables $Z_{0,j}$ and the exact expression of the sequence $(r_{n})_{n}$ are typically unknown. For instance, if at each vertex $j\in V$ we have a standard Pòlya’s urn with initial composition given by the pair $(a,b)$, then we have $Z_{0,j}=a/(a+b)$ and $r_{n}=(a+b+n+1)^{-1}$ and hence, when the initial composition is unknown, we have neither $Z_{0,j}$ nor the exact value of $r_{n}$, but we can get $\gamma=c=1$. To face this problem, here we propose asymptotic confidence intervals for $Z_{\infty}$ that do not require the observation of $Z_{n,j}$, but are based on the empirical means $N_{n,j}=\sum_{k=1}^{n}X_{k,j}/n$, where the random variables $X_{k,j}$ are typically observable. To this aim, we consider the convergence results presented in Section 3 on the asymptotic behavior of $\mathbf{N}_{n}$.

We first focus on the case $1/2<\gamma<1$ and we construct an asymptotic confidence interval for $Z_{\infty}$ based on the empirical means $N_{n,j}$, with $j\in V$, and the quantities in (i)-(ii)-(iii). Indeed, setting $\widetilde{N}_{n}:=N^{-1/2}\mathbf{v}_{1}^{\top}\mathbf{N}_{n}$ and using the relation $\mathbf{v}_{1}^{\top}\mathbf{u}_{1}=N^{-1/2}\mathbf{v}_{1}^{\top}\mathbf{1}=1$ (see (12)), from Theorem 3.2 we obtain that

[TABLE]

where $\widetilde{\sigma}_{\gamma}^{2}$ and $\widehat{\sigma}_{\gamma}^{2}$ are defined in (14) and (15), respectively. Then, for $1/2<\gamma<1$, we have the following confidence interval with approximate level $(1-\theta)$:

[TABLE]

Analogously, for $\gamma=1$ and $N=1$, from Theorem 3.3 we get

[TABLE]

When $\gamma=1$ and $N\geq 2$, we have to distinguish two cases according to the value of ${\mathcal{R}e}(\lambda^{*})$. Thus, in this case, the construction of suitable asymptotic confidence intervals for $Z_{\infty}$ requires also the knowledge of ${\mathcal{R}e}(\lambda^{*})$. Specifically, when ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$, from Theorem 3.4, using the relations $\mathbf{v}_{1}^{\top}\mathbf{u}_{1}=1$ and $\mathbf{v}_{1}^{\top}U=\mathbf{0}$ (see (11)), we obtain that

[TABLE]

where $\widetilde{\sigma}_{1}^{2}=c^{2}\|\mathbf{v}_{1}\|^{2}/N$ and $[\widehat{S}_{\mathbf{NN}}]_{1,1}=(c-1)^{2}\|\mathbf{v}_{1}\|^{2}$. Hence, in this case we find:

[TABLE]

Note that analogous asymptotic confidence intervals for $Z_{\infty}$ can be constructed replacing $\widetilde{N}_{n}$ by another real stochastic processes $(\mathbf{a}^{\top}\mathbf{N}_{n})_{n}$, where $\mathbf{a}\in\mathbb{R}^{N}$ and $\mathbf{a}^{\top}\mathbf{1}=1$.

Finally, when ${\mathcal{R}e}(\lambda^{*})=1-(2c)^{-1}$, we can not use $\widetilde{N}_{n}$ since, by Theorem 3.5 and the fact that $\mathbf{v}_{1}^{\top}U=\mathbf{0}$, we have $\sqrt{n/\ln(n)}(\widetilde{N}_{n}-Z_{\infty})\to 0$ in probability. Therefore, in this case we need to replace the vector $\mathbf{v}_{1}$ by another vector $\mathbf{a}\in\mathbb{R}^{N}$ with $\mathbf{a}^{\top}\mathbf{1}=1$ and $\mathbf{a}^{\top}U\neq\mathbf{0}$.

Example 3.2**.**

In the case of a system with $N\geq 2$ and mean-field interaction (see Example 3.1), we get the following asymptotic confidence intervals for $Z_{\infty}$ with approximate level $(1-\theta)$:

• (i)

when $1/2<\gamma<1$, setting $\widetilde{N}_{n}=N^{-1}\mathbf{1}^{\top}\mathbf{N}_{n}$, we have

[TABLE]

• (ii)

when $\gamma=1$ and $2c\alpha>1$, setting $\widetilde{N}_{n}=N^{-1}\mathbf{1}^{\top}\mathbf{N}_{n}$, we have

[TABLE]

• (iii)

when $\gamma=1$ and $2c\alpha=1$, setting $\widetilde{N}^{a}_{n}:=\mathbf{a}^{\top}\mathbf{N}_{n}$ with $\mathbf{a}^{\top}\mathbf{1}=1$ and $\mathbf{a}\neq N^{-1}\mathbf{1}$, we have

[TABLE]

where the last term follows by recalling that $UU^{\top}=I-N^{-1}\mathbf{1}\mathbf{1}^{\top}$ and noticing that

[TABLE]

(where for the last two equalities we used that $\mathbf{a}^{\top}\mathbf{1}=1$).

∎

Another possible statistical application of the convergence results of Section 3 concerns the inference on the weighted adjacency matrix $W$ based on the empirical means $N_{n,j}$, with $j\in V$, instead of the random variables $Z_{n,j}$ as done in [2]. Let us assume $N\geq 2$ (the proper “interacting” case). We propose to construct testing procedures based on the multi-dimensional real stochastic process $(UV^{\top}\mathbf{N}_{n})_{n}$. Indeed, we note that it converges to $\mathbf{0}$ almost surely because $\mathbf{N}_{n}\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}Z_{\infty}\mathbf{1}$ and $V^{\top}\mathbf{1}=\mathbf{0}$ (since (11) and (12)). Moreover, when $\gamma=1$ and ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$, from Theorem 3.4 we get that

[TABLE]

where $[\widehat{S}_{\mathbf{NN}}]_{(-1)}$ denotes the square sub-matrix obtained from $\widehat{S}_{\mathbf{NN}}$ removing its first row and its first column.

Analogously, when $\gamma=1$ and ${\mathcal{R}e}(\lambda^{*})=1-(2c)^{-1}$, from Theorem 3.5 we get that

[TABLE]

Remember that the case $\gamma=1$ includes, for instance, systems of interacting Pólya’s urns.

Example 3.3**.**

In the case of $N\geq 2$ and mean-field interaction (see Example 3.1), recalling that $U=V$, $UU^{\top}=(I-N^{-1}\mathbf{1}\mathbf{1}^{\top})$, $[\widehat{S}_{\mathbf{NN}}]_{(-1)}=\frac{1+2(c-1)\alpha^{-1}}{2c\alpha-1}I$ and $\widehat{\Sigma}^{*}_{\mathbf{NN}}=\frac{(1-\alpha)^{2}}{\alpha^{2}}UU^{\top}$, we obtain that:

• (i)

when $\gamma=1$ and $2c\alpha>1$,

[TABLE]

• (ii)

when $\gamma=1$ and $2c\alpha=1$,

[TABLE]

In this framework, it may be of interest to test whether the unknown parameter $\alpha$ can be assumed to be equal to a specific value $\alpha_{0}\in(0,1]$, i.e. we may be interested in a statistical test of the type:

[TABLE]

To this purpose, assuming $2c\alpha_{0}\geq 1$ and setting $\widetilde{N}_{n}:=N^{-1}\mathbf{1}^{\top}\mathbf{N}_{n}$, we note that:

• (i)

for $\gamma=1$ and $2c\alpha_{0}>1$, under $H_{0}$ we have that

[TABLE]

• (ii)

for $\gamma=1$ and $2c\alpha_{0}=1$, under $H_{0}$ we have that

[TABLE]

Concerning the distribution of the above quantities for $\alpha\neq\alpha_{0}$, since the eigenvectors of $W$ do not depend on $\alpha$, we have that, for any fixed $\alpha\in(0,1]\setminus\{\alpha_{0}\}$, under the hypothesis $\{W=W_{\alpha}\}\subset H_{1}$, we have that:

• (i)

for $\gamma=1$, $2c\alpha_{0}>1$ and for any $\alpha\neq\alpha_{0}$ such that $2c\alpha>1$,

[TABLE]

while, if $2c\alpha=1$, the above quantity converges in probability to infinity;

• (ii)

for $\gamma=1$, $2c\alpha_{0}=1$ and for any $\alpha$ such that $2c\alpha>1$ (which obviously implies $\alpha\neq\alpha_{0}$), we have

[TABLE]

∎

The case $1/2<\gamma<1$ requires further future investigation. Indeed, since $V^{\top}\mathbf{1}=\mathbf{0}$ (by (11) and (12)), from Theorem 3.2 we obtain that $n^{\gamma-\frac{1}{2}}UV^{\top}\mathbf{N}_{n}\to\mathbf{0}$ in probability. Then, a central limit theorem for $UV^{\top}\mathbf{N}_{n}$ with the exact convergence rate (if exists) is needful. In this paper, as we will see more ahead in Remark 4.2, by the computations done in the proofs of Section 4 we can only affirm that $n^{e}UV^{\top}\mathbf{N}_{n}\to\mathbf{0}$ in probability for all $e<\gamma/2$ and, when $e=\gamma/2$, the random vector $n^{e}UV^{\top}\mathbf{N}_{n}$ is the sum of a term converging to zero in probability and a term bounded in $L^{1}$. Therefore, further analysis on the asymptotic behavior of $n^{\gamma/2}UV^{\top}\mathbf{N}_{n}$ results to be interesting for future developments.

4. Proofs

This section contains all the proofs of the results presented in the previous Section 3.

4.1. Preliminary relations and results

We start by recalling that, given the eigen-structure of $W$ described in Section 2, the matrix ${\mathbf{u}}_{1}{\mathbf{v}}_{1}^{\top}$ has real elements and the following relations hold:

[TABLE]

which implies that the matrix $UV^{\top}$ has real elements. Moreover, using the matrix $D$ defined in Section 2, we can decompose the matrix $W^{\top}$ as follows:

[TABLE]

Now, in order to understand the asymptotic behavior of the stochastic processes $(\mathbf{Z}_{n})_{n}$ and $(\mathbf{N}_{n})_{n}$, let us express the dynamics (7) as follows:

[TABLE]

where $\Delta\mathbf{M}_{n+1}=(\mathbf{X}_{n+1}-W^{\top}\mathbf{Z}_{n})$ is a martingale increment with respect to the filtration ${\mathcal{F}}:=({\mathcal{F}}_{n})_{n}$. Furthermore, we decompose the stochastic process $(\mathbf{Z}_{n})_{n}$ as

[TABLE]

while we decompose the stochastic process $(\mathbf{N}_{n})_{n}$ as

[TABLE]

Then, the asymptotic behavior of the joint stochastic process $(\mathbf{Z}_{n},\mathbf{N}_{n})_{n}$ is obtained by establishing the asymptotic behavior of $(\widetilde{Z}_{n})_{n}$ and of $(\widehat{\mathbf{Z}}_{n},\widehat{\mathbf{N}}_{n})_{n}$.

Remark 4.1**.**

In the particular case when $W$ is doubly stochastic, we have $\mathbf{v}_{1}=\mathbf{u}_{1}=N^{-1/2}\mathbf{1}$. As a consequence, we have

[TABLE]

which represents the average of the stochastic processes $Z_{n,j}$, with $j\in V$, in the network, and

[TABLE]

Notice that the assumed normalization $W^{\top}\mathbf{1}=\mathbf{1}$ implies that symmetric matrices $W$ are also doubly stochastic. Therefore, the above equalities hold for any undirected graph for which $W$ is symmetric by definition.

Concerning the real-valued stochastic process $(\widetilde{Z}_{n})_{n}$, from [2, Section 4.2] we have that it is an $\mathcal{F}$-martingale with values in $[0,1]$ and its dynamics can be expressed as follows:

[TABLE]

In particular, we have that ${\widetilde{Z}}_{n}\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}Z_{\infty}$ and in [2] the following central limit theorem for $(\widetilde{Z}_{n})_{n}$ is established:

Theorem 4.1**.**

[2, Theorem 4.2]** For $N\geq 1$ and $1/2<\gamma\leq 1$, we have

[TABLE]

where $\widetilde{\sigma}_{\gamma}^{2}$ is defined as in (14) (also for $\gamma=1$). The above convergence is also in the sense of the almost sure conditional convergence w.r.t. ${\mathcal{F}}$.

Concerning the multi-dimensional real stochastic process $(\widehat{\mathbf{Z}}_{n})_{n}$, we firstly recall the relation

[TABLE]

which is due to (34) and (35), and, moreover, we recall that from [2, Section 4.2] we have the dynamics

[TABLE]

and $\widehat{\mathbf{Z}}_{n}\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}\mathbf{0}$.

Finally, concerning the multi-dimensional real stochastic process $(\widehat{\mathbf{N}}_{n})_{n}$, using (36), (37), (38) and the assumption $W^{\top}\mathbf{1}=\mathbf{1}$ (which implies $W^{\top}\mathbf{Z}_{n}=\widetilde{Z}_{n}\mathbf{1}+W^{\top}\widehat{\mathbf{Z}}_{n}$), we obtain the dynamics:

[TABLE]

4.2. Proof of Theorem 3.1

(Almost sure synchronization of the empirical means)

We recall that in [2, Theorem 3.1], by decomposition (37), i.e. $\mathbf{Z}_{n}=\widetilde{Z}_{n}\mathbf{1}+\widehat{\mathbf{Z}}_{n}$, and combining ${\widetilde{Z}}_{n}\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}Z_{\infty}$ and $\widehat{\mathbf{Z}}_{n}\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}\mathbf{0}$, it is proved that $\mathbf{Z}_{n}\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}Z_{\infty}\mathbf{1}$. As a consequence, using $W^{\top}\mathbf{1}=\mathbf{1}$ and (6), we obtain $E[\mathbf{X}_{n}|{\mathcal{F}}_{n-1}]\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}Z_{\infty}\mathbf{1}$ and, applying Lemma A.2 (with $c_{k}=k$, $v_{n,k}=k/n$ and $\eta=1$), we get that $\mathbf{N}_{n}\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}\ Z_{\infty}\mathbf{1}$. This concludes the proof of the first part of the theorem, concerning the synchronization result. For the second part, that is the results on the limit random variable $Z_{\infty}$, we refer to [2, Theorem 3.5 and Theorem 3.6]. ∎

Note that, by the synchronization result for $(Z_{n})$, we can state that

[TABLE]

Indeed, since $\{X_{n+1,j}:\,j=1,\dots,N\}$ are conditionally independent given $\mathcal{F}_{n}$, we have

[TABLE]

while, for each $j$, using the normalization $W^{\top}{\mathbf{1}}={\mathbf{1}}$, we have

[TABLE]

4.3. Proof of Theorem 3.2

(CLT for $({\mathbf{Z}}_{n},{\mathbf{N}}_{n})_{n}$ in the case $1/2<\gamma<1$)

In order to prove Theorem 3.2, we need to provide the asymptotic behavior of the stochastic processes $(\widehat{\mathbf{Z}}_{n})_{n}$ and $(\widehat{\mathbf{N}}_{n})_{n}$. First of all, we recall that $\widehat{\mathbf{Z}}_{n}=0$ for each $n$ when $N=1$ and, for $N\geq 2$ and $1/2<\gamma<1$, we have from [2, Theorem 4.3] that

[TABLE]

where

[TABLE]

Moreover, looking at the proof of (46) in [2], it is easy to realize that for $N\geq 2$ and $1/2<\gamma<1$ we have $\lim_{n}n^{\gamma}E\left[\,\|\widehat{\mathbf{Z}}_{n}\|^{2}\,\right]=C$, where $C$ is a suitable constant in $(0,+\infty)$, and so, recalling that $\widehat{\mathbf{Z}}_{n}=0$ for each $n$ when $N=1$, we can affirm that, for every $N\geq 1$ and $1/2<\gamma<1$, we have that

[TABLE]

Regarding the stochastic process $(\widehat{\mathbf{N}}_{n})_{n}$, we are going to prove the following convergence result:

Theorem 4.2**.**

For $N\geq 1$ and $1/2<\gamma<1$, we have that

[TABLE]

where $\widehat{\Gamma}_{\gamma}$ is the matrix defined in (15).

Proof.

We observe that by means of (42) we can write

[TABLE]

Then, using the relation

[TABLE]

we obtain that

[TABLE]

Now, we set $e:=\gamma-1/2>0$ for each $1/2<\gamma<1$ and hence from the above expression we get $n^{e}\widehat{\mathbf{N}}_{n}=t_{n}\sum_{k=1}^{n}{\mathbf{T}}_{k}+W^{\top}{\mathbf{Q}}_{n}$, where $t_{n}:=1/n^{(1-e)}$, ${\mathbf{Q}}_{n}:=t_{n}\sum_{k=1}^{n}\widehat{\mathbf{Z}}_{k-1}$ and

[TABLE]

The idea of the proof is to study separately the two terms

[TABLE]

More precisely, we are going to prove that the first term converges stably to the desired Gaussian kernel, while the second term converges in probability to zero.

*First step: the convergence result for $t_{n}\sum_{k=1}^{n}{\mathbf{T}}_{k}$.

*We note that $(\mathbf{T}_{k})_{1\leq k\leq n}$ is a martingale difference array with respect to ${\mathcal{F}}$. Therefore, we want to apply Theorem B.1 (with $k_{n}=n$, $\mathbf{T}_{n,k}=\mathbf{T}_{k}$ and ${\mathcal{G}}_{n,k}={\mathcal{F}}_{k}$). To this purpose, we observe that condition (c1) is obviously satisfied and so we have to prove only conditions (c2) and (c3).

Regarding condition (c2), we note that

[TABLE]

The convergence rate of each of the four terms will be determined in the following.

By (43) and Lemma A.2 (with $c_{k}=k$, $v_{n,k}=k/n$ and $\eta=1$), for the first term, we obtain that

[TABLE]

Moreover, regarding the second term, by (69) we have that

[TABLE]

and, since by (44) and (45) we have that

[TABLE]

by Lemma A.2 again (with $c_{k}=k$, $v_{n,k}=k^{3}r_{k}^{2}/n^{2(1-e)}$ and $\eta=\frac{c^{2}}{2(1-e)}$), we obtain that

[TABLE]

Furthermore, concerning the third term, by (69) we have that

[TABLE]

On the other hand, by means of (44) and (45), we have that

[TABLE]

and so, by Lemma A.2 again (with $c_{k}=k$, $v_{n,k}=kr_{k}/n^{1+\frac{1}{2}-e}$ and $\eta=\frac{c}{(1+1/2-e)}$), it follows

[TABLE]

Finally, for the convergence of the fourth term, we can argue as we have just done for the third one. Indeed, observing that, by (44) and (45), we have that

[TABLE]

we get

[TABLE]

Summing up, since for $1/2<\gamma<1$ we have $2(1-e)>1$ and $2(1-e)>1+1/2-e$, we obtain that

[TABLE]

Regarding condition (c3), we note that

[TABLE]

Therefore also this condition is satisfied and we can conclude that $t_{n}\sum_{k=1}^{n}{\mathbf{T}}_{k}$ converges stably to the Gaussian kernel with mean zero and random covariance matrix given by (49).

*Second step: the convergence result for $\mathbf{Q}_{n}$.

*We aim at proving that $\mathbf{Q}_{n}$ converges in probability to zero, that is each component $Q_{n,j}$ converges in probability to zero. To this purpose, we note that

[TABLE]

Therefore, recalling that, for $1/2<\gamma<1$, we have $E\left[\|\widehat{\mathbf{Z}}_{n}\|^{2}\right]=O(n^{-\gamma})$ (see (47)), we can conclude by (69) that

[TABLE]

∎

Now, the proof of Theorem 3.2 follows from the previous result, together with Theorem 4.1 and Theorem B.2.

Proof of Theorem 3.2. By Theorem 4.1, we have that

[TABLE]

Thus, from Theorem 4.2, applying Theorem B.2, we obtain that

[TABLE]

In order to conclude, it is enough to observe that

[TABLE]

where $\Phi(x,y)=(y,x+y)$ and the last term converges in probability to zero (since $\widehat{\mathbf{Z}}_{n}=0$ for each $n$ when $N=1$ and since (46) when $N\geq 2$). ∎

Remark 4.2**.**

With reference to the statistical applications discussed in Subsection 3.1, we recall that, since $V^{\top}\mathbf{1}=\mathbf{0}$ (by (34)), we have $UV^{\top}\mathbf{N}_{n}=UV^{\top}\widehat{\mathbf{N}}_{n}$ and $V^{\top}\widehat{\Gamma}_{\gamma}V$ is the null matrix, and so from (48) we can get that $n^{\gamma-\frac{1}{2}}UV^{\top}\mathbf{N}_{n}\stackrel{{\scriptstyle P}}{{\to}}\mathbf{0}$ for $1/2<\gamma<1$. More precisely, following the arguments in the proof of Theorem 4.2, it is possible to show that, when $1/2<\gamma<1$, we have $n^{e}UV^{\top}\mathbf{N}_{n}\stackrel{{\scriptstyle P}}{{\to}}\mathbf{0}$ for each $e<\gamma/2$. Indeed, from (42), together with (40) and again the relation $V^{\top}\mathbf{1}=\mathbf{0}$, we obtain

[TABLE]

and hence, setting $t_{n}:=1/n^{1-e}$, $\mathbf{T}_{k}:=UV^{\top}\Delta\mathbf{M}_{k}$ and $\mathbf{Q}_{n}:=t_{n}\sum_{k=1}^{n}\widehat{\mathbf{Z}}_{k-1}$, we get

[TABLE]

where $\frac{1}{\sqrt{n}}\sum_{k=1}^{n}\mathbf{T}_{k}$ converges stably to the Gaussian kernel $\mathcal{N}(\mathbf{0},Z_{\infty}(1-Z_{\infty})UV^{\top}VU^{\top})$ and $E[\,|\mathbf{Q}_{n}|\,]=O(t_{n}n^{1-\frac{\gamma}{2}})=O(n^{-(\frac{\gamma}{2}-e)}).$ From these relations, we can also conclude that for $1/2<\gamma<1$ and $e=\gamma/2$, we have that $n^{e}UV^{\top}\mathbf{N}_{n}$ is the sum of a term converging to zero in probability and a term bounded in $L^{1}$. Therefore the asymptotic behavior of $n^{\gamma/2}UV^{\top}\mathbf{N}_{n}$ needs further investigation.

4.4. Proof of Theorem 3.3

(CLT for $({\mathbf{Z}}_{n},{\mathbf{N}}_{n})_{n}$ in the case $N=1$ and $\gamma=1$)

The proof in the case $N=1$ and $\gamma=1$ is similar to the one for $1/2<\gamma<1$. Indeed, using the same arguments as in the proof of Theorem 4.2, together with the facts that $\widetilde{Z}_{n}=Z_{n}$, $\widehat{\mathbf{Z}}_{n}=0$ for each $n$, $\mathbf{v}_{1}=v_{1,1}=1$ and $2(1-e)=1+1/2-e=1$, we obtain that

[TABLE]

On the other hand, by Theorem 4.1, we have that

[TABLE]

Thus, applying Theorem B.2, we obtain

[TABLE]

In order to conclude, it is enough to observe that

[TABLE]

where $\Phi(x,y)=(y,x+y)$. ∎

Remark 4.3**.**

Looking at the arguments of the proof of Theorem 4.2 with $N\geq 2$ and $\gamma=1$, we find $E\left[\,|\mathbf{Q}_{n}|\,\right]=O\left(\frac{1}{n^{(1-\gamma)/2}}\right)=O(1)$ and so, from this relation, we can not conclude that $\mathbf{Q}_{n}$ converges to zero in probability. Therefore part of the proof of Theorem 4.2 does not work when $N\geq 2$ and $\gamma=1$. Moreover, since $\mathbf{Q}_{n}=\sum_{k=1}^{n}\widehat{\mathbf{Z}}_{k-1}/\sqrt{n}$ and, from [2, Theorem 4.3], we know that, when $N\geq 2$ and $\gamma=1$, the rate of convergence of $\widehat{\mathbf{Z}}_{n}$ is $\sqrt{n}$ or $\sqrt{n/\ln(n)}$ according to the value of ${\mathcal{R}e}(\lambda^{*})$, we may conjecture that, for $N\geq 2$ and $\gamma=1$, $\mathbf{Q}_{n}$ generally does not converge in probability to zero. This fact leads us to a complete different approach to the proofs of Theorem 3.4 and Theorem 3.5 concerning the case $N\geq 2$ and $\gamma=1$, that will be developed in the next sections.

4.5. Proof of Theorem 3.4

(CLT for $({\mathbf{Z}}_{n},{\mathbf{N}}_{n})_{n}$ in the case $N\geq 2$, $\gamma=1$ and ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$)

In order to prove Theorem 3.4, we need the following convergence result on $(\widehat{\mathbf{Z}}_{n},\widehat{\mathbf{N}}_{n})_{n}$:

Theorem 4.3**.**

Let $N\geq 2$, $\gamma=1$ and ${\mathcal{R}}e(\lambda^{*})<1-(2c)^{-1}$. Then, under condition (10), we have that

[TABLE]

where $\widehat{\Sigma}_{\mathbf{ZZ}}$, $\widehat{\Sigma}_{\mathbf{NN}}$ and $\widehat{\Sigma}_{\mathbf{ZN}}$ are the matrices defined in (17), (19) and (22), respectively.

Proof.

First we use (40) in (42) and we replace the term $(\widetilde{Z}_{n}-\widetilde{Z}_{n-1})$ in (42) as shown in (39), so that we obtain

[TABLE]

Then, if we define the remainder term as

[TABLE]

we can rewrite the above dynamics of $\widehat{\mathbf{N}}_{n}$ as follows:

[TABLE]

Then, setting $\boldsymbol{\theta}_{n}:=(\widehat{\mathbf{Z}}_{n},\widehat{\mathbf{N}}_{n})^{\top}$, $\Delta\mathbf{M}_{\theta,n}:=(\Delta\mathbf{M}_{n},\Delta\mathbf{M}_{n})^{\top}$ and $\mathbf{R}_{\theta,n}:=(\mathbf{0},\mathbf{R}_{n})^{\top}$, which are vectors of dimension $2N$, and combining (41) and (51), we can write

[TABLE]

where

[TABLE]

and (recalling that $\mathbf{u}_{1}=N^{-1/2}\mathbf{1}$ and $I=\mathbf{u}_{1}\mathbf{v}_{1}^{\top}+UV^{\top}$ by (12) and (34))

[TABLE]

Now, we will prove that $\sqrt{n}\boldsymbol{\theta}_{n}$ converges stably to the desired Gaussian kernel. To this end, the first step is to define the $(2N)\times(2N-1)$ matrices

[TABLE]

and observe that from (34) we have $V_{\theta}^{\top}U_{\theta}=I$ and

[TABLE]

Then, defining the $(2N)\times(2N-1)$ matrices

[TABLE]

we have that $Q=U_{\theta}S_{Q}V_{\theta}^{\top}$ and $R=U_{\theta}S_{R}V_{\theta}^{\top}$. From the above relations on $U_{\theta}$ and $V_{\theta}$, we get that $U_{\theta}V_{\theta}^{\top}\boldsymbol{\theta}_{n}=\boldsymbol{\theta}_{n}$ and hence we can write

[TABLE]

Let us now set $\alpha_{j}:=1-\lambda_{j}\in\mathbb{C}$ with $\lambda_{j}\in Sp(W)\setminus\{1\}=Sp(D)$ and recall that ${\mathcal{R}e}(\alpha_{j})>0$ for each $j$ since ${\mathcal{R}e}(\lambda_{j})<1$ for each $j$. Then, if we take an integer $m_{0}\geq 2$ large enough such that ${\mathcal{R}e}(\alpha_{j})r_{n}<1$ for all $j$ and $n\geq m_{0}$, we can write

[TABLE]

where

[TABLE]

Notice that the blocks $A^{11}_{k+1,n}$, $A^{31}_{k+1,n}$ and $A^{33}_{k+1,n}$ are all diagonal $(N-1)\times(N-1)$ matrices. In particular, setting for any $x\in\mathbb{C}$, $p_{m_{0}-1}(x):=1$ and $p_{k}(x):=\prod_{m=m_{0}}^{k}(1-r_{m}x)$ for $k\geq m_{0}$ and $F_{k+1,n}(x):=p_{n}(x)/p_{k}(x)$ for $m_{0}-1\leq k\leq n-1$, from Lemma A.5 we get

[TABLE]

Finally, we rewrite (54) as

[TABLE]

and, in the sequel of the proof, we will establish the asymptotic behavior of $\boldsymbol{\theta}_{n}$ by studying separately the terms $C_{m_{0},n}\boldsymbol{\theta}_{m_{0}}$, $\sum_{k=m_{0}}^{n}{\mathbf{T}}_{n,k}$ and $\boldsymbol{\rho}_{n}$.

Let us use the symbol ∗ for the quantities $a_{\alpha_{j}}:={\mathcal{R}e}(\alpha_{j})$ and $p_{n}(\alpha_{j})$ corresponding to $\alpha^{*}=\alpha_{j}=1-\lambda_{j}$ with $\lambda_{j}=\lambda^{*}\in\lambda_{\max}(D)$. Now, we note that, as a consequence of (55), (56) and Lemma A.3, we have

[TABLE]

Therefore, we get that $\sqrt{n}|C_{m_{0},n}\boldsymbol{\theta}_{m_{0}}|\rightarrow 0$ almost surely because $ca^{*}>1/2$ by assumption.

Concerning the term $\boldsymbol{\rho}_{n}$, notice that by (10) and (50) we have that $|\mathbf{R}_{k}|=O(k^{-1})$ and, by (58), we have that

[TABLE]

Therefore, since $\boldsymbol{\rho}_{n}=\sum_{k=m_{0}}^{n}r_{k}C_{k+1,n}{\mathbf{R}}_{\theta,k+1}=\sum_{k=m_{0}}^{n-1}r_{k}C_{k+1,n}{\mathbf{R}}_{\theta,k+1}+r_{n}C_{n+1,n}{\mathbf{R}}_{\theta,n+1}$, it follows (by (69)) that

[TABLE]

because $ca^{*}>1/2$.

We now focus on the asymptotic behavior of the second term. Specifically, we aim at proving that $\sqrt{n}\sum_{k=m_{0}}^{n}{\mathbf{T}}_{n,k}$ converges stably to the suitable Gaussian kernel. For this purpose, we set ${\mathcal{G}}_{n,k}={\mathcal{F}}_{k+1}$, and consider Theorem B.1 (recall that ${\mathbf{T}}_{n,k}$ are real random vectors). Given the fact that condition $(c1)$ of Theorem B.1 is obviously satisfied, we will check only conditions $(c2)$ and $(c3)$.

Regarding condition $(c2)$, since the relation $V_{\theta}^{\top}U_{\theta}=I$ implies $V_{\theta}^{\top}R=S_{R}V_{\theta}^{\top}$, we have that

[TABLE]

Therefore, it is enough to study the convergence of

[TABLE]

Moreover, since $O(nr_{n}^{2})=O(n^{-1})\to 0$ the last term in the above sum is negligible as $n$ increase to infinity, and hence it is enough to study the convergence of

[TABLE]

To this purpose, setting $B_{\theta,k+1}:=V_{\theta}^{\top}\,(\Delta{\mathbf{M}}_{\theta,k+1})(\Delta{\mathbf{M}}_{\theta,k+1})^{\top}\,V_{\theta}$, $B_{k+1}:=V^{\top}\,(\Delta{\mathbf{M}}_{k+1})(\Delta{\mathbf{M}}_{k+1})^{\top}\,V$, $\mathbf{b}_{k+1}:=V^{\top}\,(\Delta{\mathbf{M}}_{k+1})(\Delta{\mathbf{M}}_{k+1})^{\top}\,\mathbf{v}_{1}$ and $b_{k+1}:=\mathbf{v}_{1}^{\top}\,(\Delta{\mathbf{M}}_{k+1})(\Delta{\mathbf{M}}_{k+1})^{\top}\,\mathbf{v}_{1}$, we observe that

[TABLE]

Since in $B_{\theta,k+1}$ the first and the third row and column of blocks are the same, in (59) the $(2N-1)\times(2N-1)$ matrix $(A_{k+1,n}S_{R})$ can be rewritten as a diagonal matrix with the following diagonal blocks: $A^{1}_{k+1,n}:=A^{11}_{k+1,n}$, $A^{3}_{k+1,n}:=(A^{31}_{k+1,n}+c^{-1}A^{33}_{k+1,n})$ and $a^{2}_{k+1,n}:=(c^{-1}-1)a^{22}_{k+1,n}$. Hence, the expression in (59) can be rewritten as

[TABLE]

The elements of $A^{1}_{k+1,n}$, $a^{2}_{k+1,n}$ and $A^{3}_{k+1,n}$ in the above matrix can be rewritten in terms of $F_{k+1,n}(\cdot)$, by (56), in the following way:

[TABLE]

Hence, the almost sure convergences of all the elements in (61) can be obtained by combining the results of the following limits:

[TABLE]

for certain complex numbers $x,y\in\{\alpha_{j},\,2\leq j\leq N\}$ (remember that, by the assumption ${\mathcal{R}e}(\lambda^{*})<1-(2c)^{-1}$, we have $c(a_{x}+a_{y})>1$ with $a_{x}:={\mathcal{R}e}(x)$ and $a_{y}:={\mathcal{R}e}(y)$), a suitable sequence of random variables $\beta_{k}\in\{[B_{k}]_{h,j},[\mathbf{b}_{k}]_{j},b_{k};\,2\leq h,j\leq N\}$ and some random variable $\beta$. Indeed, using Lemma A.3 and relation (69), we have

• (1)

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}|\beta_{k+1}|O(n^{-2})\ =\ O(n^{-1})\sum_{k=m_{0}}^{n-1}O(k^{-2})\ \rightarrow 0$;

• (2)

$n\ln(n)\sum_{k=m_{0}}^{n-1}r_{k}^{2}|\beta_{k+1}|O(n^{-1})|F_{k+1,n}(c^{-1})|\ =\ O(n^{-1}\ln(n))\sum_{k=m_{0}}^{n-1}O(k^{-1})\ \rightarrow 0$;

• (3)

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}|\beta_{k+1}|O(n^{-1})|F_{k+1,n}(y)|\ =\ O(n^{-ca_{y}})\sum_{k=m_{0}}^{n-1}O(k^{-(2-ca_{y})})\ \rightarrow 0$.

In order to prove the convergences in (63), we will apply Lemma A.2 to each of the three limits. Indeed, each quantity in (63) can be written as $\sum_{k=m_{0}}^{n-1}v^{(e)}_{n,k}Y_{k}/c_{k}$, where

[TABLE]

satisfy the assumptions of Lemma A.2. More precisely, setting ${\mathcal{H}}_{n}={\mathcal{F}}_{n+1}$ we have

[TABLE]

because, by (43), we get that

[TABLE]

Moreover, we have

[TABLE]

In addition, since $|v^{(e)}_{n,k}|/c_{k}=nr_{k}^{2}\ln^{e}(n/k)|F_{k+1,n}(x)F_{k+1,n}(y)|$, from (75) in Lemma A.4 (with $u=1$) it follows that $\sum_{k=m_{0}}^{n-1}\frac{|v^{(e)}_{n,k}|}{c_{k}}=O(1)$. Analogously, using again Lemma A.4, we can prove that $\sum_{k=m_{0}}^{n-1}|v^{(e)}_{n,k}-v^{(e)}_{n,k-1}|=O(1)$ since by Remark A.1 we have

[TABLE]

Hence, condition (68) in Lemma A.2 is satisfied and so, in order to apply this lemma, it only remains to prove condition (67). To this end, we get the values of $\lim_{n}\sum_{k=m_{0}}^{n}v^{(e)}_{n,k}/c_{k}$ by (73) in Lemma A.4, and we observe that $\lim_{n}v^{(e)}_{n,n}=s\in\{0,1\}$ and, for a fixed $k$, $\lim_{n}|v^{(e)}_{n,k}|=0$ since by Lemma A.3 we have $|p_{n}(x)p_{n}(y)|=O(n^{-c(a_{x}+a_{y})})=o((n\ln^{e}(n))^{-1})$.

Now that we have proved the convergences in (63), we can use the relations in (62) to compute the almost sure limits of all the elements in (61). The results are listed below, while the technical computations are reported in Appendix A.3.1.

• •

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{1}_{k+1,n}B_{k+1}A^{1}_{k+1,n}]_{h,j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}\frac{c^{2}}{c(\alpha_{h}+\alpha_{j})-1}(\mathbf{v}_{h}^{\top}\mathbf{v}_{j})Z_{\infty}(1-Z_{\infty});$

• •

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{3}_{k+1,n}B_{k+1}A^{3}_{k+1,n}]_{h,j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}\frac{1+(c-1)(\alpha_{h}^{-1}+\alpha_{j}^{-1})}{c(\alpha_{h}+\alpha_{j})-1}(\mathbf{v}_{h}^{\top}\mathbf{v}_{j})Z_{\infty}(1-Z_{\infty})$;

• •

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}(a^{2}_{k+1,n})^{2}b_{k+1}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}(c-1)^{2}\|\mathbf{v}_{1}\|^{2}Z_{\infty}(1-Z_{\infty})$;

• •

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{1}_{k+1,n}B_{k+1}A^{3}_{k+1,n}]_{h,j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}\frac{\alpha_{h}^{-1}(c-1)+c}{c(\alpha_{h}+\alpha_{j})-1}(\mathbf{v}_{h}^{\top}\mathbf{v}_{j})Z_{\infty}(1-Z_{\infty})$;

• •

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}a^{2}_{k+1,n}[\mathbf{b}_{k+1}^{\top}A^{1}_{k+1,n}]_{j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}\frac{1-c}{\alpha_{j}}(\mathbf{v}_{1}^{\top}\mathbf{v}_{j})Z_{\infty}(1-Z_{\infty})$;

• •

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}a^{2}_{k+1,n}[\mathbf{b}_{k+1}^{\top}A^{3}_{k+1,n}]_{j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}\frac{1-c}{\alpha_{j}}(\mathbf{v}_{1}^{\top}\mathbf{v}_{j})Z_{\infty}(1-Z_{\infty})$.

Hence, recalling the definitions of the matrices $\widehat{S}_{\mathbf{ZZ}},\widehat{S}_{\mathbf{NN}}$ and $\widehat{S}_{\mathbf{ZN}}$ given in (18), (20), (21), (23) and (24), we obtain

[TABLE]

Therefore, using (61), we can state that

[TABLE]

where the last matrix coincides with the one in the statement of the theorem because of (17), (19) and (22).

Regarding condition $(c_{3})$, we observe that, using the inequalities

[TABLE]

with a suitable constant $K$, we find for any $u>1$

[TABLE]

where, for the last equality, we have used (58). Now, since $2ca^{*}>1$, by (75) in Lemma A.4 (with $x=y=\alpha^{*}=1-\lambda^{*}$, $e=0$ and $u>1$), we have

[TABLE]

which, in particular, implies $(\sup_{m_{0}\leq k\leq n}|\sqrt{n}\mathbf{T}_{n,k}|)^{2u}\stackrel{{\scriptstyle L^{1}}}{{\longrightarrow}}0$ for any $u>1$. As a consequence of the above convergence to zero, condition (c3) of Theorem B.1 holds true.

Summing up, all the conditions required by Theorem B.1 are satisfied and so we can apply this theorem and obtain the stable convergence of $\sqrt{n}\sum_{k=m_{0}}^{n}\mathbf{T}_{n,k}$ to the Gaussian kernel with random covariance matrix defined in Theorem 4.3. ∎

Now, we are ready to prove Theorem 3.4.

Proof of Theorem 3.4. By Theorem 4.1, we have that

[TABLE]

Thus, from Theorem 4.3, applying Theorem B.2, we obtain that

[TABLE]

stably. In order to conclude, it is enough to observe that

[TABLE]

where $\Phi(x,y,z)=(x+z,y+z)^{\top}$. ∎

4.6. Proof of Theorem 3.5

(CLT for $({\mathbf{Z}}_{n},{\mathbf{N}}_{n})_{n}$ in the case $N\geq 2$, $\gamma=1$ and ${\mathcal{R}e}(\lambda^{*})=1-(2c)^{-1}$)

As above, in order to prove Theorem 3.5, we need the following convergence result on $(\widehat{\mathbf{Z}}_{n},\widehat{\mathbf{N}}_{n})_{n}$:

Theorem 4.4**.**

Let $N\geq 2$, $\gamma=1$ and ${\mathcal{R}e}(\lambda^{*})=1-(2c)^{-1}$. Then, under condition (10), we have that

[TABLE]

where $\widehat{\Sigma}^{*}_{\mathbf{ZZ}}$, $\widehat{\Sigma}^{*}_{\mathbf{NN}}$ and $\widehat{\Sigma}^{*}_{\mathbf{ZN}}$ are the matrices defined in (26), (28) and (30), respectively.

Proof.

The proof of Theorem 4.4 follows analogous arguments to those used in Theorem 4.3. In particular, consider the joint dynamics of $\boldsymbol{\theta}_{n}:=(\widehat{\mathbf{Z}}_{n},\widehat{\mathbf{N}}_{n})^{\top}$ defined in (57) as follows:

[TABLE]

where $C_{k+1,n}$ is defined in (55), $R$ is defined in (52), $\Delta\mathbf{M}_{\theta,n}=(\Delta\mathbf{M}_{n},\Delta\mathbf{M}_{n})^{\top}$ and $\mathbf{R}_{\theta,n}=(\mathbf{0},\mathbf{R}_{n})^{\top}$ with $\mathbf{R}_{n}$ defined in (50). Then, we are going to prove that $\sqrt{n/\ln(n)}\sum_{k=m_{0}}^{n}{\mathbf{T}}_{n,k}$ converges stably to the desired Gaussian kernel, while $\sqrt{n/\ln(n)}|C_{m_{0},n}\boldsymbol{\theta}_{m_{0}}|$ and $\sqrt{n/\ln(n)}|\boldsymbol{\rho}_{n}|$ converge almost surely to zero.

First, note that by (58), we have that

[TABLE]

where, as before, the symbol ∗ refers to the quantities $a_{\alpha_{j}}:={\mathcal{R}e}(\alpha_{j})$ and $p_{n}(\alpha_{j})$ corresponding to $\alpha^{*}=\alpha_{j}=1-\lambda_{j}$ with $\lambda_{j}=\lambda^{*}\in\lambda_{\max}(D)$, and hence the last passage follows since $ca^{*}=1/2$ by assumption. As a consequence, we obtain

[TABLE]

and so $\sqrt{n/\ln(n)}|C_{m_{0},n}\boldsymbol{\theta}_{m_{0}}|\rightarrow 0$ almost surely.

Concerning the term $\boldsymbol{\rho}_{n}$, notice that by (10) and (50) we have that $|\mathbf{R}_{k}|=O(k^{-1})$ and, by (64), we have that

[TABLE]

Therefore, since $\boldsymbol{\rho}_{n}=\sum_{k=m_{0}}^{n}r_{k}C_{k+1,n}{\mathbf{R}}_{\theta,k+1}=\sum_{k=m_{0}}^{n-1}r_{k}C_{k+1,n}{\mathbf{R}}_{\theta,k+1}+r_{n}C_{n+1,n}{\mathbf{R}}_{\theta,n+1}$, it follows (by (69)) that

[TABLE]

We now focus on the proof of the fact that $\sqrt{n/\ln(n)}\sum_{k=m_{0}}^{n}{\mathbf{T}}_{n,k}$ converges stably to the suitable Gaussian kernel. For this purpose, we set ${\mathcal{G}}_{n,k}={\mathcal{F}}_{k+1}$, and consider Theorem B.1. Given the fact that condition $(c1)$ of Theorem B.1 is obviously satisfied, we will check only conditions $(c2)$ and $(c3)$.

Regarding condition $(c2)$, from the computations seen in the proof of Theorem 4.3 and using the fact that $O(nr_{n}^{2}/\ln(n))=O(n^{-1}/\ln(n))\to 0$, we have

[TABLE]

Then, setting $B_{\theta,k+1}$ as in (60), the limit of the above expression can be obtain by studying the convergence of the following matrix:

[TABLE]

where $A^{1}_{k+1,n},a^{2}_{k+1,n},A^{3}_{k+1,n}$ are defined in (62). Notice that the almost sure convergences of all the elements in (65) can be obtained by combining the results of the following limits:

[TABLE]

for certain complex numbers $x,y\in\{\alpha_{j},\,2\leq j\leq N\}$ with $a_{x}:={\mathcal{R}e}(x)$, $b_{x}:={\mathcal{I}m}(x)$, $a_{y}:={\mathcal{R}e}(y)$ and $b_{y}:={\mathcal{I}m}(y)$ (remember that, by the assumption on ${\mathcal{R}e}(\lambda^{*})$, we can have both cases $c(a_{x}+a_{y})>1$ and $c(a_{x}+a_{y})=1$), a suitable sequence of random variables $\beta_{k}\in\{[B_{k}]_{h,j},[\mathbf{b}_{k}]_{j},b_{k};\,2\leq h,j\leq N\}$ and some random variable $\beta$.

In order to prove the convergence in (66) for the case $c(a_{x}+a_{y})>1$, we can use the convergences in (63) established in the proof of Theorem 4.3; while for the case $c(a_{x}+a_{y})=1$ we can apply Lemma A.2 since each quantity in (66) can be written as $\sum_{k=m_{0}}^{n-1}v_{n,k}Y_{k}/c_{k}$, where

[TABLE]

satisfy the assumptions of Lemma A.2. Indeed, similarly as in the proof of Theorem 4.3, we have

[TABLE]

In addition, since $|v_{n,k}|/c_{k}=(n/\ln(n))r_{k}^{2}|F_{k+1,n}(x)F_{k+1,n}(y)|$, from (74) in Lemma A.4 (with $u=1$) it follows that $\sum_{k=m_{0}}^{n-1}\frac{|v_{n,k}|}{c_{k}}=O(1)$. Moreover, we have that $\sum_{k=m_{0}}^{n-1}|v_{n,k}-v_{n,k-1}|=O(1)$ since by Remark A.1 we have

[TABLE]

Hence, condition (68) of Lemma A.2 is satisfied and so, in order to apply this lemma, it only remains to prove condition (67). To this end, we get the value of $\lim_{n}\sum_{k=m_{0}}^{n-1}v_{n,k}/c_{k}$ from (72) in Lemma A.4, and we observe that $\lim_{n}v_{n,n}=0$ and, for a fixed $k$, $\lim_{n}|v_{n,k}|=0$ since by Lemma A.3 we have $|p_{n}(x)p_{n}(y)|=O(n^{-1})$.

Now that we have proved the convergences in (66), we can use the relations in (62) to compute the almost sure limits of all the elements in (65). The results are listed below, while the technical computations are reported in Appendix A.3.2.

• •

$\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{1}_{k+1,n}B_{k+1}A^{1}_{k+1,n}]_{h,j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}(\mathbf{v}_{h}^{\top}\mathbf{v}_{j})Z_{\infty}(1-Z_{\infty})c^{2}\mathbbm{1}_{\{b_{\alpha_{h}}+b_{\alpha_{j}}=0\}}$;

• •

$\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{3}_{k+1,n}B_{k+1}A^{3}_{k+1,n}]_{h,j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}(\mathbf{v}_{h}^{\top}\mathbf{v}_{j})Z_{\infty}(1-Z_{\infty})\frac{(\alpha_{h}-1)(\alpha_{j}-1)}{\alpha_{h}\alpha_{j}}\mathbbm{1}_{\{b_{\alpha_{h}}+b_{\alpha_{j}}=0\}}$;

• •

$\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}b_{k+1}(a^{2}_{k+1,n})^{2}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}0$;

• •

$\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{1}_{k+1,n}B_{k+1}A^{3}_{k+1,n}]_{h,j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}(\mathbf{v}_{h}^{\top}\mathbf{v}_{j})Z_{\infty}(1-Z_{\infty})\frac{c(1-\alpha_{j})}{\alpha_{h}}\mathbbm{1}_{\{b_{\alpha_{h}}+b_{\alpha_{j}}=0\}}$;

• •

$\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}a^{2}_{k+1,n}[\mathbf{b}_{k+1}^{\top}A^{1}_{k+1,n}]_{j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}0$;

• •

$\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}a^{2}_{k+1,n}[\mathbf{b}_{k+1}^{\top}A^{3}_{k+1,n}]_{j}\stackrel{{\scriptstyle a.s}}{{\longrightarrow}}0$.

Hence, recalling the definitions of the matrices $\widehat{S}^{*}_{\mathbf{ZZ}},\widehat{S}^{*}_{\mathbf{NN}}$ and $\widehat{S}^{*}_{\mathbf{ZN}}$ given in (27), (29) and (31), we obtain

[TABLE]

Therefore, using (65), we can state that

[TABLE]

where the last matrix coincides with the one in the statement of the theorem because of (26), (28) and (30).

Regarding condition $(c_{3})$, we observe that, using the inequalities

[TABLE]

with a suitable constant $K$, we find for any $u>1$

[TABLE]

where, for the last equality, we have used (64). Now, since $2ca^{*}=1$, by (74) in Lemma A.4 (with $x=y=\alpha^{*}=1-\lambda^{*}$ and $u>1$), we have

[TABLE]

which, in particular, implies $(\sup_{m_{0}\leq k\leq n}|\sqrt{(n/\ln(n))}\mathbf{T}_{n,k}|)^{2u}\stackrel{{\scriptstyle L^{1}}}{{\longrightarrow}}0$ for any $u>1$. As a consequence of the above convergence to zero, condition (c3) of Theorem B.1 holds true.

Summing up, all the conditions required by Theorem B.1 are satisfied and so we can apply this theorem and obtain the stable convergence of $\sqrt{n/\ln(n)}\sum_{k=m_{0}}^{n}\mathbf{T}_{n,k}$ to the Gaussian kernel with random covariance matrix defined in Theorem 4.4. ∎

Now, we are ready to prove Theorem 3.5.

Proof of Theorem 3.5. By Theorem 4.1, we have that

[TABLE]

Moreover, from Theorem 4.4, we have that

[TABLE]

In order to conclude, it is enough to observe that

[TABLE]

where the last term converges in probability to zero. ∎

Acknowledgments

Irene Crimaldi and Andrea Ghiglietti are members of the Italian group “Gruppo Nazionale per l’Analisi Matematica, la Probabilità e le loro Applicazioni (GNAMPA)” of the Italian Institute “Istituto Nazionale di Alta Matematica (INdAM)”.

Giacomo Aletti is a member of the Italian group “Gruppo Nazionale per il Calcolo Scientifico (GNCS)” of the Italian Institute “Istituto Nazionale di Alta Matematica (INdAM)”.

Appendix

Appendix A Some technical results

In all the sequel, given $(a_{n})_{n},(b_{n})_{n}$ two sequences of real numbers with $b_{n}\geq 0$, the notation $a_{n}=O(b_{n})$ means $|a_{n}|\leq Cb_{n}$ for a suitable constant $C>0$ and $n$ large enough. Therefore, if we also have $a_{n}^{-1}=O(b_{n}^{-1})$, then $C^{\prime}b_{n}\leq|a_{n}|\leq Cb_{n}$ for suitable constants $C,C^{\prime}>0$ and $n$ large enough. Moreover, given $(z_{n})_{n},(z^{\prime}_{n})_{n}$ two sequences of complex numbers, with $z^{\prime}_{n}\neq 0$, the notation $z_{n}=o(z^{\prime}_{n})$ means $\lim_{n}z_{n}/z^{\prime}_{n}=0$.

A.1. Asymptotic results for sums of complex numbers

We start recalling an extension of the Toeplitz lemma (see [35]) to complex numbers provided in [2], from which we get useful technical results employed in our proofs.

Lemma A.1**.**

[2*, Lemma A.2]** (Generalized Toeplitz lemma)

Let $\{z_{n,k}:\,1\leq k\leq k_{n}\}$ be a triangular array of complex numbers such that*

• i)

$\lim_{n}z_{n,k}=0$* for each fixed $k$;*

• ii)

$\lim_{n}\sum_{k=1}^{k_{n}}z_{n,k}=s\in\{0,1\}$;

• iii)

$\sum_{k=1}^{k_{n}}|z_{n,k}|=O(1)$.

Let $(w_{n})_{n}$ be a sequence of complex numbers with $\lim_{n}w_{n}=w\in{\mathbb{C}}$. Then, we have $\lim_{n}\sum_{k=1}^{k_{n}}z_{n,k}w_{k}=sw$.

From this lemma we can easily get the following corollary, which slightly extends the generalized version of the Kronecker lemma provided in [2, Corollary A.3]:

Corollary A.1**.**

*(Generalized Kronecker lemma)

Let $\{v_{n,k}:1\leq k\leq n\}$ and $(z_{n})_{n}$ be respectively a triangular array and a sequence of complex numbers such that $v_{n,k}\neq 0$ and*

[TABLE]

and $\sum_{n}z_{n}$ is convergent. Then

[TABLE]

Proof.

Without loss of generality, we can suppose $\lim_{n}v_{n,n}=s\in\{0,1\}$. Set $w_{n}=\sum_{k=n}^{+\infty}z_{k}$ and observe that, since $\sum_{n}z_{n}$ is convergent, we have $\lim_{n}w_{n}=w=0$ and, moreover, we can write

[TABLE]

The second and the third term obviously converge to zero. In order to prove that the first term converges to zero, it is enough to apply Lemma A.1 with $z_{n,k}=v_{n,k}-v_{n,k-1}$. ∎

The above corollary is useful to get the following result for complex random variables, which again slightly extends the version provided in [2, Lemma A.3]:

Lemma A.2**.**

Let ${\mathcal{H}}=({\mathcal{H}}_{n})_{n}$ be a filtration and $(Y_{n})_{n}$ a $\mathcal{H}$-adapted sequence of complex random variables such that $E[Y_{n}|{\mathcal{H}}_{n-1}]\to Y$ almost surely. Moreover, let $(c_{n})_{n}$ be a sequence of strictly positive real numbers such that $\sum_{n}E\left[|Y_{n}|^{2}\right]/c_{n}^{2}<+\infty$ and let $\{v_{n,k},1\leq k\leq n\}$ be a triangular array of complex numbers such that $v_{n,k}\neq 0$ and

[TABLE]

[TABLE]

Then $\sum_{k=1}^{n}v_{n,k}Y_{k}/c_{k}\stackrel{{\scriptstyle a.s.}}{{\longrightarrow}}\eta Y$.

Proof.

Let $A$ be an event such that $P(A)=1$ and $\lim_{n}E[Y_{n}|{\mathcal{H}}_{n-1}](\omega)=Y(\omega)$ for each $\omega\in A$. Fix $\omega\in A$ and set $w_{n}=E[Y_{n}|{\mathcal{H}}_{n-1}](\omega)$ and $w=Y(\omega)$. If $\eta\neq 0$, applying Lemma A.1 to $z_{n,k}=v_{n,k}/(c_{k}\eta)$, $s=1$ and $w_{n}$, we obtain

[TABLE]

If $\eta=0$, applying Lemma A.1 to $z_{n,k}=v_{n,k}/c_{k}$, $s=0$ and $w_{n}$, we obtain

[TABLE]

Therefore, for both cases, we have

[TABLE]

Now, consider the martingale $(M_{n})_{n}$ defined by

[TABLE]

It is bounded in $L^{2}$ since $\sum_{k=1}^{n}\frac{E[|Y_{k}|^{2}]}{c_{k}^{2}}<+\infty$ by assumption and so it is almost surely convergent, that means

[TABLE]

for $\omega\in B$ with $P(B)=1$. Therefore, fixing $\omega\in B$ and setting $z_{k}=\frac{Y_{k}(\omega)-E[Y_{k}|{\mathcal{H}}_{k-1}](\omega)}{c_{k}}$, by Corollary A.1, we get

[TABLE]

and so

[TABLE]

In order to conclude, it is enough to observe that

[TABLE]

∎

We conclude this subsection recalling the following well-known relations for $a\in{\mathbb{R}}$:

[TABLE]

More precisely, in the case $a=0$, we have

[TABLE]

where $d$ denotes the Euler-Mascheroni constant.

A.2. Asymptotic results for products of complex numbers

Fix $\gamma=1$ and $c>0$, and consider a sequence $(r_{n})_{n}$ of real numbers such that $0\leq r_{n}<1$ for each $n$ and

[TABLE]

Obviously, we have $r_{n}>0$ for $n$ large enough and so in the sequel, without loss of generality, we will assume $0<r_{n}<1$ for all $n$.

Let $x=a_{x}+i\,b_{x}\in{\mathbb{C}}$ and $y=a_{y}+i\,b_{y}\in{\mathbb{C}}$ with $a_{x},a_{y}>0$ and $c(a_{x}+a_{y})\geq 1$. Denote by $m_{0}\geq 2$ an integer such that $\max\{a_{x},\,a_{y}\}r_{m}<1$ for all $m\geq m_{0}$ and set:

[TABLE]

We recall the following result, which has been proved in [2]:

Lemma A.3**.**

[2, Lemma A.4]** We have that

[TABLE]

Inspired by the computation done in [2, 21], we can prove the following other technical result:

Lemma A.4**.**

(i) When $c(a_{x}+a_{y})=1$, we have

[TABLE]

while when $c(a_{x}+a_{y})>1$, we have

[TABLE]

*(ii) Moreover, for any $u\geq 1$, we have:

when $c(a_{x}+a_{y})=1$*

[TABLE]

while when $c(a_{x}+a_{y})>1$ and $e\in\{0,1,2\}$

[TABLE]

(note that for $u=1$ only the third case is possible).

Proof.

(i) First of all, let us notice that the limit (72) and the first of the limits (73) have already been proved in [2, Eq. (A.11),(A.18)]. Therefore, we can focus on the second and the third limits in (73). To this end, let us set

[TABLE]

so that, recalling the equality $F_{k+1,n}(x)=p_{n}(x)/p_{k}(x)$, we can write:

[TABLE]

Now, set $G_{1,k}:=c^{2}/[kp_{k}(x)p_{k}(y)]$ and recall that, as seen in [2, Proof of Lemma A.5], when $c(a_{x}+a_{y})>1$ we have

[TABLE]

Using analogous arguments, we can set $G_{2,k}:=c^{2}\ln(k)/[kp_{k}(x)p_{k}(y)]$ and observe that we have:

[TABLE]

Therefore, when $c(a_{x}+a_{y})>1$, we obtain

[TABLE]

The relations (76), (77) and the first limit in (73) imply

[TABLE]

where we have used the fact that, by Lemma A.3 and relation (69), we have

[TABLE]

For the last limit, we can set $G_{3,k}:=c^{2}\ln^{2}(k)/[kp_{k}(x)p_{k}(y)]$ and, similarly as above, observe that we have:

[TABLE]

Therefore, when $c(a_{x}+a_{y})>1$, we obtain

[TABLE]

By means of analogous computations as above, the relations (76), (77), (78) and the already proved second limit in (73) imply

[TABLE]

where we have used the fact that, by Lemma A.3 and relation (69), we have

[TABLE]

ii) For the second part of the proof, note that by condition (71) on $(r_{n})_{n}$, relation (69) and Lemma A.3, when $c(a_{x}+a_{y})=1$, we have

[TABLE]

For the case $c(a_{x}+a_{y})>1$, note that for $u\geq 1$ and $e\in\{0,1,2\}$, we have

[TABLE]

Then, for $e=0$, using relation (69), it is easy to see that

[TABLE]

(note that for $u=1$ only the third case is possible).

Now we consider the cases $e=1$ and $e=2$. Note that, setting $\alpha:=2u-uc(a_{x}+a_{y})\in\mathbb{R}$ and $\beta:=eu\geq 1$, we have that

[TABLE]

where $\epsilon\in(0,1)$ has been chosen such that $g(x)=x^{-\alpha}\ln^{\beta}(x^{-1})$ is monotone in $(0,\epsilon]$ and we recall that $(m_{0}-1)\geq 1$. Then, we have that

[TABLE]

Finally, we can conclude that, for the cases $e=1$ and $e=2$, we have

[TABLE]

(note again that for $u=1$ only the third case is possible). ∎

Remark A.1**.**

Setting $v^{(e)}_{n,k}:=(n/k)\ln^{e}(n/k)F_{k+1,n}(x)F_{k+1,n}(y)$ for any $e\in\{0,1,2\}$ and $m_{0}-1\leq k\leq n-1$, and using the relations (76), (77), (78) found in the proof of Lemma A.4, for $c(a_{x}+a_{y})>1$ we have:

[TABLE]

Moreover, setting $v_{n,k}:=v^{(0)}_{n,k}/\ln(n)$ for any $m_{0}-1\leq k\leq n-1$, in the case $c(a_{x}+a_{y})=1$ we have: $|v_{n,k}-v_{n,k-1}|=O\left(r_{k}^{2}k/\ln(n)\right)$ when $b_{x}+b_{y}\neq 0$ since Lemma A.3 and

[TABLE]

while $|v_{n,k}-v_{n,k-1}|=O\left(r_{k}^{2}/\ln(n)\right)$ when $b_{x}+b_{y}=0$ since Lemma A.3 and

[TABLE]

A.3. Technical computations for the proofs of Theorem 4.3 and

Theorem 4.4

In this subsection we collect some technical computations necessary for the proofs of Theorem 4.3 and Theorem 4.4. Therefore, the notation and the assumptions used here are the same as those used in these theorems.

The first technical result is the following:

Lemma A.5**.**

Let the matrix $A_{k+1,n}$ be defined as in (55) for $m_{0}-1\leq k\leq n-1$. Then, we have that

[TABLE]

Proof.

By means of (53) and (55), after standard calculations, the elements in $A_{k+1,n}$ for $m_{0}-1\leq k\leq n-1$ can be written as follows: $[A^{11}_{k+1,n}]_{jj}=F_{k+1,n}(\alpha_{j})$, $[A^{33}_{k+1,n}]_{jj}=a^{22}_{k+1,n}=F_{k+1,n}(c^{-1})$ and

[TABLE]

where

[TABLE]

Setting $\Delta X^{j}_{l}:=(X^{j}_{l}-X^{j}_{l-1})$, notice that we have

[TABLE]

Hence, in the case $c\alpha_{j}\neq 1$, we have that

[TABLE]

which implies

[TABLE]

Using the above expression of $S^{j}_{k+1,n}$ in the definition of $A^{31}_{k+1,n}$, we obtain (for $c\alpha_{j}\neq 1$) that

[TABLE]

When $c\alpha_{j}=1$, observing that $X^{j}_{l}=1$ for any $l\geq 1$ and using condition (71) we get

[TABLE]

where, for the last equality, we have used the fact that $k<n$ and $\sum_{l\geq k}1/l^{2}=O(1/k)$. Then, using (70) for $a=0$, we have

[TABLE]

(where the last passage follows again by the fact that $k<n$). Finally, since Lemma A.3 we have $|F_{k+1,n}(c^{-1})|=O(k/n)$, we obtain (for $c\alpha_{j}=1$) that

[TABLE]

∎

A.3.1. Computations for the almost sure limits of the elements in (61)

• •

$a.s.-\lim_{n}n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{1}_{k+1,n}B_{k+1}A^{1}_{k+1,n}]_{h,j}$:

By using the first limit in (63), we have

[TABLE]

• •

$a.s.-\lim_{n}n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{3}_{k+1,n}B_{k+1}A^{3}_{k+1,n}]_{h,j}$:

First, note that when $c\alpha_{h}\neq 1$ and $c\alpha_{j}\neq 1$, we have that $n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[B_{k+1}]_{h,j}[A^{3}_{k+1,n}]_{h,h}[A^{3}_{k+1,n}]_{j,j}$ has the same limit as

[TABLE]

Then, when $c\alpha_{h}\neq 1$ and $c\alpha_{j}\neq 1$, using the first limit in (63) we obtain, after some standard calculations,

[TABLE]

When $c\alpha_{h}=c\alpha_{j}=1$, we have that $n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[B_{k+1}]_{h,j}[A^{3}_{k+1,n}]_{h,h}[A^{3}_{k+1,n}]_{j,j}$ has the same limit as

[TABLE]

from which, using the three limits in (63), we obtain

[TABLE]

Finally, when $c\alpha_{h}\neq 1$ and $c\alpha_{j}=1$, we have that $n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[B_{k+1}]_{h,j}[A^{3}_{k+1,n}]_{h,h}[A^{3}_{k+1,n}]_{j,j}$ has the same limit as

[TABLE]

which implies, using the first two limits in (63), that

[TABLE]

The case $c\alpha_{h}=1$ and $c\alpha_{j}\neq 1$ is analogous. Therefore, we can summarize the limits in all the above cases with the formula:

[TABLE]

• •

$a.s.-\lim_{n}n\sum_{k=m_{0}}^{n-1}r_{k}^{2}(a^{2}_{k+1,n})^{2}b_{k+1}$:

Using the first limit in (63), we have

[TABLE]

• •

$a.s.-\lim_{n}n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{1}_{k+1,n}B_{k+1}A^{3}_{k+1,n}]_{h,j}$:

First, when $c\alpha_{j}\neq 1$ notice that $n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[B_{k+1}]_{h,j}[A^{1}_{k+1,n}]_{h,h}[A^{3}_{k+1,n}]_{j,j}$ has the same limit as

[TABLE]

and hence, after standard calculations, we obtain

[TABLE]

When $c\alpha_{j}=1$, $n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[B_{k+1}]_{h,j}[A^{1}_{k+1,n}]_{h,h}[A^{3}_{k+1,n}]_{j,j}$ has the same limit as

[TABLE]

and hence

[TABLE]

Therefore we can summarize the limits of the above two cases with the formula

[TABLE]

• •

$a.s.-\lim_{n}n\sum_{k=m_{0}}^{n-1}r_{k}^{2}a^{2}_{k+1,n}[\mathbf{b}_{k+1}^{\top}A^{1}_{k+1,n}]_{j}$:

Notice that

[TABLE]

which implies that

[TABLE]

• •

$a.s.-\lim_{n}n\sum_{k=m_{0}}^{n-1}r_{k}^{2}a^{2}_{k+1,n}[\mathbf{b}_{k+1}^{\top}A^{3}_{k+1,n}]_{j}$:

First, when $c\alpha_{j}\neq 1$, notice that

$n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[\mathbf{b}_{k+1}]_{j}[A^{3}_{k+1,n}]_{jj}a^{2}_{k+1,n}$ has the same limit as

[TABLE]

which implies after some calculations

[TABLE]

When $c\alpha_{j}=1$, $n\sum_{k=m_{0}}^{n-1}r_{k}^{2}[\mathbf{b}_{k+1}]_{j}[A^{3}_{k+1,n}]_{jj}a^{2}_{k+1,n}$ has the same limit as

[TABLE]

from which we can obtain

[TABLE]

Therefore, we can summarize the limits of the above two cases with the formula

[TABLE]

A.3.2. Computations for the almost sure limits of the elements in (65)

• •

$a.s.-\lim_{n}\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{1}_{k+1,n}B_{k+1}A^{1}_{k+1,n}]_{h,j}$:

By using (66), we have

[TABLE]

• •

$a.s.-\lim_{n}\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{3}_{k+1,n}B_{k+1}A^{3}_{k+1,n}]_{h,j}$:

Since $c(\alpha_{h}+\alpha_{j})=1$ implies $c\alpha_{h}\neq 1$ and $c\alpha_{j}\neq 1$, we have that

[TABLE]

has the same limit as

[TABLE]

which is equal to

[TABLE]

Hence, we have that

[TABLE]

• •

$a.s.-\lim_{n}\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}b_{k+1}(a^{2}_{k+1,n})^{2}$:

Since the calculations are analogous to those in Subsection A.3.1, we have

[TABLE]

• •

$a.s.-\lim_{n}\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}[A^{1}_{k+1,n}B_{k+1}A^{3}_{k+1,n}]_{h,j}$:

Since $c(\alpha_{h}+\alpha_{j})=1$ implies $c\alpha_{j}\neq 1$, we have that

[TABLE]

has the same limit as

[TABLE]

Hence, we have

[TABLE]

• •

$a.s.-\lim_{n}\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}a^{2}_{k+1,n}[\mathbf{b}_{k+1}^{\top}A^{1}_{k+1,n}]_{j}$:

Since the calculations are analogous to those in Subsection A.3.1, we have

[TABLE]

• •

$a.s.-\lim_{n}\frac{n}{\ln(n)}\sum_{k=m_{0}}^{n-1}r_{k}^{2}a^{2}_{k+1,n}[\mathbf{b}_{k+1}^{\top}A^{3}_{k+1,n}]_{j}$:

Since the calculations are analogous to those in Subsection A.3.1, we have

[TABLE]

Appendix B Stable convergence and its variants

This brief appendix contains some basic definitions and results concerning stable convergence and its variants. For more details, we refer the reader to [18, 20, 23, 29] and the references therein.

Let $(\Omega,{\mathcal{A}},P)$ be a probability space, and let $S$ be a Polish space, endowed with its Borel $\sigma$-field. A kernel on $S$, or a random probability measure on $S$, is a collection $K=\{K(\omega):\,\omega\in\Omega\}$ of probability measures on the Borel $\sigma$-field of $S$ such that, for each bounded Borel real function $f$ on $S$, the map

[TABLE]

is $\mathcal{A}$-measurable. Given a sub-$\sigma$-field $\mathcal{H}$ of $\mathcal{A}$, a kernel $K$ is said $\mathcal{H}$-measurable if all the above random variables $K\!f$ are $\mathcal{H}$-measurable.

On $(\Omega,{\mathcal{A}},P)$, let $(Y_{n})_{n}$ be a sequence of $S$-valued random variables, let $\mathcal{H}$ be a sub-$\sigma$-field of $\mathcal{A}$, and let $K$ be a $\mathcal{H}$-measurable kernel on $S$. Then we say that $Y_{n}$ converges $\mathcal{H}$-stably to $K$, and we write $Y_{n}\longrightarrow K$ ${\mathcal{H}}$-stably, if

[TABLE]

where $K(\cdot)$ denotes the random variable defined, for each Borel set $B$ of $S$, as $\omega\mapsto K\!I_{B}(\omega)=K(\omega)(B)$. In the case when ${\mathcal{H}}={\mathcal{A}}$, we simply say that $Y_{n}$ converges stably to $K$ and we write $Y_{n}\longrightarrow K$ stably. Clearly, if $Y_{n}\longrightarrow K$ ${\mathcal{H}}$-stably, then $Y_{n}$ converges in distribution to the probability distribution $E[K(\cdot)]$. Moreover, the $\mathcal{H}$-stable convergence of $Y_{n}$ to $K$ can be stated in terms of the following convergence of conditional expectations:

[TABLE]

for each bounded continuous real function $f$ on $S$.

In [23] the notion of $\mathcal{H}$-stable convergence is firstly generalized in a natural way replacing in (79) the single sub-$\sigma$-field $\mathcal{H}$ by a collection ${\mathcal{G}}=({\mathcal{G}}_{n})_{n}$ (called conditioning system) of sub-$\sigma$-fields of $\mathcal{A}$ and then it is strengthened by substituting the convergence in $\sigma(L^{1},L^{\infty})$ by the one in probability (i.e. in $L^{1}$, since $f$ is bounded). Hence, according to [23], we say that $Y_{n}$ converges to $K$ stably in the strong sense, with respect to ${\mathcal{G}}=({\mathcal{G}}_{n})_{n}$, if

[TABLE]

for each bounded continuous real function $f$ on $S$.

Finally, a strengthening of the stable convergence in the strong sense can be naturally obtained if in (80) we replace the convergence in probability by the almost sure convergence: given a conditioning system ${\mathcal{G}}=({\mathcal{G}}_{n})_{n}$, we say that $Y_{n}$ converges to $K$ in the sense of the almost sure conditional convergence, with respect to ${\mathcal{G}}$, if

[TABLE]

for each bounded continuous real function $f$ on $S$. The almost sure conditional convergence has been introduced in [18] and, subsequently, employed by others in the urn model literature (e.g. [6, 45]).

We now conclude this section recalling two convergence results that we need in our proofs.

From [24, Proposition 3.1], we can get the following result.

Theorem B.1**.**

Let $({\mathbf{T}}_{n,k})_{n\geq 1,1\leq k\leq k_{n}}$ be a triangular array of $d$-dimensional real random vectors, such that, for each fixed $n$, the finite sequence $({\mathbf{T}}_{n,k})_{1\leq k\leq k_{n}}$ is a martingale difference array with respect to a given filtration $({\mathcal{G}}_{n,k})_{k\geq 0}$. Moreover, let $(t_{n})_{n}$ be a sequence of real numbers and assume that the following conditions hold:

• (c1)

${\mathcal{G}}_{n,k}{\underline{\subset}}{\mathcal{G}}_{n+1,k}$* for each $n$ and $1\leq k\leq k_{n}$;*

• (c2)

$\sum_{k=1}^{k_{n}}(t_{n}{\mathbf{T}}_{n,k})(t_{n}{\mathbf{T}}_{n,k})^{\top}=t_{n}^{2}\sum_{k=1}^{k_{n}}{\mathbf{T}}_{n,k}{\mathbf{T}}_{n,k}^{\top}\stackrel{{\scriptstyle P}}{{\longrightarrow}}\Sigma$, where $\Sigma$ is a random positive semidefinite matrix;

• (c3)

$\sup_{1\leq k\leq k_{n}}|t_{n}{\mathbf{T}}_{n,k}|\stackrel{{\scriptstyle L^{1}}}{{\longrightarrow}}0$.

Then $t_{n}\sum_{k=1}^{k_{n}}{\mathbf{T}}_{n,k}$ converges stably to the Gaussian kernel ${\mathcal{N}}(\mathbf{0},\Sigma)$.

The following result combines together stable convergence and stable convergence in the strong sense.

Theorem B.2**.**

[11, Lemma 1]** Suppose that $C_{n}$ and $D_{n}$ are $S$-valued random variables, that $M$ and $N$ are kernels on $S$, and that ${\mathcal{G}}=({\mathcal{G}}_{n})_{n}$ is a filtration satisfying for all $n$

[TABLE]

If $C_{n}$ stably converges to $M$ and $D_{n}$ converges to $N$ stably in the strong sense, with respect to $\mathcal{G}$, then

[TABLE]

(Here, $M\otimes N$ is the kernel on $S\times S$ such that $(M\otimes N)(\omega)=M(\omega)\otimes N(\omega)$ for all $\omega$.)