Fluctuation of the free energy of Sherrington-Kirkpatrick model with Curie-Weiss interaction: the paramagnetic regime

TL;DR

This paper analyzes the free energy fluctuations of a mixed Sherrington-Kirkpatrick and Curie-Weiss spin model in the paramagnetic regime, showing asymptotic Gaussianity and spectral approximation methods for i.i.d. Rademacher spins.

Contribution

It extends previous spherical models to i.i.d. Rademacher spins, proving Gaussian fluctuations and spectral approximation of free energy in the paramagnetic phase.

Findings

Free energy is asymptotically Gaussian.

Approximation by linear spectral statistics.

Method relies on dense sub-graph conditioning.

Abstract

We consider a spin system with pure two spin Sherrington-Kirkpatrick Hamiltonian with Curie-Weiss interaction. The model where the spins are spherically symmetric was considered by \citet{Baiklee16} and \citet{Baikleewu18} which shows a two dimensional phase transition with respect to temperature and the coupling constant. In this paper we prove a result analogous to \citet{Baiklee16} in the "paramagnetic regime" when the spins are i.i.d. Rademacher. We prove the free energy in this case is asymptotically Gaussian and can be approximated by a suitable linear spectral statistics. Unlike the spherical symmetric case the free energy here can not be written as a function of the eigenvalues of the corresponding interaction matrix. The method in this paper relies on a dense sub-graph conditioning technique introduced by \citet{Ban16}. The proof of the approximation by the linear spectral statistics part is taken from \citet{Banerjee2017}.

Equations

$$H_n^{SK}(\sigma ):=\frac{1}{\sqrt{n}}⟨\sigma ,A\sigma ⟩=\frac{1}{\sqrt{n}}\sum _{i,j}{A}_{i,j}{\sigma }_i{\sigma }_j=\frac{2}{\sqrt{n}}\sum _{1\le i<j\le n}{A}_{i,j}{\sigma }_i{\sigma }_j.$$

$$H_n(\sigma ):=H_n^{SK}(\sigma )+H_n^{CW}(\sigma )$$

$$H_n^{CW}(\sigma ):=\frac{J}{n}\sum _{i,j=1}^n{\sigma }_i{\sigma }_j=\frac{J}{n}{\left(\sum _{i=1}^n{\sigma }_i\right)}^2.$$

$$Z_n(\beta ):=\int \exp \left\{\beta H_n(\sigma )\right\}d{\Psi }_n(\sigma )$$

$$F_n(\beta ):=\frac{1}{n}\log \left(Z_n(\beta )\right).$$

$$\begin{array}{rl}{Z}_n(\beta )& =\sum _{\sigma \in \{-1,+1{\}}^n}\frac{1}{2^n}\exp \left\{\sum _{i,j=1}^n\frac{\beta }{\sqrt{n}}{A}_{i,j}{\sigma }_i{\sigma }_j+\frac{\beta J}{n}\sum _{i,j=1}^n{\sigma }_i{\sigma }_j+\beta (J^{\prime }-J)\right\}\\ & =\sum _{\sigma \in \{-1,+1{\}}^n}\frac{1}{2^n}\exp \left\{\frac{2\beta }{\sqrt{n}}\sum _{1\le i<j\le n}^n{A}_{i,j}{\sigma }_i{\sigma }_j+\frac{2\beta J}{n}\sum _{1\le i<j\le n}{\sigma }_i{\sigma }_j+\frac{\beta }{\sqrt{n}}\sum _{i=1}^n\left(A_{i,i}+\frac{J^{\prime }}{\sqrt{n}}\right)\right\}.\end{array}$$

$$W_p\left(F^1,F^2\right):={\left[\underset{X\sim F^1;Y\sim F^2}{\inf }E\left[{\left|X-Y\right|}^p\right]\right]}^{\frac{1}{p}}.$$

$$n\left(F_n(\beta )-F(\beta )\right)\stackrel{d}{\to }N(f_1,{\alpha }_1)$$

$${\alpha }_1=-{\beta }^2-\frac{1}{2}\log \left(1-4{\beta }^2\right)$$

$$f_1=-\frac{1}{2}\log \left(1-2\beta J\right)+\beta (J^{\prime }-J)+\frac{1}{4}\log \left(1-4{\beta }^2\right).$$

$$\begin{array}{rl}& \log \left(Z_n(\beta )\right)+\frac{1}{2}\log \left(1-2\beta J\right)-(n-1){\beta }^2+\beta (J-J^{\prime })-\beta C_{n,1}-\\ & \text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\text{leavevmode}\sum _{k=2}^{m_n}\frac{2(2\beta {)}^k\left(C_{n,k}-(n-1){\mathbb{I}}_{k=2}\right)-(2\beta {)}^{2k}}{4k}\stackrel{p}{\to }0.\end{array}$$

$${\left(\frac{1-\sqrt{1-4z^2}}{2z}\right)}^r=\sum _{m=r}^{\infty }f(m,r)z^m.$$

$$\psi_{k}=\left\{\begin{array}[]{ll}0&\text{if $k$ is odd}\\ \frac{1}{\frac{k}{2}+1}\binom{k}{\frac{k}{2}}&\text{if $k$ is even}.\end{array}\right.$$

$$S_m\left(\cos (\theta )\right)=\cos (m\theta ).$$

$$P_m(x)=2S_m\left(\frac{x}{2}\right).$$

$$C_{n,k}-\left\{\mathrm{Tr}\left(P_k\left(\frac{1}{\sqrt{n}}\tilde{A}\right)\right)-E\left[\mathrm{Tr}\left(P_k\left(\frac{1}{\sqrt{n}}\tilde{A}\right)\right)\right]\right\}\stackrel{p}{\to }0.$$

$$\mathrm{Tr}\left[f(A)\right]=\sum _{i=1}^{n}f({\lambda }_i)$$

$$C_{n,k}:={\left(\frac{1}{\sqrt{n}}\right)}^k\sum _{i_0,i_1,\dots ,i_{k-1}}{A}_{i_0,i_1}{A}_{i_1,i_2}\dots A_{i_{k-1},i_0}.$$

$$C_{n,1}:=\left(\frac{1}{\sqrt{n}}\right)\sum _i{A}_{i,i}.$$

$$\underset{n\to \infty }{\mathrm{limsup}}{E}_{{\mathbb{P}}_n}\left[Y_n^2\right]\le \exp \left\{\sum _{i=1}^{\infty }\frac{{\mu }_i^2}{{\sigma }_i^2}\right\}<\infty .$$

$$\mathrm{Var}\left[\sum _{i=1}^{m_n}\frac{{\mu }_i}{{\sigma }_i^2}{W}_{n,i}\right]\to C<\infty$$

$$Y_n\stackrel{d}{\to }\exp \left\{\sum _{i=1}^{\infty }\frac{{\mu }_i{Z}_i-\frac{1}{2}{\mu }_i^2}{{\sigma }_i^2}\right\}.$$

$$\underset{m\to \infty }{\mathrm{limsup}}{\mathbb{P}}_{n_{l_m}}\left(\left|\log (Y_{n_{l_m}})-\sum _{k=1}^K\frac{2{\mu }_k{W}_{n_{l_m},k}-{\mu }_k^2}{2{\sigma }_k^2}\right|\ge ϵ\right)\le \delta .$$

$$L:=\exp \left\{\sum _{i=1}^{\infty }\frac{2{\mu }_i{Z}_i-{\mu }_i^2}{2{\sigma }_i^2}\right\},L^{(m)}:=\exp \left\{\sum _{i=1}^m\frac{2{\mu }_i{Z}_i-{\mu }_i^2}{2{\sigma }_i^2}\right\},m\in \mathbb{N}.$$

$$E\left[\exp \left\{\frac{2{\mu }_i{Z}_i-{\mu }_i^2}{2{\sigma }_i^2}\right\}\right]=1.$$

$$\operatorname{E}\left[\big{(}L^{(m)}\big{)}^{2}\right]=\prod_{i=1}^{m}\exp\left\{\frac{\mu_{i}^{2}}{\sigma_{i}^{2}}\right\}=\exp\left\{\sum_{i=1}^{m}\frac{\mu_{i}^{2}}{\sigma_{i}^{2}}\right\}.$$

$$E[L]=1,E[L^2]=\exp \left\{\sum _{i=1}^{\infty }\frac{{\mu }_i^2}{{\sigma }_i^2}\right\}.$$

$$E[\log (L)]=-\frac{1}{2}\sum _{i=1}^{\infty }\frac{{\mu }_i^2}{{\sigma }_i^2},\mathrm{Var}(\log (L))=\sum _{i=1}^{\infty }\frac{{\mu }_i^2}{{\sigma }_i^2}.$$

$$\underset{n\to \infty }{\mathrm{limsup}}{E}_{{\mathbb{P}}_n}\left[Y_n^2\right]<\infty ,$$

$$\exp \left\{\sum _{i=1}^{\infty }\frac{{\mu }_i^2}{{\sigma }_i^2}\right\}-\exp \left\{\sum _{i=1}^m\frac{{\mu }_i^2}{{\sigma }_i^2}\right\}<ϵ.$$

Full text

1 Introduction

1.1 The model description

We at first give the description of the model. We start with a symmetric matrix $A=\left(A_{i,j}\right)_{i,j=1}^{n}$ where the entries in the strict upper triangular part of $A$ are i.i.d. standard Gaussian and for simplicity one might take $A_{i,i}=0$. The Hamiltonian corresponding to the Sherrington-Kirkpatrick model without any external field is given by

[TABLE]

Here $\sigma_{i}$’s are called spins and in this paper we shall only consider the case when $\sigma_{i}\in\{-1,1\}$ for each $i$. In particular, one might consider the case when the spins $\sigma_{i}$’s are i.i.d. Rademacher random variables. This is known as the classical Sherrington- Kirkpatrick model. This model has got significant amount of interest in the study of spin glasses over the last few decades. Celebrated result like the proof of Parisi formula is considered one of the major advancements in this field. One might look at Panchenko [21], Talagrand [22] for some information in this regard.

However the main focus of this paper is the following Hamiltonian

[TABLE]

where the Curie-Weiss Hamiltonian with coupling constant $J$ is defined by

[TABLE]

Note that model corresponding to the Hamiltonian $H_{n}^{CW}(\sigma)$ is a simple model which can be studied in considerable details. One might look at Ellis [12] for a reference.

This Hamiltonian in (1.2) was defined in Talagrand [23](see (4.22)) as a model which have difficulties of the ferromagnetic interactions however with a familiar disorder. This model was introduced as a prelude to study the Hopfield model in Talagrand [23].

The main result of this paper is a limit theorem for the free energy corresponding to the Hamiltonian $H_{n}(\sigma)$ when $\beta<\frac{1}{2}$ and $\beta J<\frac{1}{2}$ whenever $\sigma_{i}$’s are i.i.d. Rademacher variables. If the spins $\mathbf{\sigma}=(\sigma_{1},\ldots,\sigma_{n})$ are distributed according to the uniform measure on the sphere $S_{n-1}$ where $S_{n-1}:=\left\{\sigma\in\mathbb{R}^{n}\leavevmode\nobreak\ |\leavevmode\nobreak\ ||\sigma||^{2}=n\right\}$, then the analogous Hamiltonian was considered in Baik and Lee [3] and Baik et al. [4]. However the results in Baik and Lee [3] are much more general than the current paper in the sense they are able to consider any $\beta>0,J>0$. Depending on the values of $\beta,J$, there are three distinct regimes where the free energy shows different behaviors. In particular, the regime $\beta<\frac{1}{2}$ and $\beta J<\frac{1}{2}$ is known as the para-magnetic regime where the result analogous to this paper was obtained in Baik and Lee [3]. The regime when $\beta>\frac{1}{2}$ and $J<1$ is known as the spin glass regime and the other case ($\beta J>\frac{1}{2}$ and $J>1$) is known as the ferromagnetic regime. Although the results in Baik and Lee [3] are much more general than the current paper in terms of possible choices of $(\beta,J)$, the technique of that paper is restricted to the case when the spins $\mathbf{\sigma}=(\sigma_{1},\ldots,\sigma_{n})$ are distributed according to the uniform measure on the sphere $S_{n-1}$ which does not cover the case when $\sigma_{i}$’s are i.i.d. Rademacher random variables. This is the problem we consider in this paper.

We now give a very brief overview of the literature for the fluctuation of free energy of classical Sherrington-Kirkpatrick model in with absence of any external field.

The classical Sherrington-Kirkpatrick model with no external field ($h=0$) under goes a phase transition at $\beta=\frac{1}{2}$. When the spins $\sigma_{i}$’s are i.i.d. Rademacher and $\beta<\frac{1}{2}$ the free energy has a Gaussian limiting distribution. One might look at Aizenman et al. [1] and Comets and Neveu [11] for some references. The case $\beta>\frac{1}{2}$ is known as the low temperature regime. To the best of our limited knowledge, very few things are known about the fluctuations of the free energy in this regime. One might look at Chatterjee [9] where it is proved that the fluctuation of the free energy of the Sherrington-Kirkpatrick model is at least $O(1)$. When the spins are uniformly distributed on $S_{n-1}$, the free energy analogously undergoes a phase transition at $\beta=\frac{1}{2}$. When $\beta<\frac{1}{2}$, the free energy has a Gaussian limiting distribution and can be approximated by a linear spectral statistics of the eigenvalues. The low temperature case ($\beta>\frac{1}{2}$) is also well-known. Here the free energy has a limiting GOE Tracy-Widom distribution with $O\left(n^{-\frac{2}{3}}\right)$ fluctuations. One might look at Baik and Lee [2] for a reference.

Finally, The model considered in this paper (Hamiltonian defined in (1.2)) was also considered in Chen [10] from the point of view of thermodynamic limit of the free energy. One might also look at Cadel and Rovira [7] where this model was studied using cavity method. However to the best of our limited knowledge the problem of fluctuation of free energy remained open.

1.2 Preliminary definitions

We now give some preliminary definitions. We start with defining a Hamiltonian which generalizes the one defined in (1.2).

Definition 1.1**.**

(interactions) Suppose $A_{i,j}$, $1\leq i\leq j\leq n$ be i.i.d. standard Gaussian random variables. Set $A_{j,i}=A_{i,j}$ for $i<j$. Let $M_{i,j}=\frac{1}{\sqrt{n}}A_{i,j}+\frac{J}{n}$ and $M_{i,i}=\frac{1}{\sqrt{n}}A_{i,i}+\frac{J^{\prime}}{n}$ for some $n$ independent non negative fixed constants $J$ and $J^{\prime}$. One considers the Hamiltonian $H_{n}(\sigma)=\langle\sigma,M\sigma\rangle$. The defined Hamiltonian is more general than the one defined in (1.2) in the following sense. Definition 1.1 allows $A_{i,i}$ to be non zero random variables with $J^{\prime}$ being any arbitrary constant.

Definition 1.2**.**

(Partition function and Free energy) Given any Hamiltonian $H_{n}(\sigma)$ where $\sigma=(\sigma_{1},\ldots,\sigma_{n})$ are distributed according to a measure $\Psi_{n}$, the partition function and free energy at an inverse temperature $\beta$ are denoted by $Z_{n}(\beta)$ and $F_{n}(\beta)$ respectively and defined as follows.

[TABLE]

and

[TABLE]

In our case we take $\Psi_{n}$ to be the uniform probability measure on the Hypercube $\{-1,+1\}^{n}$.

In our case $Z_{n}(\beta)$ is as follows:

[TABLE]

Finally we define the Wasserstein distance between two distribution functions. This distance is crucially used at many places of the proofs.

Definition 1.3**.**

We at first fix $p\geq 1$. Suppose $F^{1}$ and $F^{2}$ are two distribution functions such that $\int_{x\in\mathbb{R}}|x|^{p}dF^{1}(x)<\infty$ and $\int_{x\in\mathbb{R}}|x|^{p}dF^{2}(x)<\infty$. Then the Wasserstein distance for $p$ between $F^{1}$ and $F^{2}$ is is denoted by $W_{p}$ and defined to be

[TABLE]

Observe that the Wasserstein distance is defined for two distribution functions. However when we write $\left[\inf_{X\sim F^{1};Y\sim F^{2}}\operatorname{E}\left[\left|X-Y\right|^{p}\right]\right]^{\frac{1}{p}}$, we consider two random variables $X\sim F^{1}$ and $Y\sim F^{2}$ such that $X$ and $Y$ are defined on the same measure space such that $\operatorname{E}\left[\left|X-Y\right|^{p}\right]$ takes the lowest possible value.

The following result on the Wasserstein distance is well known.

Proposition 1.1**.**

Suppose $\left\{X_{n}\right\}_{n=1}^{\infty}$ be a sequence of random variables and $X$ be a random variable. If $W_{2}\left(F^{X_{n}},F^{X}\right)\to 0$, then $X_{n}\stackrel{{\scriptstyle d}}{{\to}}X$ and $\operatorname{E}[X_{n}^{2}]\to\operatorname{E}[X^{2}]$.

One might see Mallows [19] for a reference.

2 Main result

We are ready to state the main result of this paper.

Theorem 2.1**.**

(Asymptotic normality) Consider the Hamiltonian $H_{n}(\sigma)$ as defined in Definition 1.1. Let $F_{n}(\beta)$ be the free energy corresponding to the Hamiltonian $H_{n}(\sigma)$. When $\beta<\frac{1}{2}$ and $\beta J<\frac{1}{2}$ the following result holds:

[TABLE]

where $F(\beta)=\beta^{2}$,

[TABLE]

and

[TABLE] 2. 2.

(Approximation by signed cycle counts) For any sequence $m_{n}$ diverging to infinity such that $m_{n}=o\left(\sqrt{\log n}\right)$, one also has the following approximation result for the log partition function $\log\left(Z_{n}(\beta)\right)$.

[TABLE]

Here the statistics $C_{n,k}$’s are taken according to Definition 3.1.

Our next result is Theorem 2.1 where the approximations of $C_{n,k}$’s by linear spectral statistics are stated. Before going to Theorem 2.2, we need some important definitions. We now introduce an important generating function. Given any $r\in\mathbb{N}$, let

[TABLE]

The coefficients $f(m,r)$’s are key quantities for defining the variances and covariances of linear spectral statistics constructed from different power functions. For any $k\in\mathbb{N}$ denote

[TABLE]

So $\psi_{k}$ is the $\frac{k}{2}$-th Catalan number for every even $k$. Finally, we define a set of rescaled Chebyshev polynomials. These polynomials are important for drawing the connection between signed cycles $C_{n,k}$’s and the spectrum of adjacency matrix. The standard Chebyshev polynomial of degree $m$ is denoted by $S_{m}(x)$ and can be defined by the identity

[TABLE]

In this paper we use a slight variant of $S_{m}$, denoted by $P_{m}$ and defined as

[TABLE]

In particular, $P_{m}(2\cos(\theta))=2\cos(m\theta)$. It is easy to note that $P_{m}\left(z+z^{-1}\right)=z^{m}+z^{-m}$ for all $z\in\mathbb{C}$. One also notes that $P_{m}(\cdot)$ is even and odd whenever $m$ is even or odd respectively.

Theorem 2.2**.**

(Approximation of cycles by linear spectral statistics) Let $\tilde{A}$ be the matrix obtained by putting [math] on the diagonal of the matrix $A$. Let $P_{k}$ be as defined in (2.8) . Then to following is true for any $3\leq k=o\left(\sqrt{\log n}\right)$ under $\mathbb{P}_{n}$.

[TABLE]

Here for any function $f$ and a matrix $A$

[TABLE]

where $\lambda_{1},\ldots,\lambda_{n}$ are the eigenvalues of the matrix $A$.

The proof of Theorem 2.2 is given in Section A.

3 Proof techniques and related definitions

As mentioned earlier, the fundamental technique of the proof of Theorem 2.1 is completely different from that of Baik and Lee [3]. The proof in the current paper is based on the dense sub graph conditioning technique introduced in Banerjee [5]. The fundamental idea is to view the free energy as the log of the Radon-Nikodym derivative $\left(\log\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\right)$ of two suitably defined sequences of measures $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$. Now one introduces a class of random variables called the signed cycles (Definition 3.1) and prove that these variables asymptotically determine the full Radon-Nikodym derivative. This is done by a fine second moment argument. The argument in this part is highly motivated from a paper by Janson [15] where it is proved that a similar kind of argument holds for random regular graphs where the signed cycle counts are replaced by standard cycle counts. The technique of cycle conditioning was also used in Mossel et al. [20] in their proof of contiguity of the probability measures induced by a planted partition model and the Erdős- Rényi model in the sparse regime.

We now start with defining the signed cycles random variables.

Definition 3.1**.**

Let $A$ be a $n\times n$ symmetric matrix with with the strict upper triangular part being i.i.d. mean [math] and variance $1$. For $k\geq 2$, we define the signed cycles random variables $C_{n,k}$ as follows:

[TABLE]

Here $i_{0},\ldots,i_{k-1}$ are taken to be all distinct. For $k=1$, $C_{n,k}$ is simply defined as follows:

[TABLE]

In this paper we require the concept of mutual contiguity of two sequences of measures heavily. Now we define these concepts. If someone is interested one might have a look at Le Cam [17] and Le Cam and Yang [18] for general discussions on contiguity.

Definition 3.2**.**

(Contiguity) For two sequences of probability measures $\left\{\mathbb{P}_{n}\right\}_{n=1}^{\infty}$ and $\left\{\mathbb{Q}_{n}\right\}_{n=1}^{\infty}$ defined on $\sigma$-fields $(\Omega_{n},\mathcal{F}_{n})$, we say that $\mathbb{Q}_{n}$ is contiguous with respect to $\mathbb{P}_{n}$, denoted by $\mathbb{Q}_{n}\triangleleft\mathbb{P}_{n}$, if for any event sequence $A_{n}$, $\mathbb{P}_{n}(A_{n})\to 0$ implies $\mathbb{Q}_{n}(A_{n})\to 0$. We say that they are (asymptotically) mutually contiguous, denoted by $\mathbb{P}_{n}\triangleleft\triangleright\mathbb{Q}_{n}$, if both $\mathbb{Q}_{n}\triangleleft\mathbb{P}_{n}$ and $\mathbb{P}_{n}\triangleleft\mathbb{Q}_{n}$ hold.

The following result gives an useful way to study mutual contiguity:

Proposition 3.1**.**

Suppose that $L_{n}=\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}$, regarded as a random variable on $(\Omega_{n},\mathcal{F}_{n},\mathbb{P}_{n})$, converges in distribution to some random variable $L$ as $n\to\infty$. Then $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$ are mutually contiguous if and only if $L>0$ a.s. and $\operatorname{E}[L]=1$.

One might look at Proposition 3 of Janson [15] for a proof.

We now state a result on mutual contiguity of measures.

Proposition 3.2**.**

(Janson’s second moment method): Let $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$ be two sequences of probability measures such that for each $n$, both are defined on the common $\sigma$-algebra $(\Omega_{n},\mathcal{F}_{n})$. Suppose that for each $i\geq 1$, $W_{n,i}$ are random variables defined on $(\Omega_{n},\mathcal{F}_{n})$. Then the probability measures $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$ are asymptotically mutually contiguous if the following conditions hold simultaneously:

1. (i)

$\mathbb{Q}_{n}$* is absolutely continuous with respect to $\mathbb{P}_{n}$ for each $n$;* 2. (ii)

The likelihood ratio statistic $Y_{n}=\frac{\mathrm{d}\mathbb{Q}_{n}}{\mathrm{d}\mathbb{P}_{n}}$ satisfies

[TABLE] 3. (iii)

For any fixed $k\geq 1$, one has $\left(W_{n,1},\ldots,W_{n,k}\right)|\mathbb{P}_{n}\stackrel{{\scriptstyle d}}{{\to}}\left(Z_{1},\ldots,Z_{k}\right)$ and $\left(W_{n,1},\ldots,W_{n,k}\right)|\mathbb{Q}_{n}\stackrel{{\scriptstyle d}}{{\to}}\left(Z^{\prime}_{1},\ldots,Z^{\prime}_{k}\right)$. Further $Z_{i}\sim N(0,\sigma_{i}^{2})$ and $Z^{\prime}_{i}\sim N(\mu_{i},\sigma_{i}^{2})$ are sequences of independent random variables. 4. (iv)

Under $\mathbb{P}_{n}$, $W_{n,i}$’s are uncorrelated and there exists a sequence $m_{n}\to\infty$ such that

[TABLE]

Here the $\operatorname{Var}$ is considered with respect to the measure $\mathbb{P}_{n}$.

In addition, we have that under $\mathbb{P}_{n}$,

[TABLE]

Furthermore, given any $\epsilon,\delta>0$ there exists a natural number $K=K(\delta,\epsilon)$ such that for any sequence $n_{l}$ there is a further subsequence $n_{l_{m}}$ such that

[TABLE]

Proposition 3.2 is one of the most important results required for the proof of Theorem 2.1. In particular, the rest of the proof relies on defining the measures $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$ and $W_{n,i}$’s properly. It is worth noting that in this context the statistics $C_{n,i}$’s serve as $W_{n,i}$’s.

We now give the proof of Proposition 3.2.

Proof of Proposition 3.2:

Proof of mutual contiguity and (3.5)

This proof is broken into two steps. We focus on proving (3.5). Given (3.5), mutual contiguity is a direct consequence of Proposition 3.1.

Step 1. We first prove the random variable on the right hand side of (3.5) is almost surely positive and has mean $1$. Let us define

[TABLE]

As $Z_{i}\sim N(0,\sigma_{i}^{2})$, for any $i\in\mathbb{N}$, and so

[TABLE]

So $\{L^{(m)}\}_{m=1}^{\infty}$ is a martingale sequence and

[TABLE]

Now by the righthand side of (3.3), $L^{(m)}$ is a $L^{2}$ bounded martingale. Hence, $L$ is a well defined random variable with

[TABLE]

On the other hand $\log(L)$ is a limit of Gaussian random variables, hence $\log(L)$ is Gaussian with

[TABLE]

Hence $\mathbb{P}(L=0)=\mathbb{P}(\log(L)=-\infty)=0$.

Step 2. Now we prove $Y_{n}\stackrel{{\scriptstyle d}}{{\to}}L$. Since

[TABLE]

condition (iv) implies that the sequence $Y_{n}$ is tight. Prokhorov’s theorem further implies that there is a subsequence $\{n_{k}\}_{k=1}^{\infty}$ such that $Y_{n_{k}}$ converge in distribution to some random variable $L(\{n_{k}\})$. In what follows, we prove that the distribution of $L(\{n_{k}\})$ does not depend on the subsequence $\{n_{k}\}$. In particular, $L(\{n_{k}\})\stackrel{{\scriptstyle d}}{{=}}L$. To start with, note that since $Y_{n_{k}}$ converges in distribution to $L(\{n_{k}\})$, for any further subsequence $\{n_{k_{l}}\}$ of $\{n_{k}\}$, $Y_{n_{k_{l}}}$ also converges in distribution to $L(\{n_{k}\})$.

Given any fixed $\epsilon>0$ take $m$ large enough such that

[TABLE]

For this fixed number $m$, consider the joint distribution of $(Y_{n_{k}},W_{n_{k},1},\ldots,W_{n_{k},m})$. This sequence of $m+1$ dimensional random vectors with respect to $\mathbb{P}_{n_{k}}$ is tight by condition (ii). So it has a further subsequence such that

[TABLE]

where $H_{1}\stackrel{{\scriptstyle d}}{{=}}L(\{n_{k}\})$ and $\left(H_{2},\ldots,H_{m+1}\right)\stackrel{{\scriptstyle d}}{{=}}\left(Z_{1},\ldots,Z_{m}\right)$ We are to show that we can define the random variables $L^{(m)}$ and $L(\{n_{k}\})$ in such a way that there exist suitable $\sigma$-algebras $\mathcal{F}_{1}\subset\mathcal{F}_{2}$ such that $L^{(m)}\in\mathcal{F}_{1}$, $L(\{n_{k}\})\in\mathcal{F}_{2}$, and $\operatorname{E}\left[L(\{n_{k}\})\left|\right.\mathcal{F}_{1}\right]=L^{(m)}$.

Since $\limsup_{n\to\infty}\operatorname{E}_{\mathbb{P}_{n}}\left[Y_{n}^{2}\right]<\infty$, the sequence $Y_{n_{k_{l}}}$ is uniformly integrable. This, together with condition (i), leads to

[TABLE]

Now take any positive bounded continuous function $f:\mathbb{R}^{m}\to\mathbb{R}$. By Fatou’s lemma

[TABLE]

However for any constant $\xi$, (3.14) implies $\xi=\xi\operatorname{E}_{\mathbb{P}_{n_{k_{l}}}}[Y_{n_{k_{l}}}]\to\xi\operatorname{E}[L(\{n_{k}\})]=\xi$. Observe that given any bounded continuous function $f$ we can find $\xi$ large enough so that $f+\xi$ is a positive bounded continuous function. So (3.15) is indeed implied by Fatou’s lemma.

Now

[TABLE]

So (3.15) holds for any bounded continuous function $f$. On the other hand, replacing $f$ by $-f$ we have

[TABLE]

Now condition (ii) leads to

[TABLE]

Here $Q$ is the measure induced by $(Z_{1}^{\prime},\ldots,Z_{m}^{\prime})$. In particular, one can take the measure $Q$ such that $(Z_{1},\ldots,Z_{m})$ themselves are distributed as $(Z_{1}^{\prime},\ldots,Z_{m}^{\prime})$ under the measure $Q$. This is true since

[TABLE]

for any bounded continuous function $f$, and so $\int_{A}\mathrm{d}Q=\operatorname{E}[\mathbf{1}_{A}L^{(m)}]$ for any $A\in\sigma(Z_{1},\ldots,Z_{m})$. Now looking back into (3.17), we have for any $A\in\sigma(Z_{1},\ldots,Z_{m})$, $\operatorname{E}[\mathbf{1}_{A}L^{(m)}]=\operatorname{E}\left[\mathbf{1}_{A}L(\{n_{k}\})\right]$. Since by definition $L^{(m)}$ is $\sigma(Z_{1},\ldots,Z_{m})$ measurable, we have

[TABLE]

From Fatou’s lemma

[TABLE]

As a consequence, we have

[TABLE]

So $W_{2}(F^{L^{(m)}},F^{L(\{n_{k}\})})<\sqrt{\epsilon}$. Here $F^{L^{(m)}}$ and $F^{L(\{n_{k}\})}$ denote the distribution functions corresponding to $L^{(m)}$ and $L(\{n_{k}\})$ respectively. As a consequence, $W_{2}(F^{L^{(m)}},F^{L(\{n_{k}\})})\to 0$ as $m\to\infty.$ Hence $L^{(m)}\stackrel{{\scriptstyle d}}{{\to}}L(\{n_{k}\})$ by the result stated after Definition 1.3. On the other hand, we have already proved $L^{(m)}$ converges to $L$ in $L^{2}$. So $L(\{n_{k}\})\stackrel{{\scriptstyle d}}{{=}}L$.

Proof of (3.6)

We start with a sub sequence $\{n_{l}\}$. We shall choose $k$ large enough which shall be specified later. We also know that both the random variables $\log\left(Y_{n_{l}}\right)$ and $\left\{\sum_{i=1}^{k}\frac{2\mu_{i}W_{n_{l},i}-\mu_{i}^{2}}{2\sigma_{i}^{2}}\right\}$ are tight.

We now prove that there is a $M$ invariant of $k$ such that both the probabilities

[TABLE]

for all $n_{l}$. Since the random variable $Y_{n_{l}}$ do not depend on $k$ the first inequality is obvious. For the second inequality observe that

[TABLE]

where $m_{n}$ is a sequence increasing to infinity as mentioned in Proposition 3.2. Now

[TABLE]

for all $n_{l}$. for a deterministic constant $C^{\prime}$. As a consequence,

[TABLE]

where $M^{2}=\frac{100C^{\prime}}{\delta}$.

[TABLE]

Now $\log(\cdot)$ is an uniformly continuous function on $[e^{-M},e^{M}]$. So given $\epsilon>0$, there exists $\tilde{\epsilon}$ such that for any $x,y\in[e^{-M},e^{M}]$,

[TABLE]

We know that there is a further sub-sequence $n_{l_{m}}$ such that $(Y_{n_{l_{m}}},W_{n_{l_{m}},1},\ldots,W_{n_{l_{m}},k})$ converges jointly in distribution to

[TABLE]

Let $\mathcal{F}\{n_{l_{m}},1\}\subset\mathcal{F}\{n_{l_{m}}\}$ be the sigma algebra generated by $(H_{2},\ldots,H_{k+1})$. Here $H_{1}\stackrel{{\scriptstyle d}}{{=}}L$ and $\left(H_{2},\ldots,H_{k+1}\right)\stackrel{{\scriptstyle d}}{{=}}\left(Z_{1},\ldots,Z_{k}\right)$. Using the arguments same as the previous proof we see that

[TABLE]

As a consequence, we have

[TABLE]

We shall choose this $k$ large enough so that

[TABLE]

Now by Chebyshev’s inequality

[TABLE]

Since

[TABLE]

by continuous mapping theorem for in distributional convergence, we have

[TABLE]

Since the set $[\frac{\tilde{\epsilon}}{2},\infty)$ is closed, we have by Portmanteau theorem,

[TABLE]

As a consequence,

[TABLE]

$\square$

4 Construction of $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$ and asymptotic distribution of signed cycles

4.1 Construction of the measure $\mathbb{Q}_{n}$

We at first give the construction of measures $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$. We assume that the random variables $\left(A_{i,j}\right)_{1\leq i<j\leq n}$ are defined on $\left(\Omega_{n},\mathcal{F}_{n}\right)$.

In this paper $\mathbb{P}_{n}$ is simply taken to be the measure induced by $\left(A_{i,j}\right)_{1\leq i<j\leq n}$. We now define the measure $\mathbb{Q}_{n}$ in the following way: At first for any given $\sigma\in\{-1,+1\}^{n}$, we define the measure $\mathbb{Q}_{n,\sigma}$ by

[TABLE]

Observe that $\mathbb{Q}_{n,\sigma}$ is not in general a probability measure. In particular,

[TABLE]

Here we have used the fact that $\operatorname{E}\left[\exp{tX}\right]=\exp\left\{\frac{t^{2}}{2}\right\}$ whenever $X\sim N(0,1)$. Here $\Omega_{n}$ is the sample space on which the random variables $A_{i,j}$’s are defined. Finally, we define

[TABLE]

Observe that $\mathbb{Q}_{n}$ is a valid probability measure on $\Omega_{n}$. Let

[TABLE]

One might observe that $\tau_{n}$ is the partition function of Curie-Weiss model. From Hoeffding’s inequality we have

[TABLE]

for some $\alpha_{0}>1$. This makes the random variable $\exp\left\{\frac{\beta J}{n}\left(\sum_{i=1}^{n}\sigma_{i}\right)^{2}\right\}$ uniformly integrable. Hence

[TABLE]

Plugging in the definition of partition function in (1.4), it is worth noting that:

[TABLE]

So in order to prove Theorem 2.1 it is enough to prove a central limit theorem for $\log\left(\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\right)$ and to prove that $\log\left(\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\right)$ is asymptotically independent of $\frac{1}{\sqrt{n}}\sum_{i=1}^{n}A_{i,i}$.

4.2 Asymptotic distribution of $C_{n,i}$’s under $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$

In order to derive the limiting distribution of $C_{n,i}$’s under $\mathbb{Q}_{n}$ we at first need to define another sequence of measure $\mathbb{Q}_{n}^{\prime}$. We shall at first derive the limiting distribution of $C_{n,i}$’s under $\mathbb{Q}_{n}^{\prime}$ and then we shall find the limiting distribution of $C_{n,i}$’s under $\mathbb{Q}_{n}$.

Let for any given $\sigma\in\{-1,+1\}^{n}$, $\mathbb{Q}_{n,\sigma}^{\prime}$ be defined as

[TABLE]

Observe that $\mathbb{Q}_{n,\sigma}^{\prime}$ is a probability measure. In fact $\left(A_{i,j}\right)_{1\leq i<j\leq n}\left|{}_{\mathbb{Q}_{n,\sigma}^{\prime}}\right.$ are independent normal random variables with $A_{i,j}\left|{}_{\mathbb{Q}_{n,\sigma}^{\prime}}\right.\sim N\left(\frac{2\beta}{\sqrt{n}}\sigma_{i}\sigma_{j},1\right)$. Here $\left(A_{i,j}\right)_{1\leq i<j\leq n}\left|{}_{\mathbb{Q}_{n,\sigma}^{\prime}}\right.$ denote the joint distribution of the random variables $\left(A_{i,j}\right)_{1\leq i,j\leq n}$ under the measure $\mathbb{Q}_{n,\sigma}^{\prime}$. Finally

[TABLE]

The first result in this section gives the asymptotic distribution of $C_{n,i}$’s under $\mathbb{P}_{n}$ and $\mathbb{Q}_{n}$.

Proposition 4.1**.**

Under $\mathbb{P}_{n}$, we have for any $2\leq k_{1}<k_{2}\ldots<k_{l}=o\left(\sqrt{\log(n)}\right)$ with $l$ fixed,

[TABLE] 2. 2.

Let $\Psi_{n}$ be the uniform probability measure on the hyper cube $\{-1,+1\}^{n}$. Then there exists a set $S_{n}$ with $\Psi_{n}\left(S_{n}\right)\to 0$, we have for all $\sigma\in S_{n}^{c}$, under $\mathbb{Q}_{n,\sigma}^{\prime}$

[TABLE]

where $\mu_{i}:=\left(2\beta\right)^{i}$. This implies under $\mathbb{Q}_{n}^{\prime}$,

[TABLE] 3. 3.

Finally, $C_{n,1}\stackrel{{\scriptstyle d}}{{\to}}N(0,1)$ under $\mathbb{P}_{n}$ and is asymptotically independent of the process $\{C_{n,k}-(n-1)\mathbb{I}_{k=2}\}_{k\geq 2}$.

Here $N_{l}(\mu,\Sigma)$ denotes an $l$ dimensional normal random vector with mean parameter $\mu$ and variance parameter $\Sigma$. The proof of Proposition 4.1 is given in Section B. With Proposition 4.1, we now give the asymptotic distribution of $C_{n,i}$’s under $\mathbb{Q}_{n}$.

Proposition 4.2**.**

Under $\mathbb{Q}_{n}$, we have for any $2\leq k_{1}<k_{2}\ldots<k_{l}=o\left(\sqrt{\log(n)}\right)$ with $l$ fixed,

[TABLE]

Proof.

We assume Proposition 4.1 and give the proof. We need to prove for any bounded continuous function $f:\mathbb{R}^{l}\to\mathbb{R}$,

[TABLE]

where $Z_{k_{1}},\ldots,Z_{k_{l}}$ are independent standard Gaussian random variables. Now

[TABLE]

Here $F_{n}(\sigma)=\int_{\Omega_{n}}f\left(\frac{C_{n,k_{1}}-(n-1)\mathbb{I}_{k_{1}=2}-\mu_{k_{1}}}{\sqrt{2k_{1}}},\ldots,\frac{C_{n,k_{l}}-\mu_{k_{l}}}{\sqrt{2k_{l}}}\right)\frac{d\mathbb{Q}_{n,\sigma}^{\prime}}{d\mathbb{P}_{n}}d\mathbb{P}_{n}.$ From (4.11) of Proposition 4.1, we know that under the measure $\Psi_{n}(\cdot)$, $F_{n}(\sigma)\stackrel{{\scriptstyle p}}{{\to}}\operatorname{E}\left[f(Z_{k_{1}},\ldots,Z_{k_{l}})\right]$. Now from central limit theorem,

[TABLE]

where $Y$ is a Chi-squared random variable with $1$ degree of freedom. So by Slutsky’s theorem we have under the measure $\Psi_{n}$

[TABLE]

Further, from Hoeffding’s inequality we also have when $\beta J<\frac{1}{2}$, the sequence $\exp\left\{\frac{\beta J}{n}\left(\sum\sigma_{i}\right)^{2}\right\}$ is uniformly integrable. Since the random variables $F_{n}(\sigma)$’s are uniformly bounded, the sequence $F_{n}(\sigma)\exp\left\{\frac{\beta J}{n}\left(\sum\sigma_{i}\right)^{2}\right\}$ is also uniformly integrable. As a consequence,

[TABLE]

∎

Remark 4.1**.**

Along with Proposition 3.2, Proposition 4.2 is another important Result to prove Theorem 2.1. In particular, Proposition 4.2 allows us to verify condition $(iii)$ of Proposition 3.2 in the context of Theorem 2.1. One might observe that in proof of Proposition 4.2 there are two important facts. First of all, part (2) of Proposition 4.1 where one proves the asymptotic normality of the signed cycles holds with same parameter for almost all $\sigma$’s. This makes $F_{n}(\sigma)$ converge to $\operatorname{E}\left[f(Z_{k_{1}},\ldots,Z_{k_{l}})\right]$ for almost all $\sigma$’s. Secondly, it is also important that the partition function of the Curie-Weiss model in high temperature has a limit $\frac{1}{\sqrt{1-2\beta J}}$. Hence $\frac{1}{\tau_{n}}$ cancels out the $\frac{1}{\sqrt{1-2\beta J}}$ factor giving us the needed result.

5 Proof of Theorem 2.1

As mentioned in subsection 4.1, we at first prove a central limit theorem for $\log\left(\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\right)\left|\mathbb{P}_{n}\right.$ and finally proving $\log\left(\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\right)\left|\mathbb{P}_{n}\right.$ is asymptotically independent of $C_{n,1}$. The main idea is to use Proposition 3.2 to a class of measure $\tilde{\mathbb{Q}}_{n}$ which is close to $\mathbb{Q}_{n}$ in total variation distance. We now give a formal proof of Theorem 2.1.

**Proof of Theorem 2.1:

**We at first prove the central limit theorem for $\log\left(\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\right)\left|\mathbb{P}_{n}\right.$. The proof is broken into two steps as follows.

Step 1 (Construction of the measure $\tilde{\mathbb{Q}}_{n}$) : To begin with we shall consider a set $\Omega(\sigma)_{n}\subset\{-1,+1\}^{n}$ such that $\Psi_{n}\left(\Omega(\sigma)_{n}\right)\to 1$. The precise definition of $\Omega(\sigma)_{n}$ will be provided later. Now we consider the measure $\tilde{\mathbb{Q}}_{n}$ as follows

[TABLE]

where we define

[TABLE]

Since the sequence of random variables $\exp\left\{\frac{\beta J}{n}\left(\sum_{i=1}^{n}\sigma_{i}\right)^{2}\right\}$ is uniformly integrable it follows that for any sequence of sets $\Omega_{n}(\sigma)$ such that $\Psi_{n}\left[\Omega_{n}(\sigma)\right]\to 1,$

[TABLE]

Now we prove the sequences of measures $\mathbb{Q}_{n}$ and $\tilde{\mathbb{Q}}_{n}$ are close in the total variation sense. Let $A_{n}\in\mathcal{F}_{n}$ be a sequence of measurable sets. We have

[TABLE]

Observe that the final expression in (5.4) does not depend on the set $A_{n}$ and also it has been argued earlier that the final expression in (5.4) converges to [math]. By Proposition 4.2, under the measure $\tilde{\mathbb{Q}}_{n}$ the random variables for any $2\leq k_{1}<k_{2}\ldots<k_{l}=o\left(\sqrt{\log(n)}\right)$ with $l$ fixed,

[TABLE]

Now we prove that $\limsup_{n\to\infty}\operatorname{E}_{\mathbb{P}_{n}}\left[\left(\frac{d\tilde{\mathbb{Q}}_{n}}{d\mathbb{P}_{n}}\right)^{2}\right]\leq\exp\left\{\sum_{k=2}^{\infty}\frac{\mu_{k}^{2}}{\sigma_{k}^{2}}\right\}$ where $\mu_{k}=(2\beta)^{k}$. This will allow us to use Proposition 3.2 for $\tilde{\mathbb{Q}}_{n}$. In particular, we shall get $\left(\frac{d\tilde{\mathbb{Q}}_{n}}{d\mathbb{P}_{n}}\right)\left|\mathbb{P}_{n}\right.$ has a normal limiting distribution. Once this is done, the limiting distribution of $\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\left|\mathbb{P}_{n}\right.$ can be derived by the following arguments which proves

[TABLE]

Since both $\tau_{n}$ and $\tilde{\tau}_{n}$ have the same finite limit, the random variable

[TABLE]

has the same limiting distribution as $\frac{d\tilde{\mathbb{Q}}_{n}}{d\mathbb{P}_{n}}\left|\mathbb{P}_{n}\right.$. In particular,

[TABLE]

So it is enough to prove

[TABLE]

However,

[TABLE]

This completes the proof of

[TABLE]

Step 2 (Upper bounding $\operatorname{E}_{\mathbb{P}_{n}}\left[\left(\frac{d\tilde{\mathbb{Q}}_{n}}{d\mathbb{P}_{n}}\right)^{2}\right]$ ):

We know that

[TABLE]

Here $\Psi_{n}\otimes\Psi_{n}$ denote the two fold product of the uniform probability measure on $\{-1,1\}^{n}\times\{-1,1\}^{n}$.

Observe that the random variable

[TABLE]

where $Y_{1},Y_{2},Y_{3}$ are three independent chi-square random variables each with one degree of freedom. Our target is to prove the random variable in the l.h.s. of (5.10) is uniformly integrable. This done by proving

[TABLE]

for sufficiently small $\eta$. We at first write

[TABLE]

Here $\mathrm{T}$ denotes the transpose of a matrix and the matrix $A_{2\times n}$ is given by

[TABLE]

Since $\operatorname{E}\left[\exp\left\{\alpha^{\mathrm{T}}\sigma^{\prime}\right\}\right]\leq\exp\left\{\frac{1}{2}||\alpha||^{2}\right\}$ for any $\alpha\in\mathbb{R}^{n}$, we have the following tail estimate by Theorem 1 and Remark 1 of Hsu et al. [13]:

[TABLE]

where $\Sigma=\frac{1}{\sqrt{n}}A$. Observe that the nonzero eigenvalues of $\Sigma$ are same as the nonzero eigenvalues of $\frac{1}{n}AA^{\mathrm{T}}$. Now

[TABLE]

We now choose the set

[TABLE]

for some $\delta_{n}\to 0$ as $n\to\infty$. The existence of such $\Omega(\sigma)_{n}$ is ensured by weak law of large numbers. Now by Weyl’s interlacing inequality, we have the eigenvalues of $\frac{1}{n}AA^{\mathrm{T}}$ are given by $\left\{\beta J+O(\delta_{n}),2\beta^{2}+O(\delta_{n})\right\}$. Also note that on $\Omega(\sigma)_{n}$, $\mathrm{tr}(\Sigma)$ and $\mathrm{tr}(\Sigma^{2})$ remain uniformly bounded. So given any $\epsilon>0$ we can find a $t_{0}$ large enough such that

[TABLE]

for all $t>t_{0}.$ As a consequence, for all $t>t_{0}$

[TABLE]

where $\tilde{t}_{0}$ is another deterministic constant. Here the last step comes from replacing $(1+\eta)(1+\epsilon)2||\Sigma||t$ by $\log(t)$. Since $\max\left\{\beta J,2\beta^{2}\right\}<\frac{1}{2}$, we can choose $\epsilon$ and $\eta$ small enough such that

[TABLE]

As a consequence,

[TABLE]

On the other hand we can choose $\eta$ small enough such that $\beta J(1+\eta)<\gamma_{0}<\frac{1}{2}$. Now it is enough to prove that

[TABLE]

However we know that for any $t>0$,

[TABLE]

Here the last inequality is a straight forward application of Markov’s inequality. Now

[TABLE]

Observe that $\frac{1}{2\gamma_{0}}>1$. Hence by argument similar to (5.19) we have

[TABLE]

This completes the proof of uniform integrability of the random variable in the l.h.s. of (5.10). As a consequence,

[TABLE]

Plugging this into (5.9) we have

[TABLE]

where $\mu_{k}=(2\beta)^{k}$. Now using Proposition 3.2 with $W_{n,k}=C_{n,k+1}-(n-1)\mathbb{I}_{k=1}$, we have for the sequences of measures $\tilde{\mathbb{Q}}_{n}$ and $\mathbb{P}_{n}$

[TABLE]

where $Z_{k}\sim N\left(0,2(k+1)\right)$. Hence

[TABLE]

As we have proved earlier that $\frac{d\mathbb{\tilde{Q}}_{n}}{d\mathbb{P}_{n}}-\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\left|\mathbb{P}_{n}\right.\stackrel{{\scriptstyle p}}{{\to}}0$, this completes the proof of the asymptotic normality of $\log\left(\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}\right)\left|\mathbb{P}_{n}.\right.$

Proof of part (2) of Theorem 2.1: Before proving part $(1)$ of Theorem 2.1, we prove part $(2)$. Since

[TABLE]

in order to prove part $(2)$ of Theorem 2.1, we need to prove that

[TABLE]

We at first prove the result analogous to (5.29) for $\log\left(\frac{d\tilde{\mathbb{Q}}_{n}}{d\mathbb{P}_{n}}\right)$. (5.29) then follows from the fact that $\frac{d\mathbb{Q}_{n}}{d\mathbb{P}_{n}}-\frac{d\tilde{\mathbb{Q}}_{n}}{d\mathbb{P}_{n}}\left|\mathbb{P}_{n}\right.\stackrel{{\scriptstyle p}}{{\to}}0$ and an application of continuous mapping theorem.

By (3.6), for any given $\epsilon,\delta>0$ there exists $K=K(\epsilon,\delta)$ and for any subsequence $n_{l}$ there exists a further subsequence $n_{l_{q}}$ such that

[TABLE]

Now choose $K^{\prime}\geq K$ such that

[TABLE]

For any $K^{\prime}<k_{1}<k_{2}<m_{n}=o(\sqrt{\log n})$, the proof of Proposition 4.1 implies that $\operatorname{E}_{\mathbb{P}_{n}}\left[C_{n,k_{1}}\right]=0$, $\text{Cov}(C_{n,k_{1}},C_{n,k_{2}})=0$ and $\operatorname{Var}(C_{n,k_{i}})=2k_{i}(1+O({k_{i}^{2}}/{n}))$ for $i\in\{1,2\}.$ So

[TABLE]

Now for large values of $n_{l_{q}}$,

[TABLE]

Plugging in the estimates of (5.30) and (5.33) we have for all large values of $n_{l_{q}}$,

[TABLE]

Since (5.34) occurs to any subsequence and any $(\epsilon,\delta)$ pair, this completes the proof.

Proof of part (1) of Theorem 2.1: Consider the random variable

[TABLE]

where $W\sim N(0,\beta^{2})$ and is independent of the random variable

[TABLE]

Observe that from the proof of part $(2)$ we have

[TABLE]

So it is enough to prove that

[TABLE]

On the other hand for any fixed $K$,

[TABLE]

Since all the random variables $\beta C_{n,1}$, $\sum_{k=2}^{m_{n}}\frac{2\mu_{k}\left(C_{n,k}-(n-1)\mathbb{I}_{k=2}\right)-\mu_{k}^{2}}{4k}$ and $\sum_{k=2}^{K}\frac{2\mu_{k}\left(C_{n,k}-(n-1)\mathbb{I}_{k=2}\right)-\mu_{k}^{2}}{4k}$ have uniformly bounded second moments, they are tight. Hence we have any of their linear combination is also tight. Hence given any subsequence $n_{l}$ there exists a further subsequence $n_{l_{q}}$ such that

[TABLE]

Here the notation $M\{n_{l_{q}}\}$ means that the limiting distribution might possibly depend on the choice of the sub-sequence. On the other hand for every fixed $K$ there is a further subsequence $n_{{l_{q}}_{m}}$ (possibly dependent on $K$) such that

[TABLE]

where $M_{1}\stackrel{{\scriptstyle d}}{{=}}M\{n_{l_{q}}\}$ and $M_{2,K}\stackrel{{\scriptstyle d}}{{=}}W+\sum_{k=1}^{K-1}\frac{2\mu_{k+1}Z_{k}-\mu_{k+1}^{2}}{4(k+1)}$. Hence

[TABLE]

On the other hand by Fatou’s lemma,

[TABLE]

We know that for large enough value of $n_{{l_{q}}_{m}}$,

[TABLE]

Here we have used the identity $\operatorname{E}[X^{2}]=\operatorname{Var}[X]+\left(\operatorname{E}[X]\right)^{2}$. Given any $\epsilon>0$, we now choose $K$ large enough so that $\sum_{k=K+1}^{\infty}\frac{\mu_{k}^{2}}{2k}\leq\epsilon,$ implying $\operatorname{E}\left[(M_{1}-M_{2,K})^{2}\ \right]\leq\epsilon+\epsilon^{2}/4$. Hence the r.h.s. of (5.43) converges to [math] as $K\to\infty$. This implies $W_{2}\left(F^{M_{1}},F^{M_{2,K}}\right)\to 0$ as $K\to\infty$. Here $F^{M_{1}}$ and $F^{M_{2,K}}$ denote the distribution functions of $M_{1}$ and $M_{2,K}$ respectively. As a consequence, we have

[TABLE]

Hence $M\{n_{l_{q}}\}\stackrel{{\scriptstyle d}}{{=}}W+\sum_{k=1}^{\infty}\frac{2\mu_{k+1}Z_{k}-\mu_{k+1}^{2}}{4(k+1)}$ which does not depend on the specific choice of the subsequence $\{n_{l_{q}}\}$. This concludes the proof. $\square$

We now give proofs of Theorem 2.2 and Proposition 4.1

Appendix A Proof of Theorem 2.2

Although the proof of Theorem 2.2 is similar to the proof of Theorem 3.4 in Banerjee and Ma [6], we sketch the main details here for the shake of completeness. In order to complete the proof we first need some preliminary notations, definitions and some results. All these definitions can be found in Banerjee and Ma [6]. One might have a look at Section A.

The proof of Theorem 2.2 is divided into two parts depending upon $k$ being even or odd. We analyze each case separately. The case $k$ odd is almost similar to the case considered in Banerjee and Ma [6]. However the case $k$ even is easier here than the case considered in Banerjee and Ma [6]. This due to the fact that when $k$ is even there are words with $l(w)=2k+1$ such that $G_{w}$ is a tree. This creates additional complications. We discuss these things in details now.

As a general discussion, the fundamental idea is to show that $\operatorname{Tr}\left[\left(\frac{1}{\sqrt{n}}\right)^{2k+1}\tilde{A}^{2k+1}\right]$ can be written as a linear combination of $C_{n,r}$’s where $r$ is odd. In particular, one shows that the mean of $\operatorname{Tr}\left[\left(\frac{1}{\sqrt{n}}\right)^{2k+1}\tilde{A}^{2k+1}\right]$ is approximately [math] and the main contribution to the variance comes from a class of closed words which corresponds to some multiple of the $C_{n,r}$’s. One can show that the contribution by all the other words are negligible. The Chebyshev polynomial approximation is obtained by inverting this relation. On the other hand when $k$ is even, $\operatorname{Tr}\left[\operatorname{Tr}\left[\left(\frac{1}{\sqrt{n}}\right)^{2k}\tilde{A}^{2k}\right]\right]$ can be written as a linear combination of $C_{n,r}$’s plus an additional term which has two components. The first component is approximately equal to $k\psi_{2k}\left[\frac{1}{n}\operatorname{Tr}[\tilde{A}_{n}^{2}]-\operatorname{E}\left[\frac{1}{n}\operatorname{Tr}[\tilde{A}_{n}^{2}]\right]\right]$ and the second component gives a non-trivial contribution to the mean keeping the variance unchanged. These terms correspond to words in $\mathcal{W}_{1,2k}$ and $\mathcal{W}_{3,2k}$ in the proof. Finally the Chebyshev polynomial approximation still holds due to the fact

[TABLE]

This cancels out the term corresponding to $k\psi_{2k}\left[\frac{1}{n}\operatorname{Tr}[\tilde{A}_{n}^{2}]-\operatorname{E}\left[\frac{1}{n}\operatorname{Tr}[\tilde{A}_{n}^{2}]\right]\right]$.

We now elaborate the above discussion in some details.

1. (i)

$k$** is odd:**In this part to avoid confusion we shall use the notation $2k+1$ instead of using the terminology $k$ being odd. The proof is completed by first showing that $\operatorname{Tr}\left[\left(\frac{1}{\sqrt{n}}\right)^{2k+1}\tilde{A}^{2k+1}\right]$ can be approximated as a linear combination of the cycles $C_{n,i}$’s. Then one inverts the relation to get cycles as a linear combination of the traces.

We at first write down the following identity:

[TABLE]

where the graph $G_{w}$ has no self loops and for any closed word $w=(i_{0},i_{1},\ldots,i_{2k},i_{0})$ we define

[TABLE]

By a direct use of the parity principle (Lemma A.1 in Banerjee and Ma [6]) we get that $G_{w}$ can not be a tree. We now divide the class of words with $l(w)=2k+2$ in to two parts. In the first part we consider the cases when $G_{w}$ is an unicyclic graph such that each edge in the bracelet is repeated exactly once in particular $w\in\mathfrak{W}_{2k+2,r,t}$ for some $r$ odd and $2t-r=m$ (to be denoted by $\mathfrak{W}_{2k+2,r}$) and we denote the complement by $\mathcal{W}_{1,2k+1}$. In particular, we write

[TABLE]

Our fundamental goal is to show that

[TABLE]

and

[TABLE]

We at first analyze the second part of (A.4).

[TABLE]

Now $\operatorname{E}[X_{w}X_{x}]=0$ unless all the edges in $a=[w,x]$ are repeated at least twice. Observe that by Holder’s inequality and sub Gaussianity we have $\operatorname{E}[\left|X_{w}X_{x}\right|]\leq\left|A_{1,2}\right|^{4k+2}\leq\left(C_{1}k\right)^{C_{2}k}$. Here $C_{1}$ and $C_{2}$ are two deterministic constants. We divide this case into two further sub cases. First of all $w$ and $x$ shares an edge in particular $a$ is an weak CLT sentence If $\#V_{a}=t$ by Lemma A.5 in Banerjee and Ma [6], the number of such $a$ is bounded by $2^{4k+4}\left(C_{3}(4k+4)\right)^{C_{4}}\left(4k+4\right)^{3(4k+4-2t)}n^{t}$. By Proposition A.6 in Banerjee and Ma [6], $w,x\in\mathcal{W}_{1,2k+1}$, $a$ can not be a CLT word pair. Hence $\#V_{a}<2k+1$. Now consider the case when $w$ and $x$ don’t share an edge. In this case $\#V_{w}\leq\#E_{w}<\frac{2k+1}{2}$ and for any $t_{1}$ such that $\#V_{w}=t_{1}$ we have by Lemma A.4 in Banerjee and Ma [6] we have the cardinality of such $w$ is bounded by $2^{2k+1}(2k+1)^{3(2k-2t_{1}+3)}n^{t_{1}}$. Similarly the number of words $x$ such that for any $t_{2}$ such that $\#V_{x}=t_{2}$ is bounded by $2^{2k+1}(2k+1)^{3(2k-2t_{2}+3)}n^{t_{2}}$. As a consequence,

[TABLE]

whenever $k=o\left(\sqrt{\log n}\right)$. Hence we can neglect the second term in (A.4). Now we prove A.6.

First of all for any $w\in\mathfrak{W}_{2k+2,r}$, $\operatorname{E}[X_{w}]=0$ and if $w_{1}\in\mathfrak{W}_{2k+2,r_{1}}$ and $w_{2}\in\mathfrak{W}_{2k+2,r_{2}}$ where $r_{1}\neq r_{2}$ $\operatorname{E}[X_{w_{1}}X_{w_{2}}]=0$. Hence

[TABLE]

Here we have used Proposition A.6 in Banerjee and Ma [6] and Lemma A.10 in Banerjee and Ma [6]. On the other hand by proof of Proposition 4.1 we have

[TABLE]

Now

[TABLE]

by Lemma A.10 in Banerjee and Ma [6]. As a consequence,

[TABLE]

Here $\text{Poly}(k)$ is a deterministic polynomial in $k$. It is known that for example see Lang [16] that $f(m,r)\frac{m}{r}=\binom{m}{\frac{m+r}{2}}$ when $m$ and $r$ have same parity. By comparing coefficient it can be checked that

[TABLE]

Here $D$ is a lower triangular matrix and $D_{i,j}=P_{2i+1}[2j+1]$ where $P_{j}[i]$ is the coefficient of $z^{i}$ in $\left(z+\frac{1}{z}\right)^{j}$. Now

[TABLE]

Here

[TABLE]

and the last step of (A.14) follows from the fact that the coefficients of the Chebyshev polynomial grows at most exponentially. This completes the proof of the odd case. 2. (ii)

$k$** is even:** Here again to avoid confusion we use the notation $2k$ instead of using the term $k$ even. Like the $2k+1$ case we at first write

[TABLE]

However unlike the $2k+1$ case, in this case there are words $w$ such that $G_{w}$ is a tree. Keeping this in mind, we decompose $\operatorname{Tr}\left[\left(\frac{1}{\sqrt{n}}\right)^{2k}\tilde{A}^{2k}\right]$ in the following way:

[TABLE]

where

1. (a)

$\mathcal{W}_{1,2k}$ is the collection of all Wigner words(See Definition A.3 in Banerjee and Ma [6]). In this case $G_{w}$ is a tree and each is traversed exactly twice. 2. (b)

$\mathcal{W}_{2,2k}:=\cup_{r=4\leavevmode\nobreak\ :\leavevmode\nobreak\ r\leavevmode\nobreak\ \text{even}}^{2k}\mathfrak{W}_{2k+1,r}$. 3. (c)

$\mathcal{W}_{3,2k}$ is the collection of words such that $G_{w}$ is either an unicyclic graph with all edges repeated exactly twice or $G_{w}$ is a tree with exactly one edge repeated exactly four times and all other edges are repeated exactly twice. 4. (d)

$\mathcal{W}_{4,2k}$ is collection of all other words.

We at first prove that

[TABLE]

Note that

[TABLE]

Alike the odd case $\operatorname{E}[X_{w}X_{x}]=0$ unless all the edges in the sentence $a=[w,x]$ are repeated at least twice. Now we consider two cases firstly when the sentence $a$ is a weak CLT sentence i.e. the graphs $G_{w}$ and $G_{x}$ share an edge. In this case we prove that $\#V_{a}<2k$. We do a case by case analysis here. Since all the edges in $a$ are repeated at least twice $\#E_{a}\leq 2k$. Whenever $\#E_{a}=2k$, there are two cases when $\#V_{a}\geq 2k$. Firstly the graph $G_{a}$ is a tree and each edge is repeated exactly twice. This is impossible since otherwise in both the words $w$ and $x$ there will be an edge which are repeated exactly once however both $G_{w}$ and $G_{x}$ are tree. Now consider the other case when $\#E_{a}=\#V_{a}=2k$ in this case $G_{w}$ and $G_{x}$ are both unicyclic graphs with common bracelet. This is impossible by definition of $\mathcal{W}_{4,2k}$. Finally the when $G_{w}$ is a tree and $\#E_{a}=2k-1$ is also impossible since in this case the only possibility is both $w,x\in\mathcal{W}_{1,2k}$. As a consequence $\#V_{a}<2k$. Now we consider the other case when $G_{w}$ and $G_{x}$ don’t share an edge. In this case we have all the edges in both $G_{w}$ and $G_{x}$ are repeated at least twice. However all the words of such type having $\#V_{w}\geq k$ or $\#V_{x}\geq k$ are covered in $W_{1,2k}$, $W_{2,2k}$ and $W_{3,2k}$. Hence both $\#V_{w}$ and $\#V_{x}$ are strictly less than $k$. Now arguments exactly similar to the analysis of the second term of (A.4) proves (A.18). By arguments similar to the odd case, it can be proved that

[TABLE]

Next we prove that $\left(\frac{1}{n}\right)^{k}\sum_{w\in\mathcal{W}_{1,2k}}X_{w}-\operatorname{E}\left[\left(\frac{1}{n}\right)^{k}\sum_{w\in\mathcal{W}_{1,2k}}X_{w}\right]$ can be approximated by $k\psi_{2k}\left[\frac{1}{n}\operatorname{Tr}[\tilde{A}_{n}^{2}]-\operatorname{E}\left[\frac{1}{n}\operatorname{Tr}[\tilde{A}_{n}^{2}]\right]\right]$. This is done by second moment calculation. Observe that

[TABLE]

Now we analyze each term separately. First

[TABLE]

Since $\#\left(E_{w}\cap E_{x}\right)\geq 1$, we have $\#E_{a}\leq 2k-1$. Hence $\#V_{a}\leq 2k$. The equality $\#V_{a}=2k$ occurs when $w$ and $x$ share exactly one edge. We at first fix a word $w$, then the number of $x$’s such that $\#V_{a}=2k$ holds can be enumerated as follows. In the graph $G_{w}$ there are $k$ distinct edges we chose one of them which shares the edge with the word $x$. After fixing this edge we choose the equivalence class of the word $x$. There are $\psi_{2k}$ many of them and once an equivalence class is fixed, there are $k$ choices for the edge in $G_{x}$ to be shared. Now once this edge is also fixed there are two choices such that this edge is same as the chosen edge in $G_{w}$, one in the same order as the edge in $G_{w}$ and other in the reverse order. Now fixing all these choices there are $n^{k-1}\left(1+O\left(\frac{k^{2}}{n}\right)\right)$ choices for other vertices in $G_{x}$. As a consequence, given $w$ there are $2k^{2}\psi_{2k}n^{k-1}\left(1+O\left(\frac{k^{2}}{n}\right)\right)$ choices of $x$ such that $a$ is a weak CLT sentence and $\#V_{a}=2k$. Finally there are $\psi_{2k}n^{k+1}\left(1+O\left(\frac{k^{2}}{n}\right)\right)$. Combining all these, we have the number of $a$ such that $\#V_{a}=2k$ is given by $2k^{2}\psi_{2k}^{2}n^{2k}\left(1+O\left(\frac{k^{2}}{n}\right)\right)$. In this case $\operatorname{E}\left[\left(X_{w}-\operatorname{E}[X_{w}]\right)\left(X_{x}-\operatorname{E}[X_{x}]\right)\right]=2$ (by Gaussianity). Finally for all the cases the number of $a$’s such that $\#V_{a}=t$ is bounded by $n^{t}2^{4k+2}\left(C_{1}(4k+2)\right)^{C_{2}}\left(4k+2\right)^{3\left(4k+2-2t\right)}$. Finally for any $a$, $\operatorname{E}\left[\left|\left(X_{w}-\operatorname{E}[X_{w}]\right)\left(X_{x}-\operatorname{E}[X_{x}]\right)\right|\right]\leq\left(C_{3}k\right)^{C_{4}k}$ for some deterministic constant $C_{3}$ and $C_{4}$. Plugging all these estimates in (A.22) we have,

[TABLE]

where

[TABLE]

whenever $k=o\left(\sqrt{\log n}\right)$. Similar arguments can be used to prove that

[TABLE]

Plugging these estimates in (A.21) we have

[TABLE]

Finally for each word $w\in\mathcal{W}_{3,2k}$, we have $\#V_{w}=k$. These words give nontrivial contribution to the mean. However it can be checked that

[TABLE]

In particular, one also has an explicit expression for the mean

[TABLE]

Finally the proof for the even case can be completed by following the arguments similar to (A.14) and using the fact that

[TABLE]

One might check Banerjee and Ma [6] for a proof of (A.29).

$\square$

Appendix B Proof of Proposition 4.1

We at first give the proofs of part $(1)$ and $(3)$. The proof of part $(2)$ will be given separately.

Proof of part $(1)$ and $(3)$: We start with a very basic but fundamental observation. Note that for any $k\geq 3$

[TABLE]

It is easy to see that for $k\geq 2$, $\operatorname{Var}\left[C_{n,k}\right]\to 2k$. The proof is completed by method of moments and Wick’s formula. We at first give formal statements of these results: At first we state the method of moments.

Lemma B.1**.**

Let $(Y_{n,1},\ldots,Y_{n,l})$ be a sequence of random vectors of $l$ dimension. Then $(Y_{n,1},\ldots,Y_{n,l})\stackrel{{\scriptstyle d}}{{\to}}(Z_{1},\ldots,Z_{l})$ if the following conditions are satisfied:

1. i)

[TABLE]

exists for any fixed $m$ and $X_{n,i}\in\{Y_{n,1},\ldots,Y_{n,l}\}$ for $1\leq i\leq m$. 2. ii)

(Carleman’s Condition)**[8]**

[TABLE]

Further,

[TABLE]

Here $X_{n,i}\in\{Y_{n,1},\ldots,Y_{n,l}\}$ for $1\leq i\leq m$ and $X_{i}$ is the in distribution limit of $X_{n,i}$. In particular, if $X_{n,i}=Y_{n,j}$ for some $j\in\{1,\ldots,l\}$ then $X_{i}=Z_{j}$.

The method of moments is very well known and much useful in probability theory. We omit its proof.

Now we state the Wick’s formula for Gaussian random variables which was first proved by Isserlis(1918)[14] and later on introduced by Wick [24] in the physics literature in 1950.

Lemma B.2**.**

(Wick’s formula)Wick [24] Let $(Y_{1},\ldots,Y_{l})$ be a multivariate mean [math] random vector of dimension $l$ with covariance matrix $\Sigma$(possibly singular). Then $((Y_{1},\ldots,Y_{l}))$ is jointly Gaussian if and only if for any integer $m$ and $X_{i}\in\{Y_{1},\ldots,Y_{l}\}$ for $1\leq i\leq m$

[TABLE]

Here $\eta$ is a partition of $\{1,\ldots,m\}$ into $\frac{m}{2}$ blocks such that each block contains exactly $2$ elements and $\eta(i,j)$ denotes the $j$ th element of the $i$ th block of $\eta$ for $j=1,2$.

The proof of aforesaid lemma is also omitted.

We at first verify the asymptotic CLT, then variance calculation will be given. Given $1\leq k_{1}<k_{2}<\ldots<k_{l}=o\left(\sqrt{\log n}\right)$, we consider the random variables a $(Y_{n,1},\ldots,Y_{n,l})$ as follows:

[TABLE]

Now fix $m$ and take $X_{n,1},\ldots,X_{n,m}\in\{Y_{n,1},\ldots,Y_{n,l}\}$. Let $l_{1},\ldots,l_{m}$ be the corresponding lengths of the cycles. Now observe that

[TABLE]

By arguments similar to the arguments given right after (A.4), we have

[TABLE]

By Proposition A.2 in Banerjee and Ma [6], we have $\#V_{a}<\sum_{i=1}^{m}\frac{l_{i}}{2}$ unless $a$ is a CLT sentence. Further the CLT sentences only exists if $m$ is even. Finally, we prove we can neglect the sum corresponding all the sentences which are not CLT sentences.

[TABLE]

With (B.9) in hand, we are only left with CLT sentences. In particular for every $w_{i}$, there exists an unique $w_{j}$ such that $G_{w_{i}}$ shares an edge with $G_{w_{j}}$. On the other hand in these cases in order to get

[TABLE]

we need $G_{w_{i}}=G_{w_{j}}$. Further the random variables $X_{w_{i}}$ and $X_{w_{j}}$ are mutually independent if $G_{w_{i}}$ and $G_{w_{j}}$ are disjoint. Hence (B.7) asymptotically satisfies Wick’s formula with appropriate variance. This concludes the proof of part $(1)$ and $(3)$.

Proof of part $(2)$: We at first give a proof for $k=2$ case. Observe that under $\mathbb{Q}_{n,\sigma}$,

[TABLE]

Here $B_{i,j}\sim_{i.i.d.}N(0,1)$. By CLT we have for any $\sigma$,

[TABLE]

This completes the proof for $k=2$ case.

Now we move on to the other cases. Observe that under $\mathbb{Q}_{n,\sigma}$

[TABLE]

Here $V_{n,k,w}$ is obtained by expanding the product in (B.13) and taking all the residual terms in the product. Observe that $w=(i_{0},i_{1},\ldots,i_{k})$,

[TABLE]

Here for any edge $\{i,j\}$, $B_{e}=B_{i,j}$ and $\sigma_{e}={\sigma_{i}\sigma_{j}}$. Observe that for any $\sigma$, $\operatorname{E}[V_{n,k,w}]=0$. We now prove that

[TABLE]

We have that

[TABLE]

We now find an upper bound to $\operatorname{E}\left[V_{n,k,w}V_{n,k,x}\right]$. At first fix any word $w$ and the set $\emptyset\subsetneq E_{f}\subsetneq E_{w}$ and consider all the words $x$ such that $E_{{w}}\cap E_{x}=E_{w}\backslash E_{f}$. As every edge in $G_{w}$ and $G_{x}$ appear exactly once,

[TABLE]

The last inequality holds since $\#E^{\prime}\geq\#E_{f}$ and $\#(E_{w}\backslash E^{\prime}\subset E_{w}\backslash E_{f})\leq 2^{k}$.

Observe that the graph corresponding to the edges $E_{w}\backslash E_{f}$ is a disjoint collection of straight lines. Let the number of such straight lines be $\zeta$. Obviously $\zeta\leq\#(E_{w}\backslash E_{f})$. The number of ways these $\zeta$ components can be placed in $x$ is bounded by $k^{\zeta}\leq k^{\#(E_{w}\backslash E_{f})}$ and all other nodes in $x$ can be chosen freely. So there are at most $n^{k-\#V_{E_{w}\backslash E_{f}}}k^{\#(E_{w}\backslash E_{f})}$ choices of such $x$. Here $V_{E_{w}\backslash E_{f}}$ is the set of vertices of the graph corresponding to $(E_{w}\backslash E_{f})$. Observe that, whenever $k>\#E_{f}>0$, $E_{w}\backslash E_{f}$ is a forest so

[TABLE]

As a consequence,

[TABLE]

The right hand side of (B.18) does not depend on $E_{f}$ and there are at most $2^{k}$ nonempty subsets $E_{f}$ of $E^{w}$. So

[TABLE]

Finally there are at most $n^{k}$ many $w$. So

[TABLE]

Now we use the fact $k=o(\sqrt{\log(n)})$. In this case

[TABLE]

This concludes the proof. $\square$

Acknowledgment: The author acknowledges Prof. Jinho Baik for suggesting this problem to him while he was visiting U Michigan for a summer school. The author also thanks Prof. Wei Kuo Chen and the referees for comments.