Concentration inequalities for polynomials of contracting Ising models

TL;DR

This paper establishes concentration inequalities for degree-$d$ polynomials of spins in Ising models under contracting Glauber dynamics, extending known results for linear and quadratic cases to higher degrees.

Contribution

It provides new concentration bounds for polynomials of arbitrary fixed degree in Ising models with contracting dynamics, generalizing previous results for lower degrees.

Findings

Variance of the polynomial is $O(N^d)$.

Tail probabilities decay as $ ext{exp}(- c r^{2/d})$ for large deviations.

Results hold for fixed degree $d$ with $O(1)$ coefficients.

Abstract

We study the concentration of a degree-$d$ polynomial of the $N$ spins of a general Ising model, in the regime where single-site Glauber dynamics is contracting. For $d=1$, Gaussian concentration was shown by Marton (1996) and Samson (2000) as a special case of concentration for convex Lipschitz functions, and extended to a variety of related settings by e.g., Chazottes et al. (2007) and Kontorovich and Ramanan (2008). For $d=2$, exponential concentration was shown by Marton (2003) on lattices. We treat a general fixed degree $d$ with $O(1)$ coefficients, and show that the polynomial has variance $O(N^d)$ and, after rescaling it by $N^{-d/2}$, its tail probabilities decay as $\exp(- c\, r^{2/d})$ for deviations of $r \geq C \log N$.

Equations

$$\pi(\sigma)=\mathcal{Z}^{-1}\exp\left[-H(\sigma)\right]\quad\mbox{where}\quad\mbox{$H(\sigma)=-\sum_{i,j}J_{ij}\sigma_{i}\sigma_{j}$}\,,$$

$$\max_{\sigma,\sigma^{\prime}:\|\sigma-\sigma^{\prime}\|_{1}=1}W_{1}\Big{(}\mathbb{P}_{\sigma}(X_{1}\in\cdot),\mathbb{P}_{\sigma^{\prime}}(X_{1}\in\cdot)\Big{)}\leq\theta<1\,,$$

$$\mathbb{P}(|f-{\mathbb{E}}_{\pi }(f)|\ge u\sqrt{N})\le \exp (-c{u}^2).$$

$$\mathbb{P}(|f-{\mathbb{E}}_{\pi }(f)|\ge uN)\le \exp (-cu),$$

$$\sum_{j:j\sim i}|J_{ij}|\leq{1-\alpha}\quad\mbox{ for all $1\leq i\leq N$}$$

$${\mathrm{Var}}_{\pi }(f)\le C{K}^2{N}^d,$$

$$\mathbb{P}_{\pi}\Bigl{(}N^{-d/2}|f(\sigma)-\mathbb{E}_{\pi}[f(\sigma)]|\geq r\Bigr{)}\leq CN^{d^{2}}\exp\bigg{(}-\frac{r^{2/d}}{CK^{2/d}}\bigg{)}\,.$$

$$\sum a_{ij}{\sigma }_i{\sigma }_j=\sum a_{ij}({\sigma }_i-{\mu }_i)({\sigma }_j-{\mu }_j)+\sum a_{ij}{\sigma }_i{\mu }_j+\sum a_{ij}{\sigma }_j{\mu }_i-\sum a_{ij}{\mu }_i{\mu }_j.$$

$${\mathrm{Var}}_{\pi }(f)\le C\sum _{i,j}|a_{ij}{|}^2,$$

$$\mathbb{P}_{\pi}\Bigl{(}N^{-1}\big{|}f(\sigma)-\mathbb{E}_{\pi}[f(\sigma)]\big{|}>r\Bigr{)}\leq CN^{2}\exp\left(-\frac{r}{C\|A\|_{\infty}}\right)\,.$$

$$\text{gap}=\underset{f}{\inf }\frac{\mathcal{E}(f,f)}{{\mathrm{Var}}_{\pi }(f)}\text{mbox}where\mathcal{E}(f,f)=\frac{1}{2}\sum _{\sigma ,{\sigma }^{\prime }}\pi (\sigma )P(\sigma ,{\sigma }^{\prime }){\left|f(\sigma )-f({\sigma }^{\prime })\right|}^2.$$

$$\underset{\sigma ,{\sigma }^{\prime }}{\max }P(\sigma ,{\sigma }^{\prime })\le \gamma /N\text{mbox}forsome0<\gamma \le 1$$

$$\mathcal{E}(f,f)\le \frac{\gamma }{2N}\sum _i{\mathbb{E}}_{\pi }\left[({\nabla }_{i}f{)}^2(\sigma )\right],$$

$${\mathrm{Var}}_{\pi }(g)\le {\text{gap}}^{-1}\mathcal{E}(g,g)\le \frac{2\gamma }{\alpha }\sum _i|a_i{|}^2.$$

$$\mathcal{E}(f,f)\le \frac{2\gamma }{N}\sum _i{\mathrm{Var}}_{\pi }\left(g_i(\sigma )\right)\le \frac{4{\gamma }^2}{\alpha N}\sum _{i,j}|a_{ij}{|}^2,$$

$${\mathrm{Var}}_{\pi }(f)\le \frac{4{\gamma }^2}{{\alpha }^2}\sum _{i,j}|a_{ij}{|}^2.\text{qed}$$

$$|f(\sigma )-f({\sigma }^{\prime })|\le b\Vert \sigma -{\sigma }^{\prime }{\Vert }_1.$$

$$\underset{\sigma ,{\sigma }^{\prime }\in S:\Vert \sigma -{\sigma }^{\prime }{\Vert }_1=1}{\max }|f(\sigma )-f({\sigma }^{\prime })|\le b.$$

$$\frac{1}{N}|f(\sigma )-f({\sigma }^{\prime })|$$

$$\displaystyle\leq\frac{1}{N}\|\sigma-\sigma^{\prime}\|_{1}\Big{[}\|A\sigma\|_{\infty}+\|A^{T}\sigma^{\prime}\|_{\infty}\Big{]}\,,$$

$$S_b=\left\{\sigma :\max \left\{\Vert A\sigma {\Vert }_{\infty },\Vert A^T\sigma {\Vert }_{\infty }\right\}\le b\sqrt{N}\right\},$$

$${\mathbb{P}}_{\pi }\left(|g(\sigma )-{\mathbb{E}}_{\pi }[g(\sigma )]|>r\right)\le 2\exp \left(-\frac{(1-{\theta }^2)r^2}{2{\theta }^2{b}^2}\right).$$

$$\left|(A\sigma {)}_i-(A{\sigma }^{\prime }{)}_i\right|\le \Vert A{\Vert }_{\infty }\Vert \sigma -{\sigma }^{\prime }{\Vert }_1,$$

$${\mathbb{P}}_{\pi }(S_b^c)$$

$$\displaystyle\frac{1}{\sqrt{N}}\tilde{f}(\eta)=\min_{\sigma\in S_{b}}\bigg{[}\frac{1}{\sqrt{N}}f(\sigma)+2b\|\eta-\sigma\|_{1}\bigg{]}\,;$$

$${\mathbb{P}}_{\pi }\left(|\tilde{f}(\sigma )-{\mathbb{E}}_{\pi }[\tilde{f}(\sigma )]|>rN\right)\le 2e^{-r^2/(4\kappa b^2)}.$$

$$|{\mathbb{E}}_{\pi }[\tilde{f}(\sigma )]-{\mathbb{E}}_{\pi }[f(\sigma )]|$$

$$\le 12\Vert A{\Vert }_{\infty }{N}^3{e}^{-b^2/(\kappa \Vert A{\Vert }_{\infty }^2)},$$

$$\underset{\sigma }{\max }\{|f(\sigma )|,|\tilde{f}(\sigma )|\}\le \Vert A{\Vert }_{\infty }{N}^2+2b{N}^{3/2}\le 3\Vert A{\Vert }_{\infty }{N}^2.$$

$$|{\mathbb{E}}_{\pi }[\tilde{f}(\sigma )]-{\mathbb{E}}_{\pi }[f(\sigma )]|\le rN/3$$

$${\mathbb{P}}_{\pi }(|f(\sigma )-{\mathbb{E}}_{\pi }[f(\sigma )]|>rN)\le$$

Full text

1. Introduction

Concentration of measure for functions of random fields has been extensively studied (see, e.g., [8]). A prototypical example for a system where the underlying variables are weakly dependent is the high-temperature Ising model. The model, in its most general form without an external magnetic field, is a probability measure over configurations $\sigma\in\Omega_{N}:=\{\pm 1\}^{N}$ (assigning spins to the sites $\{1,\ldots,N\}$), defined as follows: for a set of coupling interactions $\{J_{ij}\}_{1\leq i,j\leq N}$, the corresponding Ising distribution $\pi$ is given by

[TABLE]

in which $\mathcal{Z}$ (the partition function) is a normalizer. For general $\{J_{ij}\}$ this includes ferromagnetic/anti-ferromagnetic models, and spin-glass systems on arbitrary graphs.

The Gaussian concentration of functions $f:\Omega_{N}\to\mathbb{R}$ in the high temperature regime has been studied both using analytical methods, adapting tools from the analysis of product spaces to the setting of weakly dependent random variables (see, e.g., [7, 12]), and using probabilistic tools such as coupling (cf. [1]). In the presence of arbitrary couplings $\{J_{ij}\}$, our hypothesis for capturing the high-temperature behavior of the model will be be based on contraction, as in the related works on concentration inequalities in [1, 10, 11, 13], and closely related to the Dobrushin uniqueness condition in [7].

Definition**.**

We say an Ising spin system $\pi$ is $\theta$-contracting if there exists a single-site discrete-time Markov chain $(X_{t})$ with stationary measure $\pi$ that is $\theta$-contracting, i.e.,

[TABLE]

where $W_{1}(\mu,\nu):=\inf\{\mathbb{E}[\|X-Y\|_{1}]:(X,Y)\sim(\mu,\nu)\}$ is the $L^{1}$-Wasserstein distance, and $\mathbb{P}_{\sigma}$ denotes the probability starting from an initial state $\sigma$.

The discrete-time heat-bath Glauber dynamics for the Ising model is the chain that, at every step, updates the spin of a uniformly chosen spin $i$ via $\mathbb{P}_{\pi}(\sigma_{i}\in\cdot\mid\sigma\mathord{\upharpoonright}_{\{1,\ldots,N\}\setminus\{i\}})$. It is well-known that, for the Ising model with interactions $J_{ij}$, if $\max_{i}\sum_{j}|J_{ij}|\leq 1-\alpha$, then the corresponding single-site heat-bath Glauber dynamics is $\theta$-contracting with $\theta=1-\alpha/N$, a concrete case where our results apply (see, e.g., [4, §8] and [9, §14.2]).

In this case, for linear functions $f(\sigma)=\sum_{i}a_{i}\sigma_{i}$, it is known, as a special case of results of Marton [11] regarding Gaussian concentration for Lipschitz functions (see also [13] as well as [1, 6, 7, 10]) that there exists $c=c(a_{1},\ldots,a_{N},\alpha)>0$ such that,

[TABLE]

For bilinear forms, where $f(\sigma)=\sum_{ij}a_{ij}\sigma_{i}\sigma_{j}$, Marton [12] showed that on lattices

[TABLE]

whereas Daskalakis et al. [3] showed that, for a general Ising model, in a subset of this regime (contraction as above with $\alpha>\frac{3}{4}$ vs. any $\alpha>0$), $\operatorname{Var}_{\pi}(f)=O(N^{2}\log^{3}N)$.

Our main result recovers the correct variance and, up to a polynomial pre-factor, the tail probabilities for a polynomial of any fixed degree $d$ (for matching lower bounds, one can take, for instance, the $d$-th power of the magnetization $f(\sigma)=\sum_{i}\sigma_{i}$).

Theorem 1**.**

For every $\alpha,d>0$ there exists $C(\alpha,d)>0$ so that the following holds. Let $\pi$ be the distribution of the Ising model on $N$ spins with couplings $\{J_{ij}\}$ satisfying

[TABLE]

For every polynomial $f\in\mathbb{R}[\sigma_{1},\ldots,\sigma_{N}]$ of total-degree $d$ with coefficients in $[-K,K]$,

[TABLE]

and for every $r>0$,

[TABLE]

Moreover, (1.2)–(1.3) hold for every Ising model with couplings $\{J_{ij}\}$ for which the corresponding ferromagnetic model with interactions $\{|J_{ij}|\}$ is $(1-\tfrac{\alpha}{N})$-contracting.

Remark 1.1**.**

In [3], the authors used their variance bounds for bilinear forms of Ising models to study statistical independence testing for Ising models. Namely, they gave bounds (in terms of $N$ and $\varepsilon$) on the number of samples that are required to distinguish, with high probability, between a product measure and an Ising model whose (symmetrized Kullback-Leibler) distance to any product measure is at least $\varepsilon$. In Section 4, Theorems 4.1–4.2, we present a short application of Theorem 1 to improve the upper bounds of [3] by considering fourth-order statistics of the Ising model.

Remark 1.2**.**

In this paper, we always consider polynomials of Ising models with no external field. As the following example shows, in the presence of an external field, such polynomials can be anti-concentrated. Let $\mu_{i}=\mathbb{E}[\sigma_{i}]$ for all $i$ and expand,

[TABLE]

The first term on the right-hand side should have $O(N)$ fluctuations while the second and third terms $\sum_{i}(\sum_{j}a_{ij}\mu_{j})\sigma_{i}$ can have order $N^{3/2}$ fluctuations (e.g., if $(\mu_{j}a_{ij})_{j}$ all have the same sign), implying (1.2)–(1.3) cannot hold in general under external field.

2. Concentration for quadratic functions

In this section, we prove the special and more straightforward case of concentration for quadratic functions of the Ising model. The proof of Theorem 1 in §3 requires some additional ingredients but is motivated by the proof of the following.

Theorem 2.1**.**

For every $\alpha>0$ there exists $C(\alpha)>0$ so that the following holds. Let $\pi$ be the distribution of the Ising model on $N$ spins with interaction couplings $\{J_{ij}\}$ satisfying (1.1). For $A=\{a_{ij}\}_{i,j=1}^{N}$, the function $f(\sigma)=\sum_{i,j}a_{ij}\sigma_{i}\sigma_{j}$ on $\Omega_{N}$ satisfies

[TABLE]

and for every $r>0$,

[TABLE]

Furthermore, this holds for any $\{J_{ij}\}$ such that the Ising model is $(1-\tfrac{\alpha}{N})$-contracting.

Proof of (2.1).

Recall that the variational formula for the spectral gap of a reversible Markov chain $(X_{t})$ with transition kernel $P$ and stationary distribution $\pi$ states that

[TABLE]

For any single-site discrete-time Markov chain for the Ising model, one has that

[TABLE]

(for example, under assumption (1.1), heat-bath Glauber dynamics satisfies this for a choice of $\gamma=\left[1+\tanh(2(1-\alpha))\right]/2$). Thus,

[TABLE]

where $(\nabla_{i}f)(\sigma):=f(\sigma)-f(\sigma^{i})$ with $\sigma^{i}$ the state obtained from $\sigma$ by flipping $\sigma_{i}$. Moreover, as mentioned, since this chain satisfies (1.1), it is $(1-\frac{\alpha}{N})$-contracting and therefore has $\text{\tt{gap}}\geq\alpha/N$ by the results of [2] (see also [9, Theorem 13.1]).

Consider a linear function of the form $g=\sum a_{i}\sigma_{i}$; since $|\nabla_{i}g|=2|a_{i}|$, one obtains that $\mathcal{E}(g,g)\leq 2\gamma N^{-1}\sum_{i}|a_{i}|^{2}$, and therefore (2.3) implies that

[TABLE]

Returning to the function $f$, assume w.l.o.g. that $a_{ii}=0$ for all $i$ (as $\sigma_{i}^{2}=1$) and let $g_{i}(\sigma):=\sum_{j}(a_{ij}+a_{ji})\sigma_{j}$, so $|(\nabla_{i}f)(\sigma)|=2|g_{i}(\sigma)|$. By symmetry, $\mathbb{E}_{\pi}[g_{i}(\sigma)]=0$, thus

[TABLE]

which, again applying (2.3), yields

[TABLE]

We now proceed to proving the exponential tail bounds on $f$. Throughout the paper, we say a function $f$ is $b$-Lipschitz on a set $S$ if for every $\sigma,\sigma^{\prime}\in S$,

[TABLE]

A function $f$ is $b$-Lipschitz if it is so on its whole domain, in our case $\Omega_{N}$. For subsets of a graph, e.g., $\{\pm 1\}^{N}$, endowed with the graph distance, by the triangle inequality, it suffices to consider only $\sigma,\sigma^{\prime}$ that are neighbors. Then $f$ is $b$-Lipschitz on a connected set $S\subset\Omega_{N}$ if

[TABLE]

Proof of (2.2).

We begin by bounding the Lipschitz constant of $\frac{1}{N}f$. Observe that

[TABLE]

in light of which, if we define

[TABLE]

then $\frac{1}{\sqrt{N}}f$ is $2b$-Lipschitz on $S_{b}$—note that we only consider $b\leq\|A\|_{\infty}\sqrt{N}$.

In order to upper bound $\mathbb{P}_{\pi}(S_{b}^{c})$, we will use the following version of concentration inequalities for Lipschitz functions of contracting Markov chains [10]:

Proposition 2.2** ([10, Corollary 4.4, Eq. (4.13)], cf. [11, 13]).**

Let $\pi$ be the stationary distribution of a $\theta$-contracting Markov chain with state space $\Omega$, and suppose $g:\Omega\to\mathbb{R}$ is $b$-Lipschitz. Then for all $r>0$,

[TABLE]

To see this, note that for every $i$ and every $\sigma,\sigma^{\prime}\in\Omega_{N}$,

[TABLE]

and so $\sigma\mapsto(A\sigma)_{i}$ is $\|A\|_{\infty}$-Lipschitz, and similarly $\sigma\mapsto(A^{T}\sigma)_{i}$ is $\|A\|_{\infty}$-Lipschitz. By a union bound and Proposition 2.2 with $\theta=1-\alpha/N$, there exists $\kappa(\alpha)>0$ such that

[TABLE]

Next, consider the McShane–Whitney extension of $N^{-1/2}f$ from $S_{b}$, given by

[TABLE]

by definition, $N^{-1/2}\tilde{f}$ is $2b$-Lipschitz on all of $\Omega_{N}$. As a result, by Proposition 2.2,

[TABLE]

In order to move to the desired quantity, we need to control the difference between the means of $f,\tilde{f}$ using the fact that $\tilde{f}(\sigma)=f(\sigma)$ for all $\sigma\in S_{b}$:

[TABLE]

where in the last line we used (2.8) to bound $\mathbb{P}_{\pi}(S_{b}^{c})$, as well as that

[TABLE]

Now let $b=\sqrt{{\|A\|_{\infty}r}/6}$ and observe that if $b$ is such that

[TABLE]

holds (in particular, this holds for all $b>2\sqrt{\kappa\|A\|^{2}_{\infty}\log(\|A\|_{\infty}N)}$), then

[TABLE]

By (2.10), and the choice of $b$, the first term above has

[TABLE]

Because $\tilde{f}(\sigma)=f(\sigma)$ for all $\sigma\in S_{b}$, by our choice of $b$,

[TABLE]

Replacing the requirement of $b>2\sqrt{\kappa\|A\|^{2}_{\infty}\log(\|A\|_{\infty}N)}$ with a prefactor of $N^{2}$, and combining the above two estimates, we see that

[TABLE]

holds for every $r>0$. ∎

3. Concentration for general polynomials

In order to prove Theorem 1, we will need the following intermediate lemma used to control the mean of the gradient of $f$.

Lemma 3.1**.**

For every $p,\alpha>0$ there exists $C(\alpha,p)>0$ such that the following holds. Consider an Ising model $\pi$ with couplings $\{J_{ij}\}$ and let $\tilde{\pi}$ be the Ising measure corresponding to couplings $\{|J_{ij}|\}$. If $\tilde{\pi}$ is a $(1-\frac{\alpha}{N})$-contracting Ising system and

[TABLE]

is a degree-$p$ polynomial in $(\sigma_{1},...,\sigma_{N})$ for a degree-$p$ tensor $B$, then

[TABLE]

Proof.

Begin by considering ferromagnetic models with non-negative couplings, $\{J_{ij}\}$. It is well-known that in the $\mathbb{E}_{\pi}[\sigma_{i_{1}}\cdots\sigma_{i_{p}}]\geq 0$ in the ferromagnetic Ising model with no external field (e.g., by viewing its FK representation that enjoys monotonicity). Thus,

[TABLE]

and taking $M_{p}=(\|B\|_{\infty})^{1/p}$, we see that

[TABLE]

However, $\sum_{i}M_{p}\sigma_{i}$ is clearly an $M_{p}$-Lipschitz function, and by spin-flip symmetry of the Ising system, has mean [math], so by Proposition 2.2, there exists $\kappa(\alpha)>0$ such that

[TABLE]

and therefore, by integrating, $\mathbb{E}_{\pi}[|\sum_{i}M_{p}\sigma_{i}|^{p}]\leq C\|B\|_{\infty}N^{p/2}$ for some $C(\alpha,p)>0$.

Now suppose that $\{J_{ij}\}$ are not all non-negative; using the FK representation of Ising spin systems with general couplings (not necessarily ferromagnetic)—see, e.g., [5, §11.5], and in particular Proposition 259 and Eq. (11.44)—for every $i_{1},...,i_{p}$,

[TABLE]

Then, proceeding as before, we see that

[TABLE]

Since $\tilde{\pi}$ is contracting, we can apply Proposition 2.2 as before to obtain for the same constant, $C(p,\alpha)>0$ that

[TABLE]

Proof of (1.2).

Fix $d$ and recall the variational formula for the spectral gap, (2.3). Following (2.5), we see that for $\gamma$ defined in (2.4)

[TABLE]

where $(\nabla_{\ell}f)(\sigma)=f(\sigma)-f(\sigma^{\ell})$ as before. Let

[TABLE]

with $\|A\|_{\infty}\leq K$, and w.l.o.g. (since $\sigma_{i}^{2}=1$, every polynomial can be rewritten as a sum of monomials) assume that $a_{i_{1},...,i_{d}}=0$ if $i_{k}=i_{j}$ for some $j\neq k$. Then we see that for every $\ell$ and every $\sigma$,

[TABLE]

so that $g_{\ell}(\sigma):=(\nabla_{\ell}f)^{2}(\sigma)$ is a $2(d-1)$-degree polynomial in $\sigma$ with coefficients bounded above by $4\binom{2(d-1)}{(d-1)}K^{2}$. By Lemma 3.1, there exists $C(\alpha,d)>0$ such that for every $\ell$,

[TABLE]

so that using (2.3), (2.5), and the fact that $\text{\tt{gap}}\geq\alpha/N$, for some new $C(\alpha,d)>0$,

[TABLE]

Proof of (1.3).

Observe that since we are on the hypercube $\Omega_{N}$, $\sigma_{i}^{k}=\sigma_{i}^{k\mod 2}$, so that every polynomial function $f$ of degree $d$ can be rewritten as a sum of monomials of degree at most $d$. The concentration of the lower-degree monomials can be absorbed into a constant multiple in the prefactor in (1.3) of Theorem 1. Moreover, it suffices by rescaling to prove the theorem for the case $K=1$. Hence, we proceed to prove the following concentration inequality for monomials: consider a $(1-\frac{\alpha}{N})$-contracting Ising model $\pi$; for every $d$, if $f$ is a monomial of degree $d$, i.e.,

[TABLE]

for a $d$-tensor $A$ with $\|A\|_{\infty}\leq 1$ and $a_{i_{1}...i_{d}}=0$ if $i_{j}=i_{k}$ for some $j\neq k$, there exists $C(\alpha,d)>0$ such that for every $r>0$, and every $N$,

[TABLE]

Since we are considering $d$ fixed, throughout this section, $\lesssim$ will be with respect to constants that may depend on $d$. We prove (3) inductively over $d\geq 2$. The base case $d=1$ is given by Proposition 2.2. Now assume that for every $p\leq d-1$, Eq. (3) holds and show it holds for $d$. Fix $1\leq\ell\leq N$ and let $\sigma^{\ell}$ be the configuration that differs with $\sigma$ only in coordinate $\ell$. For every $\sigma$, we can compute the gradient $N^{-d/2}(\nabla_{\ell}f)(\sigma)$ as

[TABLE]

Define the following set of configurations:

[TABLE]

Because $S_{b}$ may not be connected, Eq. (3) does not necessarily bound the Lipschitz of $f$ on $S_{b}$. Thus, for each $\eta\in S_{b}$, we set $S_{\eta,b}$ to be the connected component of $S_{b}$ containing $\eta$. By definition of $S_{\eta,b}$, the triangle inequality, and (3), for each $\eta\in S_{b}$, function $N^{-(d-1)/2}f$ is $db$-Lipschitz function on $S_{\eta,b}$.

For every $\eta$, define the McShane–Whitney extension of $N^{-(d-1)/2}f$ from $S_{\eta,b}$ as

[TABLE]

so that $N^{-(d-1)/2}\tilde{f}_{\eta}$ is ${db}$-Lipschitz on all of $\Omega_{N}$ and $\tilde{f}_{\eta}\mathord{\upharpoonright}_{S_{\eta,b}}=f\mathord{\upharpoonright}_{S_{\eta,b}}$.

Now let $(X_{t})$ be the single spin-flip Markov chain which we assumed to be $(1-\frac{\alpha}{N})$-contracting with stationary distribution $\pi$, and, for each $\eta$, bound

[TABLE]

where

[TABLE]

In order to bound $\Phi_{1}$ we will need the following result of Luczak [10]:

Proposition 3.2** ([10, Eq. (4.14)]).**

Suppose $(Y_{t})$ is a $\theta$-contracting Markov chain on $\Omega$ with stationary distribution $\pi$; suppose further that $g:\Omega\to\mathbb{R}$ is a $b$-Lipschitz function. Then for every $Y_{0}\in\Omega$,

[TABLE]

By Proposition 3.2 with the choice of $\theta=1-\frac{\alpha}{N}$, there exists $\kappa(\alpha)>0$ such that for every $\eta\in S_{b}$ and every $t$,

[TABLE]

Second, the fact that $f$ and $\tilde{f}_{\eta}$ identify on $S_{\eta,b}$ implies that

[TABLE]

where the last equality crucially used that $(X_{t})$ is a single-site dynamics (whence starting from $\eta$, exiting $S_{\eta,b}$ and exiting $S_{b}$ are equivalent).

By the definition of $\tilde{f}_{\eta}$, we have that $\|\tilde{f}_{\eta}\|_{\infty}\leq\|f\|_{\infty}+N\mathrm{Lip}(f\mathord{\upharpoonright}_{S_{\eta,b}})$, implying that

[TABLE]

Finally, if we take

[TABLE]

we have,

[TABLE]

so that for all such $t$, for every $\eta\in\Omega_{N}$, we have $\Psi_{2}=0$. Because (e.g.,  [9], a Markov chain that is $\theta$-contracting with $\theta=1-\frac{\alpha}{N}$ has $t_{\textsc{mix}}\gtrsim N\log N$) by sub-multiplicativity of total variation distance to stationarity, this holds for $t_{0}\asymp N\log^{2}(N)$.

Combining  (3.5)–(3.8), we see that for all $\eta\in S_{b}$ and $t\geq t_{0}$,

[TABLE]

If we now average both sides over $\eta\sim\pi$ and set $t=t_{0}$, we obtain

[TABLE]

where we used using stationarity of the Markov chain and a union bound over all times up to $t_{0}$, and Markov’s inequality with $\mathbb{E}_{\pi}[\mathbb{P}_{\eta}(\tau_{S_{b}^{c}}\leq t)]=\mathbb{P}_{\pi}(\tau_{S_{b}^{c}}\leq t)$.

It remains to bound the probability $\mathbb{P}_{\pi}(S_{b}^{c})$. Let, for every $1\leq\ell\leq N$, $1\leq j\leq d$,

[TABLE]

by the inductive hypothesis there exists $C^{\prime}(\alpha,d)>0$ such that uniformly over $\ell,j$,

[TABLE]

To upper bound $\mathbb{P}_{\pi}(S_{b}^{c})$, by (3.4) it suffices to show that $|\mathbb{E}_{\pi}[g_{\ell,j}]|$ is at most $bN^{(d-1)/2}/2$ and then union bound over $\ell,j$. Since for each $\ell,j$, the function $g_{\ell,j}$ is a $d-1$ degree polynomial of the form of $h(\sigma)$ in Lemma 3.1 there exists $C(\alpha,d)>0$ such that

[TABLE]

Therefore, for all $b\geq 2C$, by a union bound over $1\leq\ell\leq N$ and $1\leq j\leq d$,

[TABLE]

Plugging (3.10) into (3), by stationarity of $\pi$ and $t_{0}\asymp dN\log^{2}(N)$, we obtain

[TABLE]

at which point, the choice of $b$ given by

[TABLE]

implies the desired (3) for some different $C(\alpha,d)>0$ for all $r>0$. ∎

4. An application to testing Ising models

In [3], independence testing of Ising models was extensively studied. Namely, suppose one is given $k$ samples of $N$ bits, either from a product measure $\mathcal{I}$ or from an Ising measure $\nu$ satisfying (1.1) whose Kullback–Leibler distance to $\mathcal{I}$ is at least $\varepsilon$. The goal is to decide with high probability, using a minimum number of samples, which distribution the samples came from. Our variance bound in Theorem 1 allows us to use a fourth-order statistic to improve on the results of [3] in the high-temperature regime of (1.1), including obtaining the sharp result in the case of ferromagnetic Ising models.

Consider an Ising model with couplings $J_{ij}$ and for every $i\sim j$, denote by

[TABLE]

which in the absence of external field equals $\mathbb{E}_{\pi}[\sigma_{x}\sigma_{y}]$. We will be concerned with Ising models satisfying (1.1) and therefore in their high-temperature Dobrushin regime.

The Ising model has the special property that for two Ising models $\pi$ and $\nu$ on $N$ vertices, with couplings $\{J_{ij}^{\pi}\}$ and $\{J_{ij}^{\nu}\}$ and edge-magnetizations $\lambda_{ij}^{\pi}$ and $\lambda_{ij}^{\nu}$, the symmetrized Kullback–Leibler divergence $d_{\mathrm{SKL}}(\pi,\nu)$ is given by

[TABLE]

Let $\mathcal{I}$ be the product measure on $N$ independent, symmetric $\pm 1$ random variables. That is to say that $J_{ij}^{\mathcal{I}}=\lambda_{ij}^{\mathcal{I}}=0$ for all $i,j$ and $d_{\mathrm{SKL}}(\pi,\mathcal{I})=\sum_{i,j}J_{ij}^{\pi}\lambda_{ij}^{\pi}$. Finally, for an Ising model $\pi$, let $m$ denote the number of edges, i.e., the number of non-zero $J_{ij}^{\pi}$.

Theorem 4.1**.**

There exists a polynomial time algorithm that uses $O(N/\varepsilon)$ samples from a ferromagnetic Ising model $\pi$ on $N$ vertices satisfying (1.1), and distinguishes with probability better than $\frac{3}{4}$, whether $\pi=\mathcal{I}$ or $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$. In the specific case where the edge set $\{(ij):J^{\pi}_{ij}\neq 0\}$ is known, this is improved to $O(\sqrt{m}/\varepsilon)$ samples.

Theorem 4.2**.**

There exists a polynomial time algorithm that uses $O(N^{2}/\varepsilon^{2})$ samples from an Ising model $\pi$ on $N$ vertices satisfying (1.1), and distinguishes with probability better than $\frac{3}{4}$ whether $\pi=\mathcal{I}$ or $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$. In the specific case where the edge set $\{(ij):J^{\pi}_{ij}\neq 0\}$ is known a priori, this is improved to $O(N\sqrt{m}/\varepsilon^{2})$ samples.

(The previous results of [3] gave a bound of $O(m/\varepsilon)$ in the setting of Theorem 4.1, and a bound of $O(N^{10/3}/\varepsilon^{2})$ in the setting of Theorem 4.2.)

The algorithms we use take $k$ i.i.d. samples $(\sigma^{(1)}_{i})_{i\leq N},...,(\sigma^{(k)}_{i})_{i\leq N}$ from $\pi$ and compute the test statistic,

[TABLE]

where in the case where we do know the edge set of the underlying graph a priori, we sum only over $i\sim j$. Let $\mathbb{P}$ be the measure given by $\bigotimes_{i=1}^{k}\pi$.

Observe first that

[TABLE]

At the same time,

[TABLE]

For every fixed $k$, we can view $(\sigma_{i}^{(\ell)})_{1\leq i\leq N,1\leq\ell\leq k}$ as an Ising model on $kN$ vertices, that satisfies (1.1) since it corresponds to $k$ independent copies of an Ising model each satisfying (1.1). Therefore, by Theorem 1, specifically (1.2), we have $\mathrm{Var}(Z_{k})\leq CN^{2}/k^{2}$.

In the specific case where the underlying graph of the Ising model is known a priori, we have the following.

Lemma 4.3**.**

Consider $k$ i.i.d. samples $\sigma^{(1)},...,\sigma^{(k)}$ from an Ising model $\pi$ on a graph $G$ on $N$ vertices and $m$ edges, satisfying (1.1). Then there exists $C(\alpha)>0$ such that $\mathrm{Var}(Z_{k})\leq Cm/k^{2}$.

Proof.

Again view $(\sigma_{i}^{(\ell)})_{i,\ell}$ as an Ising model on $kN$ vertices with measure $\pi^{k}=\bigotimes_{i=1}^{k}\pi$. Recall that since $\{J^{\pi}_{ij}\}$ satisfy (1.1) for $\alpha>0$, the Ising model is $1-\alpha/N$ contracting. Since the spectral gap tensorizes, and $\pi$ is $1-\alpha/N$ contracting, $\pi^{k}$ also has inverse spectral gap satisfying $\text{\tt{gap}}^{-1}\geq\alpha/N$. Using the variational form of the spectral gap as before, we have by (2.4)–(2.5),

[TABLE]

Now we compute $(\nabla_{i,\ell}Z_{k})^{2}(\sigma)$ for fixed $(i,\ell)=(i^{\star},\ell^{\star})$ and every $\sigma$. Expanding out,

[TABLE]

When $\ell=\ell^{\prime}$, the summands in the second sum are given by $\mathbb{E}_{\pi}[\sigma_{j}\sigma_{j^{\prime}}]$, whereas when $\ell\neq\ell^{\prime}$, we have $\mathbb{E}[\sigma_{i^{\star}}^{\ell}\sigma_{j}^{\ell}\sigma_{i^{\star}}^{\ell^{\prime}}\sigma_{j}^{\ell^{\prime}}]=\mathbb{E}_{\pi}[\sigma_{i^{\star}}\sigma_{j}]\mathbb{E}_{\pi}[\sigma_{i^{\star}}\sigma_{j^{\prime}}]$. Therefore,

[TABLE]

where $\tilde{\pi}$ is the ferromagnetic analogue of $\pi$ with couplings $J_{ij}^{\tilde{\pi}}=|J_{ij}^{\pi}|$ (implying it also satisfies (1.1) with the same $\alpha$) and the last inequality follows as in (3.1) from the FK representation. But, we can write

[TABLE]

where $c_{j}=\boldsymbol{1}\{J_{i^{\star}j}\neq 0\}$. For squares of $1$-Lipschitz functions of contracting Ising models, we previously noted in (2.6) that

[TABLE]

with $d_{i^{\star}}$ being the number of nonzero couplings incident $i^{\star}$. Summing over $i^{\star}$, and plugging this bound into (4) and then into the variational form of the spectral gap, we obtain the desired bound

[TABLE]

We are now in position to prove the two theorems regarding independence testing for the Ising model.

Proof of Theorem 4.1.

The algorithm we use computes $Z_{k}$ as defined in (4.1) for $k\geq CN/\varepsilon$ (when we know the underlying graph, $k\geq C^{\prime}{\sqrt{m}/\varepsilon}$), then outputs that $\pi=\mathcal{I}$ if $Z_{k}\leq\varepsilon/4$ and outputs $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$ otherwise. We first show that with probability at lteast $\frac{9}{10}$, if $\pi=\mathcal{I}$, the algorithm outputs that. Notice that $\mathbb{E}_{\mathcal{I}}[Z_{k}]=0$, and by the above computations of the variance, $\mathrm{Var}(Z_{k})\leq CN^{2}/k^{2}$ (when we know the underlying edge set, $\mathrm{Var}(Z_{k})\leq m/k^{2}$ by Lemma 4.3). By Chebyshev’s inequality,

[TABLE]

which, after plugging in the two above bounds on $\mathrm{Var}(Z_{k})$ implies the number of samples we require of $k$ is sufficient for the right-hand side to be at most $\frac{9}{10}$.

When $\pi$ is such that $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$, we again have the same bounds on $\mathrm{Var}(Z_{k})$. We now lower bound $\mathbb{E}_{\pi}[Z_{k}]$ by (4.2) and the definition of $d_{\mathrm{SKL}}(\pi,\mathcal{I})$. Note that since $\pi$ is a ferromagnetic, for all $J_{ij}^{\pi}\leq 1$ by the FKG inequality of the ferromagnetic Ising model, $\lambda_{ij}^{\pi}\geq\tanh(J_{ij}^{\pi})\geq J_{ij}^{\pi}/2$. As a result,

[TABLE]

Applying Chebyshev’s inequality to $\mathbb{P}(Z_{k}\leq\varepsilon/4)$, we see that the desired number of samples we require of $k$ is sufficient to identify in this case that $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$ with probability at least $\frac{9}{10}$. A union bound over the two cases $\pi=\mathcal{I}$ and $\pi$ such that $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$ concludes the proof. ∎

Proof of Theorem 4.2.

The algorithm again computes the test statistic, $Z_{k}$ defined in (4.1), and now outputs that $\pi=\mathcal{I}$ if $Z_{k}\leq\varepsilon^{2}/2N$ and outputs $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$ otherwise.

First, consider the situation $\pi=\mathcal{I}$; by similar reasoning to the proof of Theorem 4.1, after $k\geq CN^{2}/\varepsilon^{2}$, (when we know the underlying graph, $k\geq C^{\prime}N\sqrt{m}/\varepsilon$, with probability at least $\frac{9}{10}$, the algorithm outputs that $\pi=\mathcal{I}$.

Now suppose that $\pi$ is such that $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$; we wish to lower bound $\mathbb{E}[Z_{k}]$. By Cauchy–Schwarz inequality,

[TABLE]

When (1.1) holds, we know that for every $i$ and some $\alpha>0$, we have $\sum_{j:j\sim i}|J_{ij}^{\pi}|\leq 1-\alpha$. Therefore,

[TABLE]

We can then use Chebyshev’s inequality to bound

[TABLE]

via the aforementioned bounds on $\mbox{Var}(Z_{k})$. Plugging in those bounds implies that the number of samples $k$ we require is sufficient to identify that in this case $d_{\mathrm{SKL}}(\pi,\mathcal{I})\geq\varepsilon$ with probability at least $\frac{9}{10}$, at which point a union bound concludes the proof. ∎

Acknowledgment

R.G. and E.L. thank Microsoft Research for its hospitality during the time some of this work was carried out. E.L. was supported in part by NSF grant DMS-1513403.