Sherali--Adams Strikes Back

TL;DR

This paper demonstrates that a certain level of the Sherali--Adams linear programming hierarchy can effectively certify bounds on Max-Cut in graphs with specific spectral properties, challenging prior beliefs about LP hierarchies' limitations.

Contribution

It shows that Sherali--Adams hierarchy can efficiently refute Max-Cut and random CSP instances in graphs with bounded eigenvalues, outperforming previous spectral and SDP methods.

Findings

Sherali--Adams certifies Max-Cut at most 50.1% in certain graphs.

Constant-round Sherali--Adams refutes random Boolean k-CSPs with many constraints.

Results contrast with the belief that LP hierarchies perform poorly for Max-Cut.

Abstract

Let $G$ be any $n$-vertex graph whose random walk matrix has its nontrivial eigenvalues bounded in magnitude by $1/\sqrt{\Delta}$ (for example, a random graph $G$ of average degree~$\Theta(\Delta)$ typically has this property). We show that the $\exp\Big(c \frac{\log n}{\log \Delta}\Big)$-round Sherali--Adams linear programming hierarchy certifies that the maximum cut in such a~$G$ is at most $50.1\%$ (in fact, at most $\tfrac12 + 2^{-\Omega(c)}$). For example, in random graphs with $n^{1.01}$ edges, $O(1)$ rounds suffice; in random graphs with $n \cdot \text{polylog}(n)$ edges, $n^{O(1/\log \log n)} = n^{o(1)}$ rounds suffice. Our results stand in contrast to the conventional beliefs that linear programming hierarchies perform poorly for \maxcut and other CSPs, and that eigenvalue/SDP methods are needed for effective refutation. Indeed, our results imply that constant-round Sherali--Adams can strongly refute random Boolean $k$-CSP instances with $n^{\lceil k/2 \rceil + \delta}$ constraints; previously this had only been done with spectral algorithms or the SOS SDP hierarchy.

Equations

$$-1\le {\mathsf{Y}}_{uv}\le 1$$

$$\text{scmax-cut}(G)\le \max \frac{1}{2}-\frac{1}{2}\cdot \frac{1}{|E|}\sum _{uv\in E}{\mathsf{Y}}_{uv}\text{s.t.}$$

$$-{\mathsf{Y}}_{uv}+{\mathsf{Y}}_{vw}+{\mathsf{Y}}_{wu}\le 1$$

$$\text{scmax-cut}(G)$$

$$\text{scmax-cut}(G)\le \underset{\begin{array}{c}U\text{ diagonal}\\ \mathrm{tr}(U)=0\end{array}}{\min }\frac{|V|}{4|E|}{\lambda }_{\text{max}}(D-A+U)$$

$$\text{scmax-cut}(G)=\frac{1}{2}-\frac{1}{2}{\mathbf{E}\text{/}}_{(u,v)\in E(G)}\widetilde{\mathbf{E}\text{/}}[x_u{x}_v],$$

$$0\le c^{\top }Xc=\Vert c{\Vert }_2^2+\sum _{u\in U}2c_r{c}_u\cdot \widetilde{\mathbf{E}\text{/}}[x_r{x}_u]=(1+{\alpha }^{2}k)+2\alpha k{\mathbf{E}\text{/}}_{(u,v)\in E(S)}\widetilde{\mathbf{E}\text{/}}[x_u{x}_v].$$

$$⟨f,Kg{⟩}_{\pi }=⟨Kf,g{⟩}_{\pi }={\mathbf{E}\text{/}}_{(\mathbf{u},\mathbf{v})\sim E}[f(\mathbf{u})g(\mathbf{v})].$$

$$\mathbf{E}\text{/}[f({\mathbf{u}}_0)g({\mathbf{u}}_d)]=⟨f,K^{d}g{⟩}_{\pi }.$$

$$⟨f,\overline{K}g{⟩}_{\pi }=⟨\overline{K}f,g{⟩}_{\pi }={\mathbf{E}\text{/}}_{(\mathbf{u},\mathbf{v})\sim E}[{\xi }_{\mathbf{u}\mathbf{v}}f(\mathbf{u})g(\mathbf{v})].$$

$${\mathbf{E}\text{/}}_{(\mathbf{u},\mathbf{v})\sim E}\left[\frac{1}{2}+\frac{1}{2}{\xi }_{\mathbf{u}\mathbf{v}}f(\mathbf{u})f(\mathbf{v})\right]=\frac{1}{2}+\frac{1}{2}⟨f,\overline{K}f{⟩}_{\pi }.$$

$$\mathbf{E}\text{/}[{\mathbf{\sigma }}_{0}f({\mathbf{u}}_0){\mathbf{\sigma }}_{d}g({\mathbf{u}}_d)]=⟨f,{\overline{K}}^{d}g{⟩}_{\pi }.$$

$${\mathsf{x}}^{\top }P\mathsf{x}\ge 0,$$

$$⟨\Psi ,A^{(0)}⟩$$

$$⟨\Psi ,A^{(1)}⟩$$

$$⟨\Psi ,A^{(d)}⟩$$

$$⟨\Psi ,A^{(2\ell )}⟩$$

$$3/2\le ⟨\Psi ,A^{(0)}⟩\le 2,⟨\Psi ,A^{(1)}⟩=k^{1/2\ell },⟨\Psi ,A^{(d)}⟩=0\text{ for 1<d<2ℓ},⟨\Psi ,A^{(2\ell )}⟩=\frac{1}{2}(k-1).$$

$${\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }\sum _{v\in V}{M}_{\mathbf{u}v}{\mathsf{X}}_{\mathbf{u}}{\mathsf{X}}_v\le {\pi }_{\ast }^{-1/2}\Vert M{\Vert }_2{\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }{\mathsf{X}}_{\mathbf{u}}^2.$$

$$M_{uv}{\mathsf{X}}_u{\mathsf{X}}_v\le \frac{M_{uv}^2}{2\gamma \pi (v)}{\mathsf{X}}_u^2+\frac{\gamma \pi (v)}{2}{\mathsf{X}}_v^2.$$

$${\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }\sum _{v\in V}{M}_{\mathbf{u}v}{\mathsf{X}}_{\mathbf{u}}{\mathsf{X}}_v\le {\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }{\mathsf{X}}_{\mathbf{u}}^2\sum _{v\in V}\frac{M_{\mathbf{u}v}^2}{2\gamma \pi (v)}+\frac{\gamma }{2}{\mathbf{E}\text{/}}_{\mathbf{v}\sim \pi }{\mathsf{X}}_v^2.$$

$$\sum _{v\in V}\frac{M_{\mathbf{u}v}^2}{2\gamma \pi (v)}\le \frac{\gamma }{2}$$

$$\beta =\frac{k{\pi }_{\ast }^{-1/2}}{2k^{1/2\ell }}\rho (\overline{K}{)}^{2\ell }+\frac{2}{k^{1/2\ell }}.$$

$$-\beta ⟨\mathsf{X},\mathsf{X}{⟩}_{\pi }\le ⟨\mathsf{X},\overline{K}\mathsf{X}{⟩}_{\pi }\le \beta ⟨\mathsf{X},\mathsf{X}{⟩}_{\pi }.$$

$$⟨\Psi ,\overline{\mathsf{x}}{\overline{\mathsf{x}}}^{\top }⟩={\overline{\mathsf{x}}}^{\top }\Psi \overline{\mathsf{x}}\ge 0.$$

$$⟨\Psi ,\mathsf{Y}⟩\ge 0,\text{ where }\mathsf{Y}=\mathbf{E}\text{/}[\overline{\mathsf{x}}{\overline{\mathsf{x}}}^{\top }].$$

$${\mathsf{Y}}_{ij}=\mathbf{E}\text{/}[\mathbf{\sigma }(i){\mathsf{X}}_{\mathbf{\varphi }(i)}\mathbf{\sigma }(j){\mathsf{X}}_{\mathbf{\varphi }(j)}]=⟨\mathsf{X},{\overline{K}}^{{\text{dist}}_S(i,j)}\mathsf{X}{⟩}_{\pi }.$$

$$\mathsf{Y}=\sum _{d=0}^{2\ell }⟨\mathsf{X},{\overline{K}}^d\mathsf{X}{⟩}_{\pi }{A}^{(d)},$$

$$0\le \sum _{d=0}^{2\ell }⟨\Psi ,A^{(d)}⟩⟨\mathsf{X},{\overline{K}}^d\mathsf{X}{⟩}_{\pi }=c_0⟨\mathsf{X},\mathsf{X}{⟩}_{\pi }+k^{1/2\ell }⟨\mathsf{X},\overline{K}\mathsf{X}{⟩}_{\pi }+\frac{1}{2}(k-1)⟨\mathsf{X},{\overline{K}}^{2\ell }\mathsf{X}{⟩}_{\pi },$$

$$⟨\mathsf{X},{\overline{K}}^{2\ell }\mathsf{X}{⟩}_{\pi }={\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }\sum _{v\in V}({\overline{K}}^{2\ell }{)}_{\mathbf{u}v}{\mathsf{X}}_{\mathbf{u}}{\mathsf{X}}_v.$$

$${\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }\sum _{v\in V}({\overline{K}}^{2\ell }{)}_{\mathbf{u}v}{\mathsf{X}}_{\mathbf{u}}{\mathsf{X}}_v\le {\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }\sum _{v\in V}\left|({\overline{K}}^{2\ell }{)}_{\mathbf{u}v}\right|\le \sqrt{|V|}{\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }\Vert {\overline{K}}_{\mathbf{u},\cdot }^{2\ell }{\Vert }_2\le \sqrt{|V|}\rho ({\overline{K}}^{2\ell })=\sqrt{|V|}\rho (\overline{K}{)}^{2\ell },$$

$${\mathbf{E}\text{/}}_{\mathbf{u}\sim \pi }\sum _{v\in V}({\overline{K}}^{2\ell }{)}_{\mathbf{u}v}{\mathsf{X}}_{\mathbf{u}}{\mathsf{X}}_v\le {\pi }_{\ast }^{-1/2}\rho (\overline{K}{)}^{2\ell }⟨\mathsf{X},\mathsf{X}{⟩}_{\pi }.$$

Full text

Our results stand in contrast to the conventional beliefs that linear programming hierarchies perform poorly for max-cut and other CSPs, and that eigenvalue/SDP methods are needed for effective refutation. Indeed, our results imply that constant-round Sherali–Adams can strongly refute random Boolean $k$-CSP instances with $n^{\lceil k/2\rceil+\delta}$ constraints; previously this had only been done with spectral algorithms or the SOS SDP hierarchy.

1 Introduction

Linear programming (LP) is a fundamental algorithmic primitive, and is the method of choice for a huge number of optimization and approximation problems. Still, there are some very basic tasks where it performs poorly. A classic example is the simplest of all constraint satisfaction problems (CSPs), the max-cut problem: Given a graph $G=(V,E)$, partition $V$ into two parts so as to maximize the fraction of ‘cut’ (crossing) edges. The standard LP relaxation for this problem [BM86, Pol92] involves optimizing over the metric polytope. Using “$\pm 1$ notation”, we have a variable $\mathsf{Y}_{uv}$ for each pair of vertices $\{u,v\}$ (with $\mathsf{Y}_{uv}$ supposed to be $-1$ if the edge is cut, $+1$ otherwise); the LP is:

[TABLE]

While this LP gives the optimal bound for some graphs (precisely, all graphs not contractible to $K_{5}$ [BM86]), it can give a very poor bound in general. Indeed, although there are graphs with maximum cut arbitrarily close to $1/2$ (e.g., $K_{n}$), the above LP bound is at least $2/3$ for every graph, since $\mathsf{Y}_{uv}\equiv-1/3$ is always a valid solution. Worse, there are graphs $G$ with $\text{{\sc max-cut}}(G)$ arbitrarily close to $1/2$ but with LP value arbitrarily close to $1$ — i.e., graphs where the integrality ratio is $2-o(1)$. For example, this is true [PT94] of an Erdős–Rényi $\mathcal{G}(n,\Delta/n)$ random graph with high probability (whp) when $\Delta=\Delta(n)$ satisfies $\omega(1)<\Delta<n^{o(1)}$.

There have been two main strategies employed for overcoming this deficiency: strengthened LPs, and eigenvalue methods.

Strengthened LPs.

One way to try to improve the performance of LPs on max-cut is to add more valid inequalities to the LP relaxation, beyond just the “triangle inequalities”. Innumerable valid inequalities have been considered: $(2k+1)$-gonal, hypermetric, negative type, gap, clique-web, suspended tree, as well as inequalities from the Lovász–Schrijver hierarchy; see Deza and Laurent [DL97, Ch. 28–30] for a review.

It is now known that the most principled and general form of this strategy is the Sherali–Adams LP hierarchy [SA90], reviewed in Section 2.4. At a high level, the Sherali–Adams LP hierarchy gives a standardized way to tighten LP relaxations of Boolean integer programs, by adding variables and constraints. The number of new variables/constraints is parameterized by a positive integer $R$, called the number of “rounds”. Given a Boolean optimization problem with $n$ variables, the $R$-round Sherali–Adams LP has variables and constraints corresponding to monomials of degree up to $R$, and thus has size $O(n)^{R}$. A remarkable recent line of work [CLRS16, KMR17] has shown that for any CSP (such as max-cut), the $R$-round Sherali–Adams LP relaxation achieves essentially the tightest integrality ratio among all LPs of its size. Nevertheless, even this most powerful of LPs arguably struggles to certify good bounds for max-cut. In a line of work [dlVM07, STT07] concluding in a result of Charikar–Makarychev–Makarychev [CMM09], it was demonstrated that for any constant $\epsilon>0$, there are graphs (random $\Delta$-regular ones, $\Delta=O(1)$) for which the $n^{\Omega(1)}$-round Sherali–Adams LP has a max-cut integrality gap of $2-\epsilon$. As a consequence, every max-cut LP relaxation of size up to $2^{n^{\Omega(1)}}$ has such an integrality gap.

Eigenvalue and SDP methods.

But for max-cut, there is a simple, non-LP, algorithm that works very well to certify that random graphs have maximum cut close to $1/2$: the eigenvalue bounds. There are two slight variants here (that coincide in the case of regular graphs): Given graph $G=(V,E)$ with adjacency matrix $A$ and diagonal degree matrix $D$, the eigenvalue bounds are

[TABLE]

Here $D-A$ and $D^{-1}A$ are the Laplacian matrix and the random walk matrix, respectively. The use of eigenvalues to bound various cut values in graphs (problems like max-cut, min-bisection, 2-xor, expansion, etc.) has a long history dating back to Fieldler and Donath–Hoffman [Fie73, DH03] among others (Equation 1 is specifically from Mohar–Poljak [MP93]). It was recognized early on that eigenvalue methods work particularly well for solving planted-random instances (e.g., of 2-xor [Hås84] and min-bisection [Bop87]) and for certifying max-cut values near $1/2$ for truly random instances. Indeed, as soon as one knows (as we now do [TY16, FO05]) that $D^{-1}A$ has all nontrivial eigenvalues bounded in magnitude by $O(1/\sqrt{\Delta})$ (whp) for a random $\Delta$-regular graph (or an Erdős–Rényi $\mathcal{G}(n,\Delta/n)$ graph with $\Delta\gtrsim\log n$), the eigenvalue bound Equation 2 certifies that $\text{{\sc max-cut}}(G)\leq 1/2+O(1/\sqrt{\Delta})$. This implies an integrality ratio tending to $1$; indeed, $\text{{\sc max-cut}}(G)=1/2+\Theta(1/\sqrt{\Delta})$ in such random graphs (whp).

Furthermore, if one extends the eigenvalue bound Equation 1 above to

[TABLE]

(as suggested by Delorme and Poljak [DP93], following [DH03, Bop87]), one obtains the polynomial-time computable semidefinite programming (SDP) bound. Goemans and Williamson [GW95] showed this bound has integrality ratio less than $1.14\approx 1/.88$ for worst-case $G$, and it was subsequently shown [Zwi99, FL01, CW04] that the SDP bound is $1/2+o(1)$ whenever $\text{{\sc max-cut}}(G)\leq 1/2+o(1)$.

LPs cannot compete with eigenvalues/SDPs?

This seemingly striking separation between the performance of LPs and SDPs in the context of random max-cut instances is now taken as a matter of course. To quote, e.g., [Tre09],

[E]xcept for semidefinite programming, we know of no technique that can provide, for every graph of max cut optimum $\leq.501$, a certificate that its optimum is $\leq.99$. Indeed, the results of [dlVM07, STT07][[CMM09]] show that large classes of Linear Programming relaxations of max cut are unable to distinguish such instances.

Specifically, the last statement here is true for $\Delta$-regular random graphs when $\Delta$ is a certain large constant. The conventional wisdom is that for such graphs, linear programs cannot compete with semidefinite programs, and cannot certify even the eigenvalue bound.

Our main result challenges this conception.

1.1 Our results

We show that whenever the eigenvalue bound Equation 2 certifies the bound ${\text{{\sc max-cut}}}(G)\leq 1/2+o(1)$, then $n^{o(1)}$-round Sherali–Adams can certify this as well.111Actually, there is a slight mismatch between our result and Equation 2: in Theorem 1.1 we need the maximum eigenvalue in magnitude to be small; i.e., we need $\lambda_{\text{min}}(-D^{-1}A)$ to be not too negative. This may well just be an artifact of our proof.

Theorem 1.1** (Simplified version of Theorem 3.6 and Corollary 3.7).**

Let $G$ be a simple $n$-vertex graph and assume that $|\lambda|<\rho$ for all eigenvalues $\lambda$ of $G$’s random walk matrix $D^{-1}A$ (excluding the trivial eigenvalue of $1$). Then for any $1\leq c\leq\Omega(\log(1/\rho))$, Sherali–Adams with $n^{O(c/\log(1/\rho))}$ rounds certifies that ${\text{{\sc max-cut}}}(G)\leq 1/2+2^{-c}$.

For example, if $G$’s random walk matrix has its nontrivial eigenvalues bounded in magnitude by $n^{-.001}$, as is the case (whp) for random graphs with about $n^{1.002}$ edges, then Sherali–Adams can certify ${\text{{\sc max-cut}}}(G)\leq 50.1\%$ with constantly many rounds. We find this result surprising, and in defiance of the common belief that polynomial-sized LPs cannot take advantage of spectral properties of the underlying graph.222We should emphasize that it is not concomitant $n^{O(1)}$ running time that is surprising, since the eigenvalue bound itself already achieves that.

(As an aside, in Proposition 4.2, we show that the $R$-round Sherali–Adams relaxation for max-cut has value at least $1/2+\Omega(1/R)$ for every graph $G$. This at least demonstrates that some dependence of the refutation strength on the number of Sherali–Adams rounds is always necessary.)

One might ask whether Theorem 1.1 even requires the assumption of small eigenvalues. That is, perhaps $n^{o(1)}$-round Sherali–Adams can certify $\text{{\sc max-cut}}\leq 1/2+o(1)$ whenever this is true. As far as we know, this may be possible; after all, the basic SDP relaxation has this property [Zwi99, FL01, CW04]. On the other hand, the eigenvalue bound itself does not have this property; there exist graphs with large (nontrivial) eigenvalues even though the maximum cut is close to $1/2$.333Consider, for example, a graph given by the union of a $\Delta$-regular random graph on $n$ vertices and a $\Delta$-regular bipartite graph on $\sqrt{n}$ vertices. This will have max-cut value close to $1/2$, but will also have large negative eigenvalues coming from the bipartite component.

1.1.1 Subexponential-sized LPs for max-cut in sparse random graphs

One setting in which the spectral radius $\rho$ is understood concretely is in random regular graphs. Building upon [FKS89, BFSU99, CGJ18], the following was recently shown:

Theorem** ([TY16]).**

There is a fixed constant $C$ such that for all $3\leq\Delta\leq n/2$ with $\Delta n$ even, it holds that a uniformly random $n$-vertex $\Delta$-regular simple graph $\bm{G}$ satisfies the following with high probability: all eigenvalues of $\bm{G}$’s normalized adjacency matrix, other than $1$, are at most $C/\sqrt{\Delta}$ in magnitude.

Combining the above with Theorem 1.1, we have the following consequence for max-cut on random regular graphs:

Corollary 1.2**.**

Let $n$, $3\leq\Delta\leq n/2$, and $1\leq c\leq\Omega(\log\Delta)$ be positive integers. Then if $G$ is a random $\Delta$-regular $n$-vertex graph, with high probability $n^{O(c/\log\Delta)}$-round Sherali–Adams can certify that ${\text{{\sc max-cut}}}(G)\leq\frac{1}{2}+2^{-c}$.

For example, if $\Delta\geq C\cdot 10^{6}$ (for $C$ the constant in the bound on $\lambda(G)$), then $n^{1/3}$-rounds of Sherali–Adams can certify $\text{{\sc max-cut}}(G)\leq.51$. This result serves as a partial converse to [CMM09]:

Theorem**.**

([CMM09, Theorem 5.3]) For every fixed integer $\Delta\geq 3$, with high probability over the choice of an $n$-vertex $\Delta$-regular random graph $G$,444In [CMM09], the graph is actually a pruned random graph, in which $o(n)$ edges are removed; this does not affect compatibility with our results, as the LP value is Lipschitz and so the pruning changes the LP value by $o(1)$. the $n^{\Theta(1/f(\Delta))}$-round Sherali–Adams relaxation for max-cut has value at least ${\text{{\sc max-cut}}}(G)\geq 0.99$, where $f(\Delta)$ is a function that grows with $\Delta$.555Though $f(\Delta)$ is not specified in [CMM09] (and in their proof $f(\Delta)$ is dictated by a combination of lemmas in prior works) it appears we can take $f(\Delta)=\log\Delta$.

While [CMM09] show that $\Delta$-regular random graphs require Sherali–Adams (and by [KMR17] any LP) relaxations of at least subexponential size, our result implies that subexponential LPs are sufficient. Further, though the function $f(\Delta)$ is not specified in [CMM09], by tracing back through citations (e.g. [ABLT06, ALN*+*12, CMM10]) to extract a dependence, it appears we may take $f(\Delta)=\log\Delta$. So our upper bound is tight as a function of $\Delta$, up to constant factors.

Prior to our result, it was unclear whether even $(n/\textnormal{polylog}n)$-round Sherali–Adams could certify that the max-cut value was $<0.9$ for sparse random regular graphs. Indeed, it was equally if not more conceivable that Charikar et al.’s result was not tight, and could be extended to $\tilde{\Omega}(n)$-rounds. In light of our result, we are left to wonder whether there are instances of max-cut which have truly exponential extension complexity.

1.1.2 Refuting Random CSPs with linear programs

With minor modifications, our argument extends as well to 2-xor. Then following the framework in [AOW15], we have the following consequence for certifying bounds on the value of random $k$-CSPs:

Theorem 1.3** (Simplified version of Theorem 5.2).**

Suppose that $P:\{\pm 1\}^{k}\to\{0,1\}$ is a $k$-ary Boolean predicate, and that $\delta,\epsilon>0$. Let $\mathop{\bf E\/}[P]$ be the probability that a random $x\in\{\pm 1\}^{k}$ satisfies $P$. Then for a random instance $\mathcal{I}$ of $P$ on $n$ variables with $m\geq n^{\lceil k/2\rceil+\delta}$ expected clauses, with high probability Sherali–Adams can certify that $\mathrm{OBJ}_{\mathcal{I}}(x)\leq\mathop{\bf E\/}[P]+\epsilon$ using $R=O_{\epsilon,\delta,k}(1)$ rounds.

This almost matches the comparable performance of sum-of-squares and spectral algorithms [AOW15], which are known to require $m\geq n^{k/2}$ clauses to certify comparable bounds in polynomial time [Gri01, Sch08, KMOW17].666 The expert may notice that we require the number of clauses $m\gg n^{\lceil k/2\rceil}$, whereas the best sum-of-squares and spectral algorithms require only $m\gg n^{k/2}$. This is because we do not know how to relate the Sherali–Adams value of the objective function to its square (local versions of the Cauchy-Schwarz argument result in a loss). Such a relation would allow us to apply our techniques immediately to prove that Sherali Adams matches the SOS and spectral performance for odd as well as even $k$. Prior to our work it was known that Sherali–Adams admits weak refutations (i.e. a certificate that $\mathrm{OBJ}\leq 1-o(1)$) when $m\geq n^{k/2}$, but it was conceivable (and even conjectured) that $O(1)$-rounds could not certify $\mathrm{OBJ}\leq 1-\delta$ for constant $\delta$ at $m=o(n^{k})$.

The result above also extends to $t$-wise independent predicates as in [AOW15] (see Section 5). Also, one may extract the dependence on the parameters $\epsilon,\delta$ to give nontrivial results when these parameters depend on $n$.777Though for 2-xor and max-cut we have done this explicitly, for higher-arity random CSPs we have left this for the interested reader.

1.2 Prior work

It is a folklore result that in random graphs with average degree $n^{\delta}$, $3$-round Sherali–Adams (SA) certifies a max-cut value of at most $\max(1-\Omega(\delta),\frac{2}{3})$ (observed for the special case of $\delta>\frac{1}{2}$ in [BHHS11, PT94]); this is simply because of concentration phenomena, since most edges participate in roughly the same number of odd cycles of length $O(\frac{1}{\delta})\geq 3$, after which one can apply the triangle inequality. However this observation does not allow one to take the refutation strength independent of the average degree.

There are some prior works examining the performance of Sherali–Adams hierarchies on random CSPs. The work of de la Vega and Mathieu [dlVM07] shows that in dense graphs, with average degree $\Omega(n)$, $O(1)$-rounds of Sherali–Adams certifies tight bounds on max-cut. This result heavily leverages the density of the graph, and is consistent with the fact that dense max-cut is known to admit a PTAS [FK96]. However, their work does not give $O(1)$-round approximations better than $2-o(1)$ for graphs with $o(n^{2})$ edges.

Another relevant line of work is a series of LP hierarchy lower bounds (both for Sherali–Adams and for the weaker Lovász-Schrijver hierarchy) for problems such as max-cut, vertex cover, and sparsest cut, including [AAT11, ABLT06, dlVM07, STT07], and culminating in the already mentioned result of Charikar, Makarychev and Makarychev; in [CMM09], they give subexponential lower bounds on the number of rounds of Sherali–Adams required to strongly refute max-cut in random regular graphs. Initially, one might expect that this result could be strengthened to prove that sparse random graphs require almost-exponential-sized LPs to refute max-cut; our result demonstrates instead that [CMM09] is almost tight.

We also mention the technique of global correlation rounding in the sum-of-squares hierarchy, which was used to give subexponential time algorithms for unique games [BRS11] and polynomial-time approximations to max-bisection [RT12]. One philosophical similarity between these algorithms and ours is that both relate local properties (correlation among edges) to global properties (correlation of uniformly random pairs). But [BRS11, RT12] use the fact that the relaxation is an SDP (whereas our result is interesting because it is in the LP-only setting), and the “conditioning” steps that drive their algorithm are a fundamentally different approach.

There are many prior works concerned with certifying bounds on random CSPs, and we survey only some of them here, referring the interested reader to the discussion in [AOW15]. The sequence of works [Gri01, Sch08, KMOW17] establishes sum-of-squares lower bounds for refuting any random constraint satisfaction problem, and these results are tight via the SOS algorithms of [AOW15, RRS17]. The upshot is that for $k$-sat and $k$-xor,888This is more generally true for any CSP that supports a $k$-wise independent distribution over satisfying assignments. $\omega(1)$ rounds of SOS are necessary to strongly refute an instance with $m=o(n^{k/2})$ clauses, and $O(1)$ rounds of SOS suffice when $m=\tilde{\Omega}(n^{k/2})$. Because SOS is a tighter relaxation than Sherali–Adams, the lower bounds [Gri01, Sch08, KMOW17] apply; our work can be seen to demonstrate that SA does not lag far behind SOS, strongly refuting with $O(1)$ rounds as soon as $m=\Omega(n^{\lceil k/2\rceil+\delta})$ for any $\delta>0$.

In a way, our result is part of a trend in anti-separation results for SDPs and simpler methods for pseudorandom and structured instances. For example, we have for planted clique that the SOS hierarchy performs no better than the Lovász-Schrijver+ hierarchy [FK03, BHK*+*16], and also no better than a more primitive class of estimation methods based on local statistics (see e.g. [RSS18] for a discussion). Similar results hold for problems relating to estimating the norms of random tensors [HKP*+*17]. Further, in [HKP*+*17] an equivalence is shown between SOS and spectral algorithms for a large class of average-case problems. Our result shows that for random CSPs, the guarantees of linear programs are surprisingly not far from the guarantees of SOS.

Finally, we mention related works in extended formulations. The sequence of works [CLRS16, KMR17] show that SA lower bounds for CSPs imply lower bounds for any LP relaxation; the stronger (and later) statement is due to [KMR17], who show that subexponential-round integrality gaps for CSPs in the Sherali–Adams hierarchy imply subexponential-size lower bounds for any LP. These works are then applied in conjunction with [Gri01, Sch08, CMM09] to give subexponential lower bounds against CSPs for any LP; our results give an upper limit to the mileage one can get from these lower bounds in the case of max-cut, as we show that the specific construction of [CMM09] cannot be strengthened much further.

1.3 Techniques

Our primary insight is that while Sherali–Adams is unable to reason about spectral properties globally, it does enforce that every set of $R$ variables behave locally according to the marginals of a valid distribution, which induces local spectral constraints on every subset of up to $R$ variables.

At first, it is unclear how one harnesses such local spectral constraints. But now, suppose that we are in a graph with a small spectral radius. This implies that random walks mix rapidly, in say $t$ steps, to a close-to-uniform distribution. Because a typical pair of vertices at distance $t$ is distributed roughly as a uniformly random pair of vertices, any subset of $R$ vertices which contains a path of length $t$ already allows us to relate global and local graph properties.

To see why this helps, we take for a moment the “pseudoexpectation” view, in which we think of the $R$-round Sherali–Adams as providing a proxy for the degree-$R$ moments of a distribution over max-cut solutions $x\in\{\pm 1\}^{n}$, with max-cut value

[TABLE]

where $\widetilde{\mathop{\bf E\/}}[x_{u}x_{v}]$ is the “pseudo-correlation” of variables $x_{u},x_{v}$. Because there is no globally consistent assignment, the pseudo-correlation $\widetilde{\mathop{\bf E\/}}[x_{u}x_{v}]$ for vertices $u,v$ sampled uniformly at random will be close to [math].999This is implicit in our proof, but intuitively it should be true because e.g. $u,v$ should be connected by equally many even- and odd-length paths. But in any fixed subgraph of size $\Omega(t)$, enforcing $\widetilde{\mathop{\bf E\/}}[x_{u}x_{v}]\approx 0$ for pairs $u,v$ at distance $t$ has consequences, and limits the magnitude of correlation between pairs of adjacent vertices as well. In particular, because the pseudo-second moment matrix $\widetilde{\mathop{\bf E\/}}[x_{S}x_{S}^{\top}]$ for $x_{S}$ the restriction of $x$ to a set $S$ of up to $R$ vertices must be PSD, forcing some entries to [math] gives a constraint on the magnitude of edge correlations.

For example, suppose for a moment that we are in a graph $G$ with $t=2$, and that $S$ is a star graph in $G$, given by one “root” vertex $r$ with $k\leq R-1$ children $U=\{u_{1},\ldots,u_{k}\}$, and call $X=\widetilde{\mathop{\bf E\/}}[x_{S}x_{S}^{\top}]\succeq 0$. Notice that pairs of distinct children $u_{i},u_{j}$ are at distance $t=2$ in $S$. If we then require $\widetilde{\mathop{\bf E\/}}[x_{u_{i}}x_{u_{j}}]=0$ for every $u_{i}\neq u_{j}$, the only nonzero entries of $X$ are the diagonals (which are all $\widetilde{\mathop{\bf E\/}}[x_{i}^{2}]=1$), and the entries corresponding to edges from the root to its children, $(r,u_{i})$, which are $\widetilde{\mathop{\bf E\/}}[x_{r}x_{u_{i}}]$. Now defining the vector $c\in\mathbbm{R}^{S}$ with a $1$ at the root $r$, $c_{r}=1$ and $\alpha$ on each child $u\in U$, $c_{u}=\alpha$, we have from the PSDness of $X$ that

[TABLE]

Choosing $\alpha=k^{-1/2}$, this implies that within $S$, the average edge correlation is lower bounded by $\mathop{\bf E\/}_{(u,v)\in E[S]}\widetilde{\mathop{\bf E\/}}[x_{u}x_{v}]\geq-k^{-1/2}$. Of course, for a given star $S$ we cannot know that $\widetilde{\mathop{\bf E\/}}[x_{u_{i}}x_{u_{j}}]=0$, but if we take a well-chosen weighted average over all stars, this will (approximately) hold on average.

Our strategy is to take a carefully-chosen average over specific subgraphs $S$ of $G$ with $|S|=\Omega(t)$. By our choice of distribution and subgraph, the fact that the subgraphs locally have PSD pseudocorrelation matrices has consequences for the global average pseudocorrelation across edges, which in turn gives a bound on the objective value eq. 3. This allows us to show that Sherali–Adams certifies much better bounds than we previously thought possible, by aggregating local spectral information across many small subgraphs.

Organization

We begin with technical preliminaries in Section 2. In Section 3 we prove our main result. Section 4 establishes a mild lower bound for arbitrary graphs. Finally, Section 5 applies Theorem 1.1 to the refutation of arbitrary Boolean CSPs.

2 Setup and preliminaries

We begin by recalling preliminaries and introducing definitions that we will rely upon later.

2.1 Random walks on undirected graphs

Here, we recall some properties of random walks in undirected graphs that will be of use to us.

Definition 2.1**.**

Let $G=(V,E)$ be an undirected finite graph, with parallel edges and self-loops allowed101010Self-loops count as “half an edge”, and contribute $1$ to a vertex’s degree., and with no isolated vertices. The standard random walk on $G$ is the Markov chain on $V$ in which at each step one follows a uniformly random edge out of the current vertex. For $u\in V$, we use the notation $\bm{v}\sim u$ to denote that $\bm{v}$ is the result of taking one random step from $u$.

Definition 2.2**.**

We write $K$ for the transition operator of the standard random walk on $G$. That is, $K$ is obtained from the adjacency matrix of $G$ by normalizing the $u$th row by a factor of $1/\deg(u)$.

Definition 2.3**.**

We write $\pi$ for the probability distribution on $V$ defined by $\pi(v)=\frac{\deg(v)}{2|E|}$. As is well known, this is an invariant distribution for the standard random walk on $G$, and this Markov chain is reversible with respect to $\pi$. For $\bm{u}\sim\pi$ and $\bm{v}\sim\bm{u}$, the distribution of $(\bm{u},\bm{v})$ is that of a uniformly random (directed) edge from $E$. We will also use the notation $\pi_{*}=\min_{v\in V}\{\pi(v)\}$.

Definition 2.4**.**

For $f,g:V\to\mathbbm{R}$ we use the notation $\langle f,g\rangle_{\pi}$ for $\mathop{\bf E\/}_{\bm{u}\sim\pi}[f(\bm{u})g(\bm{u})]$. This is an inner product on the vector space $\mathbbm{R}^{V}$; in case $G$ is regular and hence $\pi$ is the uniform distribution, it is the usual inner product scaled by a factor of $1/|V|$. It holds that

[TABLE]

Definition 2.5**.**

A stationary $d$-step walk is defined to be a sequence $(\bm{u}_{0},\bm{u}_{1},\dots,\bm{u}_{d})$ formed by choosing an initial vertex $\bm{u}_{0}\sim\pi$, and then taking a standard random walk, with $\bm{u}_{t}\sim\bm{u}_{t-1}$. Generalizing Equation 4, it holds in this case that

[TABLE]

2.2 Tree-indexed random walks

To prove our main theorem we define a class of homomorphisms we call tree-indexed random walks.

Definition 2.6**.**

Suppose we have a finite undirected tree with vertex set $T$. A stationary $T$-indexed random walk in $G$ is a random homomorphism $\bm{\phi}:T\to V$ defined as follows: First, root the tree at an arbitrary vertex $i_{0}\in T$. Next, define $\bm{\phi}(i_{0})\sim\pi$. Then, independently for each “child” $j$ of $i_{0}$ in the tree, define $\bm{\phi}(j)\sim\bm{\phi}(i_{0})$; that is, define $\bm{\phi}(j)\in V$ to be the result of taking a random walk step from $\bm{\phi}(i_{0})$. Recursively repeat this process for all children of $i_{0}$’s children, etc., until each vertex $k\in T$ has been assigned a vertex $\bm{\phi}(k)\in V$.

We note that the homomorphism $\bm{\phi}$ defining the $T$-indexed random walk need not be injective. Consequently, if $T$ is a tree with maximum degree $D$, we can still have a $T$-indexed random walk in a $d$-regular graph with $d<D$.

The following fact is simple to prove; see, e.g., [LP17].

Fact 2.7**.**

The definition of $\bm{\phi}$ does not depend on the initially selected root $i_{0}\in T$. Further, for any two vertices $i,j\in T$ at tree-distance $d$, if $i=i_{0},i_{1},\dots,i_{d}=j$ is the unique path in the tree between them, then the sequence $(\bm{\phi}(i_{0}),\bm{\phi}(i_{1}),\dots,\bm{\phi}(i_{d}))$ is distributed as a stationary $d$-step walk in $G$.

2.3 2XOR and signed random walks

The 2-xor constraint satisfaction problem is defined by instances of linear equations in $\mathbbm{F}_{2}^{n}$. For us it will be convenient to associate with these instances a graph with signed edges, and on such graphs we perform a slightly modified random walk.

Definition 2.8**.**

We assume that for each vertex pair $(u,v)$ where $G$ has edge, there is an associated sign $\xi_{uv}=\xi_{vu}\in\{\pm 1\}$.111111If $G$ has multiple $(u,v)$ edges, we think of them as all having the same sign. We arrange these signs into a symmetric matrix $\Xi=(\xi_{uv})_{uv}$. If $G$ has no $(u,v)$ edge then the entry $\Xi_{uv}$ will not matter; we can take it to be [math].

Definition 2.9**.**

We write $\overline{K}=\Xi\circ K$ for the signed transition operator. The operator $\overline{K}$ is self-adjoint with respect to $\langle\cdot,\cdot\rangle_{\pi}$, and hence has real eigenvalues. It also holds that

[TABLE]

Definition 2.10**.**

We may think of $G$ and $\Xi$ as defining a 2-xor constraint satisfaction problem (CSP), in which the task is to find a labeling $f:V\to\{\pm 1\}$ so as to maximize the fraction of edges $(u,v)\in E$ for which the constraint $f(u)f(v)=\xi_{uv}$ is satisfied. The fraction of satisfied constraints is

[TABLE]

We will typically ignore the $\frac{1}{2}$’s and think of the 2-xor CSP as maximizing the quadratic form $\langle f,\overline{K}f\rangle_{\pi}$. When all signs in the matrix $\Xi$ are $-1$, we refer to this as the max-cut CSP.

Definition 2.11**.**

We say that a signed stationary $d$-step walk is a sequence of pairs $(\bm{u}_{t},\bm{\sigma}_{t})\in\{\pm 1\}\times V$ for $0\leq t\leq d$, chosen as follows: first, we choose a stationary $d$-step walk $(\bm{u}_{0},\bm{u}_{1},\dots,\bm{u}_{d})$ in $G$; second, we choose $\bm{\sigma}_{0}\in\{\pm 1\}$ uniformly at random; finally, we define $\bm{\sigma}_{t}=\bm{\sigma}_{t-1}\xi_{\bm{\sigma}_{t-1}\bm{\sigma}_{t}}$. Generalizing Equation 5, it holds in this case that

[TABLE]

Definition 2.12**.**

We extend the notion from Definition 2.6 to that of a signed stationary $T$-indexed random walk in $G$. Together with the random homomorphism $\bm{\phi}:T\to V$, we also choose a random signing $\bm{\sigma}:T\to\{\pm 1\}$ as follows: for the root $i_{0}$, the sign $\bm{\sigma}(i_{0})\in\{\pm 1\}$ is chosen uniformly at random; then, all other signs are deterministically chosen — for each $j$ of $i_{0}$ we set $\bm{\sigma}(j)=\xi_{i_{0}j}\bm{\sigma}(i_{0})$, and in general $\bm{\sigma}(k)=\xi_{k^{\prime}k}\bm{\sigma}(k)$ where $k^{\prime}$ is the parent of $k$. Again, it is not hard to show that the definition of $(\bm{\phi},\bm{\sigma})$ does not depend on the choice of root $i_{0}$, and that for any path $i_{0},i_{1},\dots,i_{d}$ of vertices in the tree, the distribution of $(\bm{\phi}(i_{0}),\bm{\sigma}(i_{0})),(\bm{\phi}(i_{1}),\bm{\sigma}(i_{1})),\dots(\bm{\phi}(i_{d}),\bm{\sigma}(i_{d}))$ is that of a signed stationary $d$-step walk in $G$.

2.4 Proof systems

Our central object of study is the Sherali–Adams proof system, but our results also apply to a weaker proof system which we introduce below.

Definition 2.13**.**

We define the $R$-local, degree-$D$ (static) sum of squares (SOS) proof system over indeterminates $\mathsf{X}_{1},\dots,\mathsf{X}_{n}$ as follows. The “lines” of the proof are real polynomial inequalities in $\mathsf{X}=(\mathsf{X}_{1},\dots,\mathsf{X}_{n})$. The “default axioms” are any real inequalities of the form $p(\mathsf{X}_{u_{1}},\dots,\mathsf{X}_{u_{R}})^{2}\geq 0$, where $p$ is a polynomial in at most $R$ variables and of degree at most $D/2$. The “deduction rules” allow one to derive any nonnegative linear combination of previous lines. This is a sound proof system for inequalities about $n$ real numbers $\mathsf{X}_{1},\dots,\mathsf{X}_{n}$.

In addition to the default axioms, one may also sometimes include problem-specific “equalities” of the form $q(\mathsf{X})=0$. In this case, one is allowed additional axioms of the form $q(\mathsf{X})s(\mathsf{X})\gtreqless 0$ the polynomial $q(\mathsf{X})s(\mathsf{X})$ depends on at most $R$ indeterminates and has degree at most $D$.

Fact 2.14**.**

The case of $R=\infty$ (equivalently, $R=n$) corresponds to the well-known degree-$D$ SOS proof system.

Definition 2.15**.**

Suppose one includes the Boolean equalities, meaning $\mathsf{X}_{u}^{2}-1=0$ for all $1\leq i\leq n$.121212Or alternatively, $\mathsf{X}_{u}^{2}-\mathsf{X}_{u}=0$ for all $i$. In this case $D=\infty$ is equivalent to $D=R$, and the corresponding proof system is the well-known $R$-round Sherali–Adams proof system. It is well known that every inequality $p(\mathsf{X}_{u_{1}},\dots,\mathsf{X}_{u_{R}})\geq 0$ that is true for $\pm 1$-valued $\mathsf{X}_{u_{1}},\dots,\mathsf{X}_{u_{R}}$ is derivable in this system.

Fact 2.16**.**

There is a $\textnormal{poly}(n^{R},L)$-time algorithm based on Linear Programming for determining whether a given polynomial inequality $p(\mathsf{X})\geq 0$ of degree at most $R$ (and rational coefficients of total bit-complexity $L$) is derivable in the $R$-round Sherali–Adams proof system.

Fact 2.17**.**

We will often be concerned with the $R$-local, degree-$2$ SOS proof system, where all lines are quadratic inequalities. In this case, we could equivalently state that the default axioms are all those inequalities of the form

[TABLE]

where $\mathsf{x}=(\mathsf{X}_{u_{1}},\dots,\mathsf{X}_{u_{R}})$ is a length-$R$ subvector of $\mathsf{X}$, and $P$ is an $R\times R$ positive semidefinite (PSD) matrix.

Remark 2.18**.**

In fact, we will often be concerned with the $R$-round, degree-$2$ Sherali–Adams proof system. Despite the restriction to $D=2$, we only know the $\textnormal{poly}(n^{R},L)$-time algorithm for deciding derivability of a given quadratic polynomial $p(\mathsf{X})\geq 0$ (of bit-complexity $L$).

3 2XOR certifications from spider walks

In this section, we prove our main theorem: given a 2-xor or max-cut instance on a graph $G$ with small spectral radius, we will show that the $R$-local degree-$2$ SOS proof system gives nontrivial refutations with $R$ not too large.

Our strategy is as follows: we select a specific tree $T$ of size $\propto R$, and we consider the distribution over copies of $T$ in our graph given by the $T$-indexed stationary random walk. We will use this distribution to define the coefficients for a degree-$2$, $R$-local proof that bounds the objective value of the CSP. We will do this by exploiting the uniformity of the graph guaranteed by the small spectral radius, and the fact that degree-$2$ $R$-local SOS proofs can certify positivity of quadratic forms $c^{\top}\mathsf{X}|_{S}\mathsf{X}|_{S}^{\top}c$, where $X|_{S}$ is the restriction of $\mathsf{X}$ to a set $S$ of variables with $|S|\leq R$ and $c\in\mathbbm{R}^{|S|}$.

Intuitively, in the “pseudoexpectation” view, the idea of our proof is as follows. When there is no globally consistent assignment, a uniformly random pair of vertices $u,v\in V$ will have pseudocorrelation close to zero. On the other hand, if $t$-step random walks mix to a roughly uniform distribution over vertices in the graph, then pairs of vertices at distance $t$ will also have pseudocorrelation close to zero. But also, in our proof system the degree-$2$ pseudomoments of up to $R$ variables obey a positive-semidefiniteness constraint. By choosing the tree $T$ with diameter at least $t$, while also choosing $T$ to propagate the effect of the low-pseudocorrelation at the diameter to give low-pseudocorrelation on signed edges, we show that the proof system can certify that the objective value is small. Specifically, we will choose $T$ to be a spider graph:

Definition 3.1**.**

For integers $k,\ell\in\mathbbm{N}^{+}$, we define a $(k,\ell)$-spider graph to be the tree formed by gluing together $k$ paths of length $\ell$ at a common endpoint called the root. This spider has $k\ell+1$ vertices and diameter $2\ell$.

While we were not able to formally prove that the spider is the optimal choice of tree, intuitively, we want to choose a tree that maximizes the ratio of the number of pairs at maximum distance (since such pairs relate the local properties to the global structure) to the number of vertices in the tree (because we need to take our number of rounds $R$ to be at least the size of the tree). Among trees, the spider is the graph that maximizes this ratio.

Let us henceforth fix a $(k,\ell)$-spider graph, where the parameters $k$ and $\ell$ will be chosen later. We write $S$ for the vertex set of this tree (and sometimes identify $S$ with the tree itself).

Definition 3.2**.**

For $0\leq d\leq 2\ell$, we define the matrix $A^{(d)}\in\mathbbm{R}^{S\times S}$ to be the “distance-$d$” adjacency matrix of the spider; i.e., $A^{(d)}_{ij}$ is $1$ if $\textnormal{dist}_{S}(i,j)=d$ and is [math] otherwise. (We remark that $A^{(0)}$ is the identity matrix.)

The following key technical theorem establishes the existence of a matrix $\Psi$ which will allow us to define the coefficients in our $R$-local degree-$2$ SOS proof. It will be proven in Section 3.2:

Theorem 3.3**.**

For any parameter $\alpha\in R$, there is a PSD matrix $\Psi=\Psi_{\alpha}\in\mathbbm{R}^{S\times S}$ with the following properties:

[TABLE]

Here we are using the notation $\langle B,C\rangle$ for the “matrix (Frobenius) inner product” $\Tr(B^{\top}C)$.

Corollary 3.4**.**

Assuming that $k\geq 3^{\ell}$ and taking $\alpha=k^{1/2\ell}$, the PSD matrix $\Psi$ satisfies

[TABLE]

We will also use the following small technical lemma:

Lemma 3.5**.**

Let $M\in\mathbbm{R}^{V\times V}$ and recall $\pi_{*}=\min_{v\in V}\{\pi(v)\}>0$. Then the $2$-local, degree-$2$ SOS proof system can derive

[TABLE]

Proof.

The proof system can derive the following inequality for any $\gamma>0$, since the difference of the two sides is a perfect square:

[TABLE]

Thus it can derive

[TABLE]

We’ll take $\gamma=\pi_{*}^{-1/2}\|M\|_{2}$. Since we can certainly derive $a\mathsf{X}_{u}^{2}\leq b\mathsf{X}_{u}^{2}$ whenever $a\leq b$, we see that it suffices to establish

[TABLE]

for every outcome of $\bm{u}$. But this is implied by $\sum_{v}M^{2}_{\bm{u}v}\leq(\pi(v)/\pi_{*})\|M\|^{2}_{2}$ for all $v\in V$, which is indeed true. ∎

We can now prove the following main theorem:

Theorem 3.6**.**

Given parameters $k\geq 3^{\ell}$, let $R=k\ell+1$ and define

[TABLE]

Then $R$-local, degree-$2$ SOS can deduce the bound “$\rho(\overline{K})\leq\beta$”; more precisely, it can deduce the two inequalities

[TABLE]

Before proving this theorem, let us simplify the parameters. For any $\epsilon>0$, we can choose $\ell$ to be the smallest integer so that $(\frac{1}{\epsilon}\rho(\overline{K}))^{2\ell}\pi_{*}^{-1/2}\leq\epsilon$, and $k=\lceil(\frac{1}{\epsilon})^{2\ell}\rceil$. This gives the corollary:

Corollary 3.7**.**

Suppose we have a graph $G=(V,E)$ with signed transition operator $\overline{K}$ and $\pi_{*}=\min_{v\in V}\frac{\deg(v)}{2|E|}$. Given $\epsilon>\min(\pi_{*}^{-1/2},\rho(\overline{K}))$, take \ell=\Big{\lceil}\frac{1}{4}\frac{\log(\epsilon^{2}\pi_{*})}{\log(\rho(\overline{K})/\epsilon)}\Big{\rceil}, and take $k=\lceil(\frac{1}{\epsilon})^{2\ell}\rceil$. Then for $R=k\ell+1$, it holds that $R$-local degree-$2$ SOS can deduce the bound $\rho(\overline{K})\leq\frac{5}{2}\epsilon$. In particular, if we think of $G,\Xi$ as a 2-xor CSP, it holds that $R$-round Sherali–Adams can deduce the bound $\mathrm{OBJ}\leq\frac{1}{2}+\frac{5}{4}\epsilon$.

Proof.

Taking the parameters as above, and using that the constraints $\mathsf{X}_{u}^{2}=1$ imply that $R$-round Sherali–Adams can deduce that $\langle\mathsf{X},\mathsf{X}\rangle_{\pi}=1$ whenever $R\geq 2$, and that as noted in eq. 6, $\mathrm{OBJ}(\mathsf{X})=\frac{1}{2}+\frac{1}{2}\langle\mathsf{X},\overline{K}\mathsf{X}\rangle_{\pi}$, so Theorem 3.6 gives the result. ∎

Corollary 3.7 implies the 2-xor version of Theorem 1.1 since in simple graphs, $\log\frac{1}{\pi^{*}}=\Theta(\log n)$.

Proof of Theorem 3.6.

For our $(k,\ell)$-spider graph on $S$, let $(\bm{\phi},\bm{\sigma})$ be a signed stationary $S$-indexed random walk in $G$. Define $\bm{\overline{\mathsf{x}}}$ to be the $S$-indexed vector with $\bm{\overline{\mathsf{x}}}_{i}=\bm{\sigma}(i)\mathsf{X}_{\bm{\phi}(i)}$. Then letting $\Psi$ be the PSD matrix from Corollary 3.4, the $R$-local, degree-$2$ SOS proof system can derive

[TABLE]

(This is in the form of Equation 7 if we take $P=\operatorname{diag}(\bm{\sigma})\Psi\operatorname{diag}(\bm{\sigma})$.) Furthermore, the proof system can deduce this inequality in expectation; namely,

[TABLE]

Now by the discussion in Definitions 2.11 and 2.12,

[TABLE]

Thus recalling the notation $A^{(d)}$ from Definition 3.2,

[TABLE]

and hence from Equation 9 we get that $R$-local, degree-$2$ SOS can deduce

[TABLE]

for some constant $3/2\leq c_{0}\leq 2$ (here we used Corollary 3.4). Regarding the last term, we have:

[TABLE]

If we cared only about the Sherali–Adams proof system with Boolean equalities, we would simply now deduce

[TABLE]

and later combine this with $c_{0}\langle\mathsf{X},\mathsf{X}\rangle_{\pi}=c_{0}$. But proceeding more generally, we instead use Lemma 3.5 to show that our proof system can derive

[TABLE]

Putting this into Equation 13 and Equation 12 we get

[TABLE]

Repeating the derivation with $-\overline{K}$ in place of $\overline{K}$ completes the proof. ∎

3.1 Max-Cut

The following theorem is quite similar to Theorem 3.6. In it, we allow $K$ to have the large eigenvalue $1$, and only certify that it has no large-magnitude negative eigenvalue. The subsequent corollary is deduced identically to Corollary 3.7.

Theorem 3.8**.**

Given transition operator $K$ for the standard random walk on $G$, let $K^{\prime}=K-J$, where $J$ is the all-$1$’s matrix. For parameters $k\geq 3^{\ell}$, let $R=k\ell+1$ and define

[TABLE]

(Note that $\rho(K^{\prime})$ is equal to maximum-magnitude eigenvalue of $K$ when the trivial $1$ eigenvalue is excluded.) Then $2R$-local, degree-$2$ SOS can deduce the bound “$\lambda_{\textnormal{min}}(K)\geq-\beta$”; more precisely, it can deduce the inequality

[TABLE]

Corollary 3.9**.**

Suppose we have a graph $G=(V,E)$ with transition operator $K$ and centered transition operator $K^{\prime}=K-J$, and $\pi_{*}=\min_{v\in V}\frac{\deg(v)}{2|E|}$. Given $\epsilon>\min(\pi_{*}^{-1/2},\rho(K^{\prime}))$, take \ell=\Big{\lceil}\frac{1}{4}\frac{\log(\epsilon^{2}\pi_{*})}{\log(\rho(\overline{K})/\epsilon)}\Big{\rceil}, and take $k=\lceil(\frac{1}{\epsilon})^{2\ell}\rceil$. Then for $R=k\ell+1$, it holds that $2R$-local degree-$2$ SOS can deduce the bound $\rho(K^{\prime})\leq\frac{5}{2}\epsilon$. In particular, if we think of $G$ as a max-cut CSP, it holds that $R$-round Sherali–Adams can deduce the bound $\mathrm{OBJ}\leq\frac{1}{2}+\frac{5}{4}\epsilon$.

Again, Corollary 3.9 implies Theorem 1.1 since in simple graphs, $\log\frac{1}{\pi^{*}}=\Theta(\log n)$.

Proof of Theorem 3.8..

The proof is a modification of the proof of Theorem 3.6. Letting $S$ be the $(k,\ell)$-spider vertices, instead of taking a signed stationary $S$-indexed random walk in $G$, we take two independent unsigned stationary $S$-indexed random walks, $\bm{\phi}_{1}$ and $\bm{\phi}_{2}$. For $j\in\{1,2\}$, define $\bm{\mathsf{x}}_{j}$ to be the $S$-indexed vector with $i$th coordinate equal to $\mathsf{X}_{\bm{\phi}_{j}(i)}$, and write $\bm{\dot{\mathsf{x}}}$ for the concatenated vector $(\bm{\mathsf{x}}_{1},\bm{\mathsf{x}}_{2})$. Also, for $0<\theta<1$ a parameter131313This parameter is introduced to fix a small annoyance; the reader might like to imagine $\theta=1$ at first. slightly less than $1$, let $\Psi$ be the PSD matrix from Corollary 3.4, and define the PSD block-matrix

[TABLE]

Then as before, the $2R$-local, degree-$2$ SOS proof system can derive

[TABLE]

and $\bm{\mathsf{x}}$ (which will play the role of $\bm{\overline{\mathsf{x}}}$) denotes the common distribution of $\bm{\mathsf{x}}_{1}$ and $\bm{\mathsf{x}}_{2}$. Similar to Equations 10 and 11, we now have

[TABLE]

and by independence of $\bm{\mathsf{x}}_{1}$ and $\bm{\mathsf{x}}_{2}$ we have

[TABLE]

Thus applying Corollary 3.4 to Equation 14, our proof system can derive

[TABLE]

By selecting $\theta$ appropriately, we can arrange for the factor $c_{0}+k^{1/2\ell}+\tfrac{1}{2}(k-1)$ on the right to equal $\iota\cdot\tfrac{1}{2}(k-1)$. Inserting this choice into Equation 15 and then dividing through by $\iota$, we conclude that the proof system can derive

[TABLE]

cf. Equation 12. Recalling now that $K$ has the constantly-$1$ function as an eigenvector, with eigenvalue $1$, we have the identity

[TABLE]

Now the remainder of the proof is just as in Theorem 3.3, with $K-J$ in place of $\overline{K}$, except we do not have the step of repeating the derivation with $-\overline{K}$ in place of $\overline{K}$. ∎

3.2 A technical construction of coefficients on the spider

Proof of Theorem 3.3.

We are considering the $(k,\ell)$-spider graph on vertex set $S$. We write $V_{t}$ for the set of all vertices at distance $t$ from the root (so $|V_{0}|=1$ and $|V_{t}|=k$ for $1\leq t\leq\ell$). We will be considering vectors in $\mathbbm{R}^{S}$, with coordinates indexed by the vertex set $S$. For $0\leq t\leq\ell$ define the vector

[TABLE]

where $e_{i}=(0,\dots,0,1,0,\dots,0)$ is the vector with the $1$ in the $i$th position. Further define vectors

[TABLE]

with the understanding that $\mu_{\ell+1}=0$. Next, define the PSD matrix

[TABLE]

This will almost be our desired final matrix $\Psi$. Let us now compute

[TABLE]

To do this, we observe that

[TABLE]

and

[TABLE]

From this we can compute the following (with a bit of effort):

[TABLE]

Now, for a parameter $\eta>0$ to be chosen shortly, we finally define the PSD matrix

[TABLE]

We have

[TABLE]

Therefore by carefully choosing

[TABLE]

we get all of the desired inner products in the theorem statement. ∎

4 Lower Bounds

In this section, we show that degree-$R$ Sherali–Adams cannot refute a random 2-xor or max-cut instance better than $\frac{1}{2}+\Omega(\frac{1}{R})$. This is a straightforward application of the framework of Charikar, Makarychev and Makarychev [CMM09]. In that work, the authors show that if every subset of $r$ points in a metric can be locally embedded into the unit sphere, then Goemans-Williamson rounding can be used to give a $\Theta(r)$-round Sherali–Adams feasible point. The upshot is the following theorem appearing in [CMM09] (where it is stated in slightly more generality, for the $0/1$ version of the cut polytope):

Theorem 4.1** (Theorem 3.1 in [CMM09]).**

Let $(X,\rho)$ be a metric space, and assume that every $r=2R+3$ points of $(X,\rho)$ isometrically embed in the Euclidean sphere of radius $1$. Then the following point is feasible for $R$-rounds of the Sherali–Adams relaxation for the cut polytope:

[TABLE]

Proposition 4.2**.**

In any 2-xor or max-cut instance, $R$-rounds of Sherali–Adams cannot certify that

[TABLE]

Proof.

Suppose that we are given a 2-xor (equivalently, max-cut) instance on the graph $G$, so that on each edge $(i,j)\in E(G)$ we have the constraint $x_{i}x_{j}b_{ij}=1$ for some $b_{ij}\in\{\pm 1\}$. Define the metric space on $(X,\rho)$ as follows: let $X=\{x_{1},\ldots,x_{n}\}$ have a point for each vertex of $G$, and set $\rho(x_{i},x_{j})=\sqrt{2\left(1-b_{ij}\frac{1}{R}\right)}$.

We claim that any $r=2R+3$ points of $X$ embed isometrically into the Euclidean sphere of radius $1$. To see this, fix a set $S\subset X$, and define the $|S|\times|S|$ matrix $B_{S}$ so that

[TABLE]

So long as $|S|\leq r$, the matrix $M_{S}=\mathbbm{1}+B_{S}$ is diagonally dominant, and therefore positive semidefinite, so from the Cholesky decomposition of $M_{S}$ we assign to each $x_{i}\in S$ a vector $v_{i}$ so that $\|v_{i}\|_{2}=1$, and so that for every pair $x_{i},x_{j}\in S$, $\|v_{i}-v_{j}\|^{2}=2-2b_{ij}\frac{1}{r}=\rho(i,j)^{2}$.

Applying Theorem 4.1, we have that the solution

[TABLE]

is feasible. For convenience, let $f(z)=1-\frac{2}{\pi}\arccos(z)$. We use the following properties of $f$:

Claim 4.3**.**

The function $f(z)=1-\frac{2}{\pi}\arccos(z)$ exhibits the rotational symmetry $f(z)=-f(-z)$, and further $f(z)\geq\frac{2}{\pi}z$ for $z\in[0,1]$.

We give the proof of the claim (using straightforward calculus) below. Now, because $f(z)=-f(-z)$, we have that

[TABLE]

We conclude that $R=\frac{1}{2}(r-3)$ rounds of Sherali–Adams are unable to certify that $\mathrm{OBJ}<\frac{1}{2}+\frac{2}{\pi}\frac{1}{2R+3}$, as desired. ∎

Proof of 4.3.

The rotational symmetry follows from simple manipulations:

[TABLE]

For the second claim, we use that the derivative of $f(z)-\frac{2}{\pi}z$ is positive in the interval $[0,\frac{1}{2}]$:

[TABLE]

and that at $z=0$, $f(z)-\frac{2}{\pi}z=0$. ∎

5 Refutation for any Boolean CSP

In this section, we argue that $R$-round Sherali–Adams can also refute any non-trivial Boolean CSP. First, for any predicate $P:\{\pm 1\}^{k}\to\{0,1\}$ we define a parameterized distribution over the CSP with constraints from $P$:

Definition 5.1**.**

Let $P:\{\pm 1\}^{k}\to\{0,1\}$ be a predicate. Then we define a random instance of $P$ on $n$ vertices with $m$ expected clauses to be an instance sampled as follows: define $p=\frac{m}{n^{k}}$, and for each ordered multiset $S\subset[n]$ with $|S|=k$, independently with probability $p$ we sample a uniformly random string $\zeta_{S}\in\{\pm 1\}^{k}$ and add the constraint that $P(x^{S}\odot\zeta_{S})=1$, where $\odot$ denotes the entry-wise (or Hadamard) product.

This is one of several popular models, and in our case it is the most convenient to work with. By employing some manipulations, results from this model transfer readily to the others (see for example Appendix D of [AOW15] for details).

Our result is as follows:

Theorem 5.2**.**

Suppose that $P:\{\pm 1\}^{k}\to\{0,1\}$ and that $\delta,\epsilon>0$ are fixed constants. Let $\mathop{\bf E\/}[P]$ be the probability that a random $x\in\{\pm 1\}^{k}$ satisfies $P$. Then with high probability, for a random instance $\mathcal{I}$ of $P$ on $n$ variables with $m\geq n^{\lceil k/2\rceil+\delta}$ expected clauses, the $R$-round Sherali–Adams proof system can certify that $\mathrm{OBJ}_{\mathcal{I}}(x)\leq\mathop{\bf E\/}[P]+\epsilon$ when $R=O_{\epsilon,\delta,k}(1)$ rounds. More specifically, $R=k\ell\left(\frac{3\cdot 2^{k/2-1}}{\epsilon}\right)^{2\ell}+k$ for $\ell=\lceil\lceil\tfrac{k}{2}\rceil\tfrac{1}{2\delta}\rceil$.

We can also prove a more fine-grained result, to obtain strong refutation at lower clause densities when the predicate has certain properties.

Definition 5.3**.**

We say that a predicate $P:\{\pm 1\}^{k}\to\{0,1\}$ is $\eta$-far from $t$-wise supporting if every $t$-wise uniform distribution has probability mass at least $\eta$ on the set of unsatisfying assignments $P^{-1}(0)$.

Theorem 5.4**.**

Suppose that $P:\{\pm 1\}^{k}\to\{0,1\}$ is $\eta$-far from $t$-wise supporting, and that $\delta,\epsilon>0$. Then with high probability, for a random instance $\mathcal{I}$ of $P$ on $n$ variables and $m\geq n^{\lceil t/2\rceil+\delta}$ expected clauses, the $R$-round Sherali–Adams proof system can certify that $\mathrm{OBJ}_{\mathcal{I}}(x)\leq 1-\eta+\epsilon$ with $R=O_{\epsilon,\delta,t}(1)$ rounds. More specifically, $R=t\ell\left(\frac{3\cdot 2^{t/2-1}}{\epsilon}\right)^{2\ell}+t$ for $\ell=\lceil\lceil\tfrac{t}{2}\rceil\tfrac{1}{2\delta}\rceil$.

Following the strategy introduced in [AOW15], we will do this by first refuting weighted random instances of $k$-xor for $k\geq 1$. After this, any predicate $P:\{\pm 1\}^{k}\to\{0,1\}$ can be decomposed according to its Fourier decomposition, which will yield a weighted sum of $t$-xor instances for $t\leq k$, and our proof system will refute each individually.

5.1 Higher-arity XOR

Ultimately, we will reduce each $k$-CSP to a sum over weighted $t$-xor instances with $t\leq k$:

Definition 5.5**.**

Let $\mathcal{W}$ be a distribution over signed integers. We say that $\mathcal{I}$ is a random $k$-xor instance weighted according to $\mathcal{W}$ if it is sampled as follows: for each ordered multiset $S\subset[n]$ with $|S|=k$, we take a $b_{S}$ to be equal to a uniformly random sample from $\mathcal{W}$, and finally set the objective function to be $\sum_{S}b_{S}\cdot x^{S}$.

Following the standard strategy introduced by [GK01, FGK05] and subsequently honed in many works, we will reduce refuting these $t$-xor instances to refuting 2-xor instances.

5.1.1 Even $k$-XOR

In this case, we perform a standard transformation to view the $k$-xor instance as a 2-xor instance on super-vertices given by subsets of vertices of size $k/2$.

Definition 5.6**.**

Suppose $k>1$ is an integer and $\mathcal{I}$ is a $2k$-xor instance on $n$ variables $x_{1},\ldots,x_{n}$, with objective $\sum_{U\in[n]^{2k}}b_{U}\cdot x^{U}$ where the sum is over ordered multisets $U\subset[n],|U|=2k$. Then we let its flattening, $\mathcal{I}_{flat}$, be the 2-xor instance on $n^{k}$ variables given by associating a new variable $y_{S}$ for each ordered multiset $S\subset[n],|S|=k$, and for each $U\subset[n]$ with $|U|=2k$, choosing the partition of $U$ into the ordered multisets $S,T$ with $S$ containing the first $k$ elements and $T$ containing the last $k$, taking the objective function $\sum_{S,T}b_{U}\cdot y_{S}y_{T}$.

Lemma 5.7**.**

Suppose that $\mathcal{I}$ is a $2k$-xor instance, and let $\mathcal{I}_{flat}$ be the 2-xor instance given by its flattening. Then if the $R$-round Sherali–Adams proof system can certify that $\mathrm{OBJ}_{\mathcal{I}_{flat}}(x)\leq c$, then the $k\cdot R$-round Sherali–Adams proof system can certify that $\mathrm{OBJ}_{\mathcal{I}}(x)\leq c$.

Proof.

Every degree-$R$ Sherali–Adams proof for $\mathcal{I}_{flat}$ can be transformed into a Sherali–Adams proof of degree at most $kR$ for $\mathcal{I}$ by applying the transformation $y_{S}=\prod_{i\in S}x_{i}=x^{S}$. Further, this transformation exactly relates the objective functions of $\mathcal{I}_{flat}$ and $\mathcal{I}$. This proves the claim. ∎

If the $2k$-xor instances that we start with are random weighted instances, then their flattenings are also random weighted 2-xor instances.

Claim 5.8**.**

Suppose that $\mathcal{I}$ is a random $2k$-xor instance on $n$ vertices weighted according to $\mathcal{W}$. Then the flattening $\mathcal{I}_{flat}$ is a random 2-xor instance on $n^{k}$ vertices weighted according to $\mathcal{W}$.

Proof.

This fact is immediate, since the ordered multisets $U\subset[n]$, $|U|=2k$ are in bijection with ordered pairs of multisets $S,T\subset[n]$, $|S|=|T|=k$. ∎

We will require the following proposition, which applies our main theorem in the context of random $k$-xor instances with random weights from well-behaved distributions.

Proposition 5.9**.**

Suppose that $\mathcal{W}$ is a distribution over integers which is symmetric about the origin, and let $n,k\geq 1$ be positive integers. Let $\mathop{\bf E\/}$ denote the expectation under the measure $\mathcal{W}$, and let $\sigma^{2}\geq Ew^{2}$ be a bound on the variance. Furthermore, suppose that

• •

The expected absolute value is at least $\mathop{\bf E\/}|w|\gg\sigma\sqrt{\frac{\log n}{n^{k}}}$,

• •

With high probability over $n^{2k}$ i.i.d. samples $w_{1},\ldots,w_{n^{2k}}\sim\mathcal{W}$, $\max_{i\in[n^{2k}]}|w_{i}|\leq M\ll\sigma^{2}n^{k}$.

Now, define

[TABLE]

Then if $\mathcal{I}$ is a random $2k$-xor instance on $n$ variables weighted according to $\mathcal{W}$, with high probability $\mathcal{I}$ has $\mathop{\bf E\/}|w|\cdot n^{2k}\pm O(\sigma n^{k}\sqrt{\log n})$ constraints. Further, choosing $\ell\in\mathbbm{N}_{+}$ large enough so that $n^{k/4\ell}\rho\leq\frac{1}{2}\epsilon^{2\ell}$ and setting $R=2k\cdot\ell\cdot\left(\frac{1}{\epsilon}\right)^{2\ell}$, $R$ rounds of Sherali–Adams can deduce the bound $\mathrm{OBJ}_{\mathcal{I}}(x)\leq\frac{1}{2}+\frac{3}{2}\epsilon$.

To prove the above, we require the following standard matrix Bernstein inequality:

Theorem 5.10** (Theorem 6.6.1 in [T*+*15]).**

Let $A_{1},\ldots,A_{m}\in\mathbbm{R}^{N\times N}$ be independent random matrices, with $\mathop{\bf E\/}A_{i}=0$ for all $i\in[m]$ and $\|A_{i}\|\leq M$ for all $i\in[m]$. Let $A=\sum_{i\in[m]}A_{i}$ denote their sum, and suppose that $\|\mathop{\bf E\/}AA^{\top}\|,\|EA^{\top}A\|\leq\sigma^{2}$. Then

[TABLE]

Proof of Proposition 5.9.

Given a weighted $2k$-xor instance on $n$ variables with weights from $\mathcal{W}$, we consider its flattening $\mathcal{I}_{flat}$ with objective function $\mathrm{OBJ}(x)=\frac{1}{m}\sum_{i,j\in[N]}\frac{1}{2}(1+b_{ij}x_{i}x_{j})$ for $m$ the absolute sum of weights, we construct its signed adjacency matrix as follows: first take the matrix $W$ defined so that $W_{i,j}=b_{ij}$, and obtain a new matrix $B=\frac{1}{2}(W+W^{\top})$. For any $x$, applying Lemma 5.7 we have that $\frac{1}{2}+\frac{1}{2m}x^{\top}Bx=\mathrm{OBJ}_{\mathcal{I}}(x)$.

Since $\mathcal{W}$ is a distribution over integers, $2B$ has signed integer entries. We think of $2B$ as defining a multigraph $G$ on $n^{k}$ vertices with signed edges, so that there are $2\cdot|B_{ij}|$ multiedges between $i,j\in[n^{k}]$, each with sign $\textnormal{sgn}(B_{ij})$. Let $2\cdot D$ be the degree matrix of $G$, let $A=|2B|$ be the adjacency matrix of $G$, let $\Xi=\textnormal{sgn}(B)$ be the matrix of signs of $B$, and let $K=(2D^{-1})A=D^{-1}B\otimes\Xi$ be the transition matrix for the random walk on $G$.

To apply Corollary 3.7, we must upper bound the spectral radius of $\Xi\circ K=D^{-1}B$, as well as bound the minimum degree of $G$ and the total number of edges. We will use the bound

[TABLE]

First, we bound $\|B\|_{op}$. Take $B^{\prime}$ to be the truncated version of $B$, so that $B_{i,j}^{\prime}=\textnormal{sgn}(B_{i,j})\cdot\max(|B_{i,j}|,M)$. Thinking of the matrix $B^{\prime}$ as the sum of $\binom{n^{k}}{2}+n^{k}$ symmetric matrices, one for each pair $i,j\in[n^{k}]$, satisfies the requirements of Theorem 5.10. We have that $\mathop{\bf E\/}[B^{\prime}B^{\prime\top}]\preceq n^{k}\sigma^{2}\cdot\mathbbm{1}$, so applying Theorem 5.10 with $t=O\max(\sqrt{\sigma^{2}n^{k}\log n},M\log n)$ we get that with high probability,

[TABLE]

where we have also used that with high probability $B=B^{\prime}$ by the properties of $\mathcal{W}$.

Now, we bound the sum of degrees $2m$ and the minimum degree $d_{\min}$. We have that the total sum of the degrees is given by $2m=\sum_{i,j\in[n^{k}]}|b_{ij}|$ with $b_{ij}\sim\mathcal{W}$. By a Bernstein inequality,

[TABLE]

so since by assumption $\sigma^{2}n^{2k}\gg M$, setting $s=O(\sigma n^{k}\sqrt{\log n})$ we have that with high probability

[TABLE]

By our assumptions on $\mathcal{W}$ we have that for every $i\in[n^{k}]$, $\mathop{\bf E\/}\deg_{G}(i)=n^{k}\mathop{\bf E\/}|w|$. Applying a Bernstein inequality gives that

[TABLE]

so using that $M\leq n^{k}\sigma^{2}$ and taking $t=O\left(\sqrt{n^{k}\sigma^{2}\log n}\right)$ we get that $d_{\min}=n^{k}\cdot\mathop{\bf E\/}|w|\pm O(\sqrt{n^{k}\sigma^{2}\log n})$ with high probability. This gives that with high probability,

[TABLE]

where we have used that $\sigma\sqrt{\log n}\ll\sqrt{n^{k}}\mathop{\bf E\/}|w|$.

Now, the result follows by applying Corollary 3.7 and Lemma 5.7. ∎

5.1.2 Odd $k$-XOR

For odd integers $k$, $k$-xor instances do not have the same natural, symmetric flattenings. Instead, we define what we call a lift:

Definition 5.11**.**

Suppose $k\geq 1$ is an integer and $\mathcal{I}$ is a $(2k+1)$-xor instance on $n$ variables $x_{1},\ldots,x_{n}$, with objective $\sum_{U\in[n]^{2k+1}}b_{U}\cdot x^{U}$. Then we let its lift, $\mathcal{I}_{lift}$, be the bipartite 2-xor instance on parts each containing $n^{k+1}$ variables created as follows:

• •

Create new variables $w_{1},\ldots,w_{n}$

• •

For each $U\in[n]^{2k+1}$, choose a random index $i_{U}\in[n]$ and add modify the objective to $\sum_{U\in[n]^{2k+1}}b_{U}\cdot x^{U}\cdot w_{i_{U}}$

• •

For each ordered multiset $S$ associate a new variable $y_{S}$, and for each ordered multiset $T\in[n]^{k}$ and index $i\in[n]$ associate a new variable $z_{T,i}$ We understand $y_{S}=\prod_{i\in S}x_{i}$, and $z_{T,i}=\left(\prod_{j\in T}x_{j}\right)\cdot w_{i}$.

• •

For each $U\in[n]^{2k+1}$, we take the ordered multiset $V=(U,i_{U})$ and assign it a new coefficient $b^{\prime}_{V}=b_{U}$. For the remaining $b^{\prime}_{V}$, we set $b^{\prime}_{V}=0$.

Finally, $\mathcal{I}_{lift}$ is the instance with the objective function $\sum_{S\in[n]^{k+1},T\in[n]^{k},i\in[n]}b^{\prime}_{S\cup T\cup i}\cdot y_{S}z_{T,i}$.

We obtain a statement analogous to Lemma 5.7 for odd $k$-xor:

Lemma 5.12**.**

Suppose that $\mathcal{I}$ is a weighted $(2k+1)$-xor instance, and let $\mathcal{I}_{flat}$ be the bipartite 2-xor instance given by its flattening. Then if the $R$-round Sherali–Adams proof system can certify that $\mathrm{OBJ}_{\mathcal{I}_{flat}}(x)\leq c$, then the $(k+1)\cdot R$-round Sherali–Adams proof system can certify that $\mathrm{OBJ}_{\mathcal{I}}(x)\leq c$.

Proof.

The only modification to the proof of Lemma 5.12 is that for $z_{T,i}$ we substitute $z_{T,i}=x^{T}$ (where we have implicitly substituted $w_{i}=1$ for all $i\in[n]$). ∎

However, the lifting procedure does not preserve the weighting distribution $\mathcal{W}$, because of the step in which a random index $i_{U}$ is chosen to lift $U$. For this reason, we prove an analog of Proposition 5.9:

Proposition 5.13**.**

Suppose that $\mathcal{W}$ is a distribution over integers which is symmetric about the origin, and let $n,k\geq 1$ be integers. Let $\mathop{\bf E\/}$ denote the expectation under the measure $\mathcal{W}$, and let $\sigma^{2}\geq Ew^{2}$ be a bound on the variance. Furthermore, suppose that

• •

The expected absolute value is at least $\mathop{\bf E\/}|w|\gg\sigma\sqrt{\frac{\log n}{n^{k}}}$,

• •

With high probability over $n^{2k+1}$ i.i.d. samples $w_{1},\ldots,w_{n^{2k+1}}\sim\mathcal{W}$, $\max_{i\in[n^{2k+1}]}|w_{i}|\leq M\ll\sigma^{2}n^{k}$.

Now, define

[TABLE]

Then if $\mathcal{I}$ is a random $(2k+1)$-xor instance on $n$ variables weighted according to $\mathcal{W}$, with high probability it has $\mathop{\bf E\/}|w|\cdot n^{2k+1}\pm O(\sigma n^{k}\sqrt{\log n})$ constraints. Furthermore, choosing $\ell\in\mathbbm{N}_{+}$ large enough so that $n^{(k+1)/4\ell}\rho\leq\frac{1}{2}\epsilon^{2\ell}$ and setting $R=(2k+2)\ell\cdot\left(\frac{1}{\epsilon}\right)^{2\ell}$, $R$ rounds of Sherali–Adams can deduce the bound $\mathrm{OBJ}_{\mathcal{I}}(x)\leq\frac{1}{2}+\frac{3}{2}\epsilon$.

Proof.

The thread of the proof is the same as that of Proposition 5.9. We will refute $\mathcal{I}_{lift}$, since by Lemma 5.12 this is sufficient. We begin by associating with $\mathcal{I}_{lift}$ a multigraph $G$ (which we may do because $\mathcal{W}$ is a distribution over integers). The multigraph $G$ is a bipartite graph, with one bipartition corresponding to variables $y_{S}$ for $S\in[n]^{k+1}$, and one bipartition corresponding to variables $z_{T,i}$ for $T\in[n]^{k}$ and $i\in[n]$. We let the block matrix $B$ be (the $\tfrac{1}{2}$-scaled) signed adjacency matrix of $G$, let $\Xi$ be the matrix of signs, $D$ be the diagonal degree matrix, and $K\circ\Xi=D^{-1}B$ be the signed transition matrix of the random walk on $G$. In order to apply Corollary 3.7 we must bound $\|K\circ\Xi\|$ and $\pi^{*}=d_{\min}(G)/2m$.

First, we bound the vertex degrees. For a vertex of the form $(T,i)$, the expected value of the incident edge $(S,T\cup i)$ is $b_{S,T}\cdot\frac{1}{n}$, where $b_{S,T}$. The degree of $T\cup i$ is simply the sum

[TABLE]

a sum of independent random variables with expectation $\frac{1}{n}\mathop{\bf E\/}|w|$ and variances $\frac{1}{n}\sigma^{2}$. Applying a Bernstein inequality, we have that

[TABLE]

so taking $t=O(\sqrt{\sigma^{2}n^{k}\log n})$ (and using that $M\ll n^{k}\sigma^{2}$), we have that the degree of $T\cup i$ vertices is $\deg_{G}(T\cup i)=n^{k}\mathop{\bf E\/}|w|\pm O(\sqrt{\sigma^{2}n^{k}\log n})$ with high probability.

A similar argument applies to the $S$ vertices; the total degree of such a vertex is

[TABLE]

since only one of the $b_{S,T\cup i}$ will be nonzero. The inner sums are independent random variables with mean $\mathop{\bf E\/}|w|$ and variance $\sigma^{2}$, therefore

[TABLE]

so taking $t=O(\sqrt{\sigma^{2}n^{k}\log n})$ (and using that $M\ll n^{k}\sigma^{2}$), we have that the degree of $S$ vertices is also $\deg_{G}(S)=n^{k}\mathop{\bf E\/}|w|\pm O(\sqrt{\sigma^{2}n^{k}\log n})$ with high probability.

We finally bound $\|K\circ\Xi\|\leq\|D^{-1}\|\cdot\|B\|$. As before, $B$ is a sum of independent symmetric matrices, one for each coefficient $b_{U}$ from $\mathcal{I}$. That is, we can define matrices $B_{U}$ for each $U\in[n]^{2k+1}$ with $U=S,T$ for $S\in[n]^{k+1},T\in[n]^{k}$ where $B_{U}$ has a number $b_{U}\sim\mathcal{W}$ in one off-diagonal block entry $(S,T\cup i)$ and the other off-diagonal block entry $(T\cup i,S)$ for a randomly chosen $i\in[n]$. Thus, $\mathop{\bf E\/}B_{U}B_{U}^{\top}$ is a diagonal matrix with $\frac{1}{n}\sigma^{2}$ on each diagonal of the form $(T\cup i,T\cup i)$ and $\sigma^{2}$ on each block diagonal of the form $(S,S)$. We then have that $\mathop{\bf E\/}BB^{\top}\preceq n^{k}\sigma^{2}\mathbbm{1}$, since for each $S$ there is a sum over $n^{k}$ matrices $B_{U}$ and for each $T\cup i$ there is a sum over $n^{k+1}$ matrices $B_{U}$. Applying Theorem 5.10 by using the same truncation trick again, we have that

[TABLE]

and from this we have that with high probability,

[TABLE]

After which we can apply Corollary 3.7. ∎

5.2 From Boolean CSPs to $k$-XOR

Following [AOW15], we prove Theorem 5.2 via reduction to XOR.

Proof of Theorem 5.2.

Given a random instance of the CSP defined by the predicate $P$, and $p=n^{-\lfloor k/2\rfloor+\delta}$, a Bernstein inequality gives us that the number of constraints $m$ is with high probability given by $m=n^{\lceil k/2\rceil+\delta}\pm 10\sqrt{n^{\lceil k/2\rceil+\delta}\log n}$. Set $\ell=\lceil\frac{1}{2\delta}\rceil$.

Since $P$ is a Boolean predicate, we can write $P$ in its Fourier expansion:

[TABLE]

Using this expansion, we re-write the objective function. Recall that $[n]^{k}$ is the set of all ordered multisets of $k$ elements of $[n]$. For each $S\in[n]^{k}$, let $b_{S}$ be the $0/1$ indicator that there is a constraint on $S$. Then, if the total number of constraints is $m$,

[TABLE]

Now, define for each $\alpha\subseteq[k]$ with $|\alpha|=t>0$ the $t$-xor instance

[TABLE]

where we have taken $w_{T}=\sum_{S\in[n]^{k},S|_{\alpha}=T}b_{S}\cdot\prod_{a\in\alpha}(\zeta_{S})_{i_{a}}$. So that from Equation 19,

[TABLE]

Let $\mathcal{W}_{n^{k-t}}$ be the distribution defined so that $w\sim\mathcal{W}_{t}$ is a sum of $n^{k-t}$ independent variables taking value $\{\pm 1\}$ with probability $p$ and value [math] otherwise. Since for each $S\supseteq T$, the quantity $\prod_{a\in\alpha}(\zeta_{S})_{i_{a}}$ is an independent uniform sign in $\{\pm 1\}$ and $b_{S}$ is an independent Bernoulli-$p$ variable, we have that the coefficients $w_{T}$ in $\mathcal{I}^{\alpha}$ are i.i.d. from $\mathcal{W}_{n^{k-t}}$. The following lemma establishes some properties of $\mathcal{W}_{N}$ (we will prove the lemma in Appendix A):

Lemma 5.14**.**

Let $\mathcal{W}_{N}(p)$ be the distribution defined so that $X\sim\mathcal{W}_{N}$ is given by $X=\sum_{t=1}^{N}Y_{t}\cdot Z_{t}$, where the $\{Y_{t}\}_{t},\{Z_{t}\}_{t}$ are i.i.d with $Y_{t}\sim\mathrm{Ber}(p)$ and $Z_{t}\sim\{\pm 1\}$. Then for $X\sim\mathcal{W}_{N}(p)$, $\mathop{\bf E\/}X=0$ and $\mathop{\bf E\/}X^{2}=pN$. Further, so long as $pN\geq 1$, $\mathop{\bf E\/}|X|\geq\frac{2}{e^{3/2}}\sqrt{pN}$, and $\mathop{\bf Pr\/}(|X|>2t\sqrt{pN})\leq 2\exp\left(-t^{2}\right)$. Otherwise, if $pN\leq 1$, $\mathop{\bf E\/}|X|\geq\frac{1}{2e}\log\frac{1}{1-pN}$, and $\mathop{\bf Pr\/}(|X|\geq 1+t)\leq\exp\left(-\frac{1}{2}t\right)$.

From Lemma 5.14, we have that $\mathop{\bf E\/}w_{T}^{2}=pn^{k-t}$, and by Cauchy-Schwarz $\mathop{\bf E\/}|w_{T}|\leq\sqrt{\mathop{\bf E\/}w_{T}^{2}}$. Let $m_{\alpha}$ be the total absolute weight of constraints in $\mathcal{I}^{\alpha}$,

[TABLE]

Notice that in all cases, $m_{\alpha}\leq m$.

Now, we show that SA can certify upper bounds on $|\mathcal{I}^{\alpha}(x)|$ for every $\alpha$. First, consider $\alpha$ with $|\alpha|=t=1$. In this case, Sherali–Adams with $R=1$ can certify that

[TABLE]

From an application of Bernstein’s inequality (the same as in the proof of Proposition 5.9), $m_{\alpha}\leq n\cdot\sqrt{\mathop{\bf E\/}w_{T}^{2}}+\sqrt{pn^{k}\log n}$ with high probability whenever $pn^{k}\gg\sqrt{pn^{k-1}}$, and applying our bound on $m$ we conclude that with high probability SA will certify that

[TABLE]

The lower bound on $\mathcal{I}^{\alpha}(x)$ follows from identical reasoning with its negation, so we can conclude that with high probability SA can certify that

[TABLE]

Now, we tackle $\alpha$ with $|\alpha|=t$ for $2\leq t\leq k$. We will verify that the conditions of Propositions 5.9 and 5.13 hold. First, consider the $\alpha$ with $|\alpha|=t$ for $pn^{k-t}\geq 1$. From Lemma 5.14, in this case we have that $\mathop{\bf E\/}|w_{T}|\geq\frac{2}{e^{3/2}}\sqrt{pn^{k-t}}$, and with high probability, $|w_{T}|\leq O(\sqrt{tpn^{k-t}\log n})$ for all $T\in[n]^{t}$. Letting $M=O(\sqrt{tpn^{k-t}\log n})$, and $\sigma^{2}=pn^{k-t}$, we meet the conditions for Proposition 5.9:

[TABLE]

so long as $pn^{k-t}\geq 1$, which we have assumed. So applying Propositions 5.9 and 5.13 to both $\frac{m}{m_{\alpha}}\mathcal{I}^{\alpha}$ and $-\frac{m}{m_{\alpha}}\mathcal{I}^{\alpha}$, we have

[TABLE]

and so long as $\frac{m_{\alpha}}{m}\cdot n^{\lceil t/2\rceil/4\ell}\rho\leq\frac{1}{2}\epsilon^{2\ell}$, with high probability over $\mathcal{I}^{\alpha}$, $t(\ell r+1)$ rounds of Sherali–Adams certify that $|\mathcal{I}^{\alpha}(x)|\leq\frac{3}{2}\epsilon$. We confirm that

[TABLE]

whenever $t\geq 1$ and $\ell\geq 1$.

Finally, we handle $\alpha$ with $|\alpha|=t$ for $2\leq t$ and $pn^{k-t}<1$. From Lemma 5.14 we have $\mathop{\bf E\/}|w_{T}|\geq\frac{1}{e}\log\frac{1}{1-pn^{k-t}}$, and with high probability, $|w_{T}|\leq 4\log n$ for all $T\in[n]^{t}$. Taking $M=4\log n$, we have that we meet the conditions of Propositions 5.9 and 5.13

[TABLE]

where the last inequality is true whenever $pn^{k-t+\lfloor t/2\rfloor}=n^{\lceil k/2\rceil-\lceil t/2\rceil+\delta}\gg\log n$, which we have by assumption. So applying Propositions 5.9 and 5.13 to both $\frac{m}{m_{\alpha}}\mathcal{I}^{\alpha}$ and $-\frac{m}{m_{\alpha}}\mathcal{I}^{\alpha}$, we have that for

[TABLE]

so long as $\frac{m_{\alpha}}{m}n^{\lceil t/2\rceil/4\ell}\rho\leq\frac{1}{2}\epsilon^{2\ell}$, $R=t(\ell r+1)$ rounds of Sherali–Adams certify that $|\mathcal{I}^{\alpha}(x)|\leq\frac{3}{2}\epsilon$ with high probability. Verifying,

[TABLE]

which is maximized at $t=k$. By our choice of $\ell$, the condition holds.

We therefore have (using Parseval’s identity and $\|x\|_{1}\leq\sqrt{k}\|x\|_{2}$ for $x\in\mathbbm{R}^{k}$ to simplify Equation 20) that the same number of rounds certifies that

[TABLE]

as desired. ∎

Using arguments analogous to the above along with the reasoning outlined in Theorem 4.9, proof 2 and Claim 6.7 from [AOW15], we can also prove Theorem 5.4.

Acknowledgments

We thank Luca Trevisan for helpful comments, and Boaz Barak for suggesting the title. We also thank the Schloss Dagstuhl Leibniz Center for Informatics (and more specifically the organizers of the CSP Complexity and Approximability workshop), as well as the Casa Mathemática Oaxaca (and more specifically the organizers of the Analytic Techniques in TCS workshop); parts of this paper came together during discussions at these venues. T.S. also thanks the Oberwolfach Research Institute for Mathematics (and the organizers of the Proof Complexity and Beyond workshop), the Simons Institute (and the organizers of the Optimization semester program), and the Banff International Research Station (and the organizers of the Approximation Algorithms and Hardness workshop) where she tried to prove the opposite of the results in this paper, as well as Sam Hopkins, with whom some of those efforts were made.

Appendix A Characteristics of distributions of XOR-subformula coefficients

We now prove Lemma 5.14. We will use the following estimate for the mean absolte deviation of a binomial random variable.

Lemma A.1** (e.g. [Bly80]).**

If $X$ is distributed according to the binomial distribution $X\sim\mathrm{Bin}(n,p)$, then $\mathop{\bf E\/}|X-\mathop{\bf E\/}X|=\sqrt{\frac{2}{\pi}np(1-p)}+O(\frac{1}{\sqrt{n}})$.

Proof of lemma 5.14.

We calculate the absolute value directly. Given that there are exactly $k$ nonzero $Y_{t}$, the absolute value of $X$ is distributed according to $|\mathrm{Bin}(k,\tfrac{1}{2})-\tfrac{1}{2}k|$. Using the method of conditional expectations,

[TABLE]

where we have applied the estimate from Lemma A.1. Letting $D(a\|b)=a\ln\frac{a}{b}+(1-a)\ln\frac{1-a}{1-b}$ be the relative entropy, we then have from Stirling’s inequality that

[TABLE]

Now, if $pN<1$, we take

[TABLE]

as desired.

If $pN\geq 1$, applying the change of variables $\ell=k-\lfloor pN\rfloor$ and $\delta=\frac{\ell}{N}$,

[TABLE]

Now using the Taylor expansion for $\log(1-x)$ to simplify and restricting the sum over the range $\ell=[-\lfloor\sqrt{pN}\rfloor,\lfloor\sqrt{pN}\rfloor]$, we get the bound

[TABLE]

as desired.

The first and second moment we can also obtain by calculation; the $Z_{t}$ ensure that the summands have mean [math], and the $Y_{t}$ give that the variance of the summands is $p$, which gives the result.

The tail bound $\mathop{\bf Pr\/}(|X|\geq(1+2t)\sqrt{pN})\leq 2\exp(-t^{2})$ comes from an application of Bernstein’s inequality if $pN\geq 1$; when $pN<1$, we again apply Bernstein’s inequality, in which case we have

[TABLE]

and choosing $s=1+t$ gives the result. ∎