Some Applications and Limitations of Convex Optimization Hierarchies for Discrete and Continuous Optimization Problems

TL;DR

This thesis investigates the capabilities and limitations of convex relaxation hierarchies in approximating various discrete and continuous optimization problems, revealing new hardness results and algorithmic trade-offs.

Contribution

It provides a dichotomy for CSP approximability via LP relaxations, analyzes the sum-of-squares hierarchy for polynomial optimization, and establishes NP-hardness for matrix norm approximation in hypercontractive cases.

Findings

LP relaxations match Sherali-Adams hierarchy approximations for CSPs

Trade-offs between approximation ratio and runtime for polynomial optimization

NP-hardness of hypercontractive matrix norm approximation

Abstract

This thesis explores algorithmic applications and limitations of convex relaxation hierarchies for approximating some discrete and continuous optimization problems. - We show a dichotomy of approximability of constraint satisfaction problems (CSPs) by linear programming (LP) relaxations: for every CSP, the approximation obtained by a basic LP relaxation, is no weaker than the approximation obtained using relaxations given by super-constant levels of the Sherali-Adams hierarchy on instances of size $n$. - For the problem of approximating the absolute maximum of an n-variate degree-d homogeneous polynomial f with real coefficients over the unit sphere, we analyze the optimum value of the level-t sum-of-squares (SoS) SDP relaxation of the problem. Our results offer a trade-off between the approximation ratio and running time, which can take advantage of additional structure in the polynomial, such as non-negativity or sparsity of the coefficients. - We study the problem of approximating the $p \to q$-norm of a matrix $A$, and prove the first NP-hardness result for approximating norms in the hypercontractive case $1< p < q < \infty$. We also prove almost tight algorithmic results for the case when $p \geq q$ (with $2 \in [q,p]$) where constant factor approximations for the matrix norms are possible. A common theme for these results is their connection to geometry. For the discrete optimization problem of CSP, geometry appears as a crucial tool for our lower bound proof. For the problem of polynomial optimization, we show that SDPs capture and extend earlier algorithms based on diameter estimation for convex bodies. For the matrix (operator) norm problem, the definition itself is geometric in nature and embedding theorems play a crucial role in our proofs.

Equations

$$\left|\mathrm{OPT}(\Phi )-\underset{x\in \{0,1{\}}^k}{\mathbb{E}}\left[f(x)\right]\right|\le \epsilon \mathrm{FRAC}(\Phi )\ge 1-\epsilon ,$$

$$C_i\equiv f(x_{i_1}+b_{i_1},\dots ,x_{i_k}+b_{i_k})$$

$$\mathrm{OPT}(\Phi )=\underset{\sigma :\{x_1,\dots ,x_n\}\mapsto [q]}{\max }\mathsf{sat}(\sigma ).$$

$$\underset{C\in \Phi }{\mathbb{E}}\left[\sum _{\alpha \in [q{]}^k}f(\alpha \cdot b_C)\cdot x_{(S_C,\alpha )}\right]$$

$$x_{(S,\alpha )}$$

$$x_{(\varnothing ,\varnothing )}$$

$$\sum _{\begin{array}{c}\alpha \in [q{]}^S\\ \alpha {|}_T=\beta \end{array}}{x}_{(S,\alpha )}$$

$$\underset{C\in \Phi }{\mathbb{E}}\left[\sum _{\alpha \in [q{]}^k}f(\alpha +b_C)\cdot x_{(S_C,\alpha )}\right]$$

$$\sum _{j\in [q]}{x}_{(i,b)}$$

$$\sum _{\begin{array}{c}\alpha \in [q{]}^{S_C}\\ \alpha (i)=b\end{array}}{x}_{(S_C,\alpha )}$$

$$x_{(S_C,\alpha )}$$

$${\mathrm{cl}}_R(S)=S\cup \bigcup _{\begin{array}{c}P:P\text{ is a path in H}\\ P\text{ connects }u,v\in S\\ |P|\le R\end{array}}V(P).$$

$$d_{\mu }^H(u,v):=1-{(1-\mu )}^{2\cdot d_H(u,v)}\text{ and }{\rho }_{\mu }^H(u,v):=\sqrt{\frac{2\cdot d_{\mu }^H(u,v)+\mu }{1+\mu }},$$

$$\mathbb{E}\left[E_S\right]\le \sum _{u\in V(H^{\prime })}\mathbb{E}\left[\mathrm{deg}(u)\cdot \text{mathds}{1}_{u\in S}\right]$$

$$\le \sum _{u\in V(H^{\prime })}\mathbb{E}\left[\mathrm{deg}(u)\mid u\in S\right]\cdot \exp \left(-D/4\right)$$

$$\le \sum _{u\in V(H^{\prime })}\left(D+k\gamma \right)\cdot \exp \left(-D/4\right)$$

$$\le (n\cdot n_0)\cdot 2D\cdot \exp \left(-D/4\right).$$

$$\underset{H\sim {\mathcal{H}}_k\left(m,n,n_0,\Gamma \right)}{\mathbb{E}}\left[Z_l(H)\right]\le {\left(k^2\cdot \frac{m}{n}\right)}^{2l}.$$

$$\left(\genfrac{}{}{0ex}{}{n}{h}\right){\left(\frac{m{k}^2}{n^2}\right)}^h\le n^h{\left(\frac{m{k}^2}{n^2}\right)}^h={\left(k^2\cdot \frac{m}{n}\right)}^h$$

$$\underset{H\sim {\mathcal{H}}_k\left(m,n,n_0,\Gamma \right)}{\mathbb{P}}\left[H\text{is not}(\tau ,\eta )\text{-sparse}\right]\le 3\cdot {\left(\frac{k^{3k}\cdot \gamma }{n^{\eta /4}}\right)}^{1/k}.$$

$$\left|E(S)\right|\ge \frac{\left|S\right|}{k-1-\eta }\Rightarrow \left|E(S^{\prime })\right|\ge \frac{\left|S\right|}{k-1-\eta }\ge \frac{\left|S^{\prime }\right|}{k-1-\eta }.$$

$$\sum _{h=1}^{{\tau }^{\prime }\cdot n}\left(\genfrac{}{}{0ex}{}{n}{h}\right)\cdot \left(\genfrac{}{}{0ex}{}{h^k}{r}\right)\cdot {\left(\frac{k!\cdot m}{n^k}\right)}^r,$$

$$\sum _{h=1}^{{\tau }^{\prime }\cdot n}\left(\genfrac{}{}{0ex}{}{n}{h}\right)\cdot \left(\genfrac{}{}{0ex}{}{h^k}{r}\right)\cdot {\left(\frac{k!\cdot m}{n^k}\right)}^r$$

$$=\sum _{h=1}^{{\tau }^{\prime }\cdot n}{\left(e^{k-\eta }\cdot (k-1-\eta )\cdot k^k\cdot \gamma \cdot {\left(\frac{h}{n}\right)}^{\eta }\right)}^{h/(k-1-\eta )}$$

$$\le \sum _{h=1}^{{\tau }^{\prime }\cdot n}{\left(k^{3k}\cdot \gamma \cdot {\left(\frac{h}{n}\right)}^{\eta }\right)}^{h/(k-1-\eta )}$$

$$\sum _{h=n^{\theta }}^{{\tau }^{\prime }\cdot n}{\left(k^{3k}\cdot \gamma \cdot {\left(\frac{h}{n}\right)}^{\eta }\right)}^{h/(k-1-\eta )}$$

$$\le 2\cdot \exp \left(-\frac{n^{\theta }}{k}\right)$$

$$\le 2\cdot \frac{k}{n^{\theta }},$$

$$\sum _{h=1}^{n^{\theta }}{\left(k^{3k}\cdot \gamma \cdot {\left(\frac{h}{n}\right)}^{\eta }\right)}^{h/(k-1-\eta )}$$

$$\le {\left(\frac{k^{3k}\cdot \gamma }{n^{\eta /4}}\right)}^{1/k}=k^3\cdot {\gamma }^{1/k}\cdot n^{-\theta /2}.$$

$$\mathcal{E}\stackrel{\mathrm{def}}{=}\left\{(v,e)\mid v\in V(H),e\in E(H),v\in e\right\}.$$

$$\left|E^{\prime }\right|\le \left|E(S)\right|\le \frac{\left|S\right|}{k-1-\eta }.$$

$$\left|S\right|-\left|V^{\prime }\right|\le k\cdot \left|E^{\prime }\right|-\left|{\mathcal{E}}^{\prime }\right|.$$

$$(k-1-\eta )\cdot \left|E^{\prime }\right|\le \left|V^{\prime }\right|+k\cdot \left|E^{\prime }\right|-\left|{\mathcal{E}}^{\prime }\right|⟹\left|{\mathcal{E}}^{\prime }\right|\le (1+\eta )\cdot \left|E^{\prime }\right|+\left|V^{\prime }\right|.$$

$$d_{\mu }^H(u,v):=1-(1-\mu {)}^{2\cdot d_H(u,v)}\text{ and }{\rho }_{\mu }^H(u,v):=\sqrt{\frac{2\cdot d_{\mu }^H(u,v)+\mu }{1+\mu }},$$

$$d_{\mu }^G(u,v):=1-{(-1)}^{d_G(u,v)}{(1-\mu )}^{d_G(u,v)}\text{ and }{\rho }_{\mu }^G(u,v):=\sqrt{\frac{2\cdot d_{\mu }^G(u,v)+\mu }{1+\mu }}.$$

$$\Vert {\psi }_S(u)-{\psi }_S(v{)\Vert }_2={\rho }_{\mu }^G(u,v).$$

Full text

We show a dichotomy of approximability of constraint satisfaction problems (CSPs) by linear programming (LP) relaxations: for every CSP, the approximation obtained by a basic LP relaxation, is no weaker than the approximation obtained using relaxations given by $\Omega\left(\frac{\log n}{\log\log n}\right)$ levels of the Sherali-Adams hierarchy on instances of size $n$.

The polynomial optimization problem we consider: given an n-variate degree-d homogeneous polynomial f with real coefficients, compute a unit vector $x\in\mathbb{R}^{n}$ that maximizes $|f(x)|$. We give approximation algorithms for this problem that offer a trade-off between the approximation ratio and running time: in $n^{O(q)}$ time (for $2d\operatorname{\operatorname{|}}q$ and $q\leq n$), we get an approximation within factor $O_{d}(n/q)^{d/2-1}$ for arbitrary polynomials, $O_{d}(n/q)^{d/4-1/2}$ for polynomials with non-negative coefficients, and $(m/q)^{1/2}$ for sparse polynomials with m monomials. The approximation guarantees are with respect to the optimum of the level-q sum-of-squares (SoS) SDP relaxation of the problem, even though our algorithms do not rely on actually solving the SDP. For polynomials with non-negative coefficients, we prove an $\tilde{\Omega}(n^{1/6})$ gap for the degree-$4$ case, for a related relaxation.

The $p\to q$-norm of a matrix $A\in\mathbb{R}^{m\times n}$ is defined as $\|A\|_{p\to q}~\stackrel{{\scriptstyle\mathrm{def}}}{{=}}~\sup_{x\in\mathbb{R}^{n}\setminus\{0\}}\frac{\|Ax\|_{q}}{\|x\|_{p}}$. The regime when $p<q$, known as hypercontractive norms. The case with $p=2$ and $q>2$ was studied in [BBH*+*12]. They proved a hardness of approximation result based on the Exponential Time Hypothesis. We prove the first NP-hardness result for approximating hypercontractive norms: for any $1<p<q<\infty$ with $2\notin[p,q]$, $\|A\|_{p\to q}$ is hard to approximate within $2^{O((\log n)^{1-\varepsilon})}$ assuming $\textsf{NP}\not\subseteq\textsf{BPTIME}\left(2^{(\log n)^{O(1)}}\right)$. We also prove almost tight results for the case when $p\geq q$ with $2\in[q,p]$. For such $p$ and $q$, we show that $\|A\|_{p\to q}$ is NP-hard to approximate within any factor smaller than $1/(\gamma_{p^{*}}\cdot\gamma_{q})$, where $p^{*}:=p/(p-1)$ is the dual norm of $p$. We also give an approximation algorithm with approximation ratio $(1+\epsilon_{0})/(\sinh^{-1}(1)\cdot\gamma_{p^{*}}\,\gamma_{r^{*}})$ for some fixed $\epsilon_{0}\leq 0.00863$. For the algorithm we develop a generalization of random hyperplane rounding using Hölder-duals of Gaussian projection, which may prove useful for rounding $\ell_{p}$-norm bounded vectors arising from convex relaxations ($p\geq 2$).

Acknowledgements

I thank my advisor, Madhur Tulsiani, for his constant support and encouragement over the years. He has been incredibly patient and magnanimous with his time and advice, both in academic and non-academic matters. I also thank other members of my thesis committee, Yury Makarychev and Aaron Potechin, who have been supportive throughout the process.

I am especially thankful to my co-authors Vijay Bhattiprolu and Euiwoong Lee for their friendship. Working with them has been both fun and enlightening. I also thank Venkatesan Guruswami for lending us his valuable time and expertise.

TTIC is a stimulating and nurturing place for graduate students. Plenty of workshops, seminars, and social events promote collaboration in a relaxed environment. Much of the credit goes to administrative staff and the leadership. Special thanks to Mary who is a smiling presence day and night at the institute. Thank you, Adam, Amy, Chrissy, Erica, Jessica and everyone else who makes navigating the system easy. I also thank David McAllester and Avrim Blum for fostering an excellent learning environment during their tenure as Chief Academic Officer.

I was fortunate to partake in useful theory courses at TTIC and UChicago offered by László Babai, Alexander Razborov, Andrew Drucker, Aaron Potechin, Julia Chuzhoy, Yury Makarychev, and Madhur Tulsiani.

The students at TTIC and Uchicago made up an enjoyable social circle in Chicago. I would like to thank Haris, Shubhendu, Behnam, Somaye, Pooya, and Denis for welcoming me to TTIC and Uchicago. I am also grateful to many friends I met along the way, including, Fernando, Leonardo, Chris, Goutham, Pushkar, Shubham, and Sudarshan, among others. Special thanks to Rachit for organizing many of our trips around Chicago, and for the numerous trips to Devon. Many thanks to Shashank and Akash for their constant friendship and encouragement to complete this thesis.

Last but not least, I thank my family for their love. My parents always supported and encouraged me to pursue an academic career, and I dedicate this thesis to them.

Contents

1. 1 Introduction

2. 2 Lifting LP Lower Bounds for CSPs to Sherali-Adams

3. 2.1 The Result in Context: Background and Overview

4. 2.2 Preliminaries of CSPs and Their Relaxations

5. 2.2.1 Constraint Satisfaction Problems

6. 2.2.2 The LP Relaxations for Constraint Satisfaction Problems

7. 2.3 Hypergraphs

8. 2.3.1 Properties of Random Hypergraphs

9. 2.4 Local L2-embeddability of the Metric rho

10. 2.5 Decompositions of Hypergraphs from Local Geometry

11. 2.6 The Sherali-Adams Integrality Gaps Construction

12. 2.6.1 Integrality Gaps from the Basic LP

13. 2.6.2 Integrality Gaps for Resistant Predicates

14. 2.6.3 Lower-bounds for LP Extended Formulations

15. 3 Optimizing Polynomials Over Sphere

16. 3.1 Introduction and Context

17. 3.1.1 Our Algorithmic Results

18. 3.1.2 Connection to sum-of-squares hierarchy

19. 3.1.3 Related Previous and Recent Works

20. 3.2 Overview of Proofs and Techniques

21. 3.2.1 Warmup: $(n^{2}/q^{2})$-Approximation

22. 3.2.2 Exploiting Easy Substructures via Folding and Improved Approximations

23. 3.2.3 Lower Bounds for Polynomials with Non-negative Coefficients

24. 3.3 Preliminaries on the SoS Hierarchy for Polynomials

25. 3.3.1 Pseudoexpectations and Moment Matrices

26. 3.3.2 Matrix Representations of Polynomials and Relaxations

27. 3.3.3 An additional operation on folded polynomials

28. 3.4 Intermediate Results for Degree d (general) Polynomials and NNC Polynomials

29. 3.4.1 Reduction to Multilinear Polynomials

30. 3.4.2 ${(n/q)}^{d/4}$-Approximation for Non-negative Coefficient Polynomials

31. 3.4.3 ${(n/q)}^{d/2}$-Approximation for General Polynomials

32. 3.4.4 $\sqrt{m/q}$-Approximation for $m$-sparse polynomials

33. 3.5 Improved Approximations via Folding

34. 3.5.1 Reduction to Multilinear Folded Polynomials

35. 3.5.2 Relating Evaluations of $f$ to Evaluations of $F_{2\alpha}$

36. 3.5.3 Bounding $\Lambda_{C}\left(\right)$ of Multilinear Folded Polynomials

37. 3.5.4 Final Approxiation Result for NNC Polynomials

38. 3.5.5 Final Approximation for General Polynomials

39. 3.5.6 Algorithms

40. 3.6 Oracle Lower Bound

41. 3.7 Constant Level Lower Bounds for Polynomials with Non-negative Coefficients

42. 3.7.1 Upper on the Clique Polynomial

43. 3.7.2 Lower Bound on $\Lambda\left(f\right)$

44. 3.8 Lifting Spectral Lower Bounds to Higher Levels

45. 3.8.1 Gap Between Spectral Relaxation and Optimal for NNC Polynomials

46. 3.8.2 Tetris Theorem

47. 3.8.3 Lifting Stable Degree-$4$ Lower Bounds

48. 3.8.4 Proof of Tetris Theorem

49. 4 Matrix Norm: Hardness and Approximation

50. 4.1 Introduction

51. 4.1.1 Hardness Results: Collected

52. 4.1.2 The Search For Optimal Constants and Optimal Algorithms

53. 4.1.3 Relation to Factorization Theory

54. 4.1.4 Approximability and Factorizability

55. 4.1.5 Proof Overview: Hardness

56. 4.1.6 Proof Overview: Algorithm

57. 4.2 Preliminaries and Notation

58. 4.2.1 Matrix Norms

59. 4.2.2 Preliminaries for Hardness Results

60. 4.2.3 Preliminaries for Algorithm

61. 4.3 Hardness of $2\to r$ with $r<2$

62. 4.3.1 Reduction and Completeness

63. 4.3.2 Soundness

64. 4.4 Hardness of $p\to q$

65. 4.4.1 Hardness for $p\geq 2\geq q$

66. 4.4.2 Reduction from $p\to 2$ via Approximate Isometries

67. 4.4.3 Derandomized Reduction

68. 4.4.4 Hypercontractive Norms Productivize

69. 4.4.5 A Simple Proof of Hardness for the Case $2\notin[q,p]$

70. 4.5 Analyzing the Approximation Ratio via Rounding

71. 4.5.1 Krivine’s Rounding Procedure

72. 4.5.2 Generalizing Random Hyperplane Rounding – HölderDual Rounding

73. 4.5.3 Generalized Krivine Transformation and the Full Rounding Procedure

74. 4.5.4 Auxiliary Functions

75. 4.5.5 Bound on Approximation Factor

76. 4.6 Hypergeometric Representation

77. 4.6.1 Hermite Coefficients via Parabolic Cylinder Functions

78. 4.6.2 Taylor Coefficients of f

79. 4.7 Bound on Defect

80. 4.7.1 Behavior of The Coefficients of The Inverse Function

81. 4.7.2 Bounding Inverse Coefficients

82. 4.8 Factorization of Linear Operators

83. 4.8.1 Integrality Gap Implies Factorization Upper Bound

84. 4.8.2 Improved Factorization Bounds for Certain p,q-norms

85. 4.8.3 Grothendieck Bound on Approximation Ratio

Chapter 1 Introduction

Time complexity classes are usually defined for decision problems, i.e., to decide whether a binary string is in a given language. Some of the most well-known complexity classes are P, NP, and EXP. Separating P from NP is one of the central open questions of complexity theory. In contrast, optimization problems require optimizing some quantity over a set of feasible solutions. For example, in the Maximum Clique problem, we want to find the size of the largest clique in a given graph $G$. Such optimization problems can easily be turned into decision problems. For Maximum Clique, the decision version is the language consisting of binary encodings of the pairs $(G,k)$ where $G$ is a graph and $k\in\mathbb{N}$ is the size of the largest clique in $G$. Thus, notions of P, NP, NP-hardness, and NP-completeness of decision problems can be ported to the optimization version.

The solution spaces for optimization problems come in two forms: discrete and continuous. For a discrete optimization problem, the space of solutions is finite for any given instance, whereas for a continuous optimization problem, the solution space is uncountable. The Maximum Clique problem is a discrete optimization problem. Finding the largest eigenvalue of a Hermitian matrix is an example of a continuous optimization problem (which is efficiently computable).

Many natural optimization problems of interest turn out to be NP-complete or NP-hard. As solving these exactly would show $\textsf{P}=\textsf{NP}$, these are believed to be computationally intractable. The next best thing might be to hope for an approximate solution to these optimization problems. For an instance $I$ of an optimization problem, we use $\operatorname{\operatorname{OPT}}(I)\geq 0$ to denote the value of the optimal solution. For a maximization problem, the value $v$ is said to be $\alpha$-approximation (approximation factor for us will be $\geq 1$) if $v\cdot\alpha\leq\operatorname{\operatorname{OPT}}(I)$ – approximation factor for minimization problems may be defined similarly. For many NP-complete problems, efficient non-trivial approximations have been achieved. So we may ask what are the limits to such approximation. For the NP-complete Maximum Clique problem, we observe that approximating the value within factor $\nicefrac{{n}}{{(n-1)}}$ for graphs on $n$ vertices would solve the problem exactly (and thus establishing an easy NP-completeness of approximation). Interestingly, although all NP-complete problems are equivalent in terms of exact solvability, they can behave very differently when it comes to approximability. For example, 3-SAT is shown to be NP-hard to approximate within factor better than $\nicefrac{{8}}{{7}}$ in [Hås01]. Whereas, for the Maximum Independent Set problem, the result in [Hås96b] shows hardness of approximation within factor $n^{1-\varepsilon}$ for any $\varepsilon>0$.

The problem of establishing NP-completeness of approximation for various optimization problems turned out to be a very interesting and fruitful area of research. A powerful tool, the Probabilistically Checkable Proof (PCP) was developed to prove NP-hardness of approximation. PCP has a long history, and an overview can be found in [Hås01]. PCPs and their variants have been used to prove optimal inapproximability of some problems, i.e., one can prove NP-hardness of $\alpha$-factor approximation. Simultaneously, we have efficient algorithms, perhaps probabilistic, to compute $\alpha+\varepsilon$-approximation for $\varepsilon>0$ (the runtime may or may not depend on $\varepsilon$). The problem 3-SAT is such an example: while it is NP-hard to approximate within factor $\nicefrac{{8}}{{7}}$, for any constant $\varepsilon>0$, there is an efficient, randomized algorithm which approximates the value within factor $\nicefrac{{8}}{{7}}+\varepsilon$. However, for many other optimization problems there are significant gaps between the approximation factors achievable by efficient algorithms and the approximation factors for which we can prove NP-hardness. We refer to this difference as the approximation gap. Another approach was proposed in [Kho02b], known as Unique Games Conjecture (UGC), for tightening of these approximation gaps for some optimization problems. Here the starting point is to assume NP-hardness of so-called Unique Games and deduce hardness of approximation. This has shown success in improving the hardness factor. Known NP-hardness of approximation for MAX-CUT is given in [Hås01] to be $\nicefrac{{17}}{{16}}$ (this has been improved in some special cases to $\nicefrac{{11}}{{8}}$ in [HHM*+*15]); these are not tight as the beautiful algorithm of Goemans-Williamson [GW95] achieves approximation factor $\approx 0.878$. However, by using UGC, tight approximation hardness was shown in [KKMO04], matching the approximation factor of the aforementioned algorithm. Also, for CSPs, optimal approximation hardness has been obtained in [Rag08]. Still there are optimization problems for which approximation gap remains. Two other notable assumptions that have been used for starting point of hardness reductions for better approximation hardness are: Feige’s R3SAT hypothesis proposed in [Fei02] and Exponential Time Hypothesis (ETH) proposed in [IP01]. We note here that Feige’s R3SAT hypothesis assumes the hardness of randomly generated instances – that is, average-case hardness as opposed to worst-case hardness.

We may consider another weakening of requirements for hardness results: instead of proving hardness for all efficient algorithms, we restrict ourselves to a large family of efficient algorithms. In other words, can we close the approximation gap for some class of algorithms $\mathcal{A}$ for some of these optimization problems? Note that such result would be unconditional. In particular, a tight result for an optimization problem $O$ will state that there is an algorithm $A\in\mathcal{A}$ which achieves $\alpha$-approximation whereas no algorithm in $\mathcal{A}$ can achieve approximation factor $\alpha-\varepsilon$ for (some, all) $\varepsilon>0$. Some reasonable classes of such algorithms are convex relaxation hierarchies.

As the name suggests, the convex relaxation hierarchies are sequences of successively stronger convex relaxations. We will consider two variants of these hierarchies: the LP relaxation hierarchy, a.k.a., the Sherali-Adams hierarchy and the SDP relaxation hierarchy, a.k.a., the Sum of Square ( SoS) hierarchy. The levels of these hierarchies are denoted $\textsf{SA}^{(d)}\!$ and $\textsf{SoS}^{(d)}\!$, respectively. For discrete optimization problems, these relaxations may be viewed as defining “local distributions”. We will skip the details here. Details of this relaxation for the discrete optimization problem of CSP may be found in Section˜2.2.2. Details for the $\textsf{SoS}^{(d)}$​ relaxation for a continuous optimization problem is given in Section˜3.3.1. The constraints of the relaxation $\textsf{SA}^{(d+1)}\!$ contains the constraints of $\textsf{SA}^{(d)}\!$ for any $d>0$. The same is true for the SoS relaxation as well. So the higher levels of these hierarchies provide tighter relaxations and hence provide a (possibly) better approximation factor. However, the downside is higher levels of the hierarchy require larger description sizes (which may, in turn, require more time to solve). These hierarchies form powerful classes of algorithms: most efficient algorithms constructed (with some known exceptions) are known to be captured by one of these hierarchies.

Unconditional hardness results for SA or SoS for an optimization problem thus reduce to demonstrating an integrality gap at certain levels of the hierarchy. That is, showing the existence of an instance $I$ of the optimization problem such that the optimal value for $I$ is $s$ and the $d$-level relaxation has value $c$. For maximization problems, this proves integrality gap factor $\nicefrac{{c}}{{s}}$; this is not quite the approximation factor. But this is used to show evidence of hardness approximation for these convex relaxation hierarchies.

As these hierarchies constitute some of the most powerful classes of algorithms, we can approach the problem of diminishing the approximation gap from the other end. That is, we may want to analyze the performance of these convex relaxations and their rounding to provide approximation algorithms for optimization problems. In [BKS17], the authors designed a new algorithm using SoS hierarchy to give an approximation factor which was previously unknown.

In this thesis, we establish unconditional hardness for the discrete optimization problem of CSP. For continuous optimization problems, we present an approximation algorithm for polynomial optimization problem using SoS hierarchy. For another, related, continuous optimization problem of matrix norm estimation, we show some new and improved conditional hardness of approximations. We also show almost tightness of our hardness by designing a convex relaxation based algorithm for some parameter regime. The rest of the document is divided in sections for these problems. Each section provides some details of our results and some natural questions that arise.

An interesting aspect of these problems and results is the role of geometry. In case of the matrix norm optimization problem, the connection to geometry of Banach spaces is direct and apparent. The proofs make essential use of embedding and projection results in Banach spaces, which, in retrospect, is not surprising. The connections to geometry in the cases of CSPs and polynomial optimization are more subtle and are described below.

For CSP, construction of the feasible solution in the soundness case requires building local distributions (described in Section˜2.2.2). Our approach constructs these local distributions while remaining oblivious to the parts of the instance outside the neighborhood of the variables of interest. Within this neighborhood, we partition the instance into forests and then randomly satisfy each tree to obtain a local distribution (see Section˜2.5). To ensure consistency of the local distributions, the partitioning procedure must itself be local and oblivious to “far variables” (even when the set under consideration may contain them). We achieve this by using an $\ell_{2}$-embeddable metric (from [CMM07a]) on the neighborhood and using a random unitary-invariant partition of the space to generate the forest decomposition. A crucial advantage of this process is that it allows us to “step out of the instance hypergraph” and partition it externally, as it resides on an $\ell_{2}$-ball. This external viewpoint ensures the required obliviousness, which is essential for proving local consistency of the distributions constructed. The same metric was also used more directly in [CMM09] to establish that the MAX-CUT problem has integrality gap $\nicefrac{{1}}{{2}}$ for polynomial levels of SA.

For polynomial optimization, the geometric connection arises in a subroutine of one of the works we extend. A common approach to polynomial optimization (over the sphere) is to decouple the variables, reducing the problem to multilinear polynomial optimization (where each multilinear component lies on the sphere). This is achieved by examining a monomial and replacing each occurrence of a variable with a fresh copy of a new variable (see Section˜3.2.1). This step introduces a multiplicative factor of roughly $d^{d}$ to the approximation ratio for a degree $d$ homogeneous polynomial. In [So11b], the authors then recursively approximate the multilinear optimization problem, with a crucial step in their approach relying on tools from geometry. In each recursive step of [So11b], the problem is reduced to approximating the diameter of a related convex body. By applying known tools from convex geometry, the authors solve the inductive step, leading to an overall approximation for the multilinear problem and, ultimately, for polynomial optimization. The connection to convex body estimation also appears in the works of [KN08].

Chapter 2 Lifting LP Lower Bounds for CSPs to Sherali-Adams

2.1 The Result in Context: Background and Overview

Given a finite alphabet $[q]=\left\{0,\ldots,q-1\right\}$ and a predicate $f:[q]^{k}\to\{0,1\}$, an instance of the problem $\textsf{MAX k-CSP}(f)$ consists of (say) $m$ constraints over a set of $n$ variables $x_{1},\ldots,x_{n}$ taking values in the set $[q]$. Each constraint $C_{i}$ is of the form $f(x_{i_{1}}+b_{i_{1}},\ldots,x_{i_{k}}+b_{i_{k}})$ for some $k$-tuple of variables $(x_{i_{1}},\ldots x_{i_{k}})$ and $b_{i_{1}},\ldots,b_{i_{q}}\in[q]$, and the addition is taken to be modulo $q$. We say an assignment $\sigma$ to the variables satisfying the constraint $C_{i}$ if $C_{i}(\sigma(x_{i_{1}}),\ldots,\sigma(x_{i_{k}}))=1$. Given an instance $\Phi$ of the problem, the goal is to find an assignment $\sigma$ to the variables satisfying as many constraints as possible. The approximability of the $\textsf{MAX k-CSP}(f)$ problem has been extensively studied for various predicates $f$ (see e.g., [sta07] for a survey), and special cases include several interesting and natural problems such as MAX 3-SAT, MAX 3-XOR and MAX-CUT.

A topic of much recent interest has been the efficacy of Linear Programming (LP) and Semidefinite Programming (SDP) relaxations. For a given instance $\Phi$ of $\textsf{MAX k-CSP}(f)$, let $\operatorname{\operatorname{OPT}}(\Phi)$ denote the fraction of constraints satisfied by an optimal assignment, and let $\operatorname{\operatorname{FRAC}}(\Phi)$ denote the value of the convex (LP/SDP) relaxation for the problem. Then, the performance guarantee of this algorithm is given by the integrality gap which equals the supremum of $\frac{\operatorname{\operatorname{FRAC}}(\Phi)}{\operatorname{\operatorname{OPT}}(\Phi)}$, over all instances $\Phi$.

The study of unconditional lower bounds for general families of LP relaxations was initiated by Arora, Bollobás and Lovász [ABL02] (see also [ABLT06]). They studied the Lovász-Schrijver [LS91] LP hierarchy and proved lower bounds on the integrality gap for Minimum Vertex Cover (their technique also yields similar bounds for MAX-CUT). De la Vega and Kenyon-Mathieu [dlVKM07] and Charikar, Makarychev and Makarychev [CMM09] proved a lower bound of $2-o(1)$ for the integrality gap of the LP relaxations for MAX-CUT given respectively by $\Omega(\log\log n)$ and $n^{\Omega(1)}$ levels of the Sherali-Adams LP hierarchy [SA90]. Several follow-up works have also shown lower bounds for various other special cases of the MAX k-CSP problem, both for LP and SDP hierarchies [AAT05, Sch08, Tul09, RS09a, BGMT12, BCK15, KMOW17a].

A recent result by Chan et al. [CLRS13] shows a connection between strong lower bounds for the Sherali-Adams hierarchy, and lower bounds on the size of LP extended formulations for the corresponding problem. In fact, their result proved a connection not only for a lower bound on the worst case integrality gap, but for the entire approximability curve. We say that $\Phi$ is $(c,s)$-integrality gap instance for a relaxation of $\textsf{MAX k-CSP}(f)$, if we have $\operatorname{\operatorname{FRAC}}(\Phi)\geq c$ and $\operatorname{\operatorname{OPT}}(\Phi)<s$. And we say that $\Phi$ is $(c,s)$-approximable by a relaxation of $\textsf{MAX k-CSP}(f)$, if for instances with $\operatorname{\operatorname{OPT}}(\phi)<s$, we have $\operatorname{\operatorname{FRAC}}(\Phi)\leq c$. They showed that for any fixed $t\in{\mathbb{N}}$, if there exist $(c,s)$-integrality gap instances of size $n$ for the relaxation given by $t$ levels of the Sherali-Adams hierarchy, then for all $\varepsilon>0$ and sufficiently large $N$, there exists a $(c-\varepsilon,s+\varepsilon)$ integrality gap instance of size (number of variables) $N$, for any linear extended formulation of size at most $N^{t/2}$. They also give a tradeoff when $t$ is a function of $n$. This was recently improved by Kothari et al. [KMR17] and we describe the improved tradeoff later.

We strengthen the above results by showing that for all $c,s\in[0,1]$, $(c,s)$-integrality gap instances for a “basic LP” can be used to construct $(c-\varepsilon,s+\varepsilon)$ integrality gap instances for $\Omega_{\varepsilon}\left(\frac{\log n}{\log\log n}\right)$ levels of the Sherali-Adams hierarchy. The basic LP uses only a subset of the constraints in the relaxation given by $k$ levels of the Sherali-Adams hierarchy for $\textsf{MAX k-CSP}(f)$. In particular, this shows that a lower bound on the integrality gap for even the basic LP, implies a similar lower bound on the integrality gap of any polynomial size extended formulation. This can also be viewed as a dichotomy result showing that for any predicate $f$, either $\textsf{MAX k-CSP}(f)$ is $(c,s)$-approximable by the basic LP relaxation (which is of size linear in the size of the instance) or for all $\varepsilon>0$, a $(c-\varepsilon,s+\varepsilon)$ cannot be achieved by any polynomial sized LP extended formulation. We note that both the above results and our result apply to all $f,q$ and all $c,s\in[0,1]$.

Comparison with (implications of) Raghavendra’s UGC hardness result.

A remarkable result by Raghavendra [Rag08] shows that a $(c,s)$-integrality gap instance for a “basic SDP” relaxation of $\textsf{MAX k-CSP}(f)$ implies hardness of distinguishing instances $\Phi$ with $\operatorname{\operatorname{OPT}}(\Phi)<s$ from instances with $\operatorname{\operatorname{OPT}}(\Phi)\geq c$, assuming the Unique Games Conjecture (UGC) of Khot [Kho02c]. The basic SDP considered by Raghavendra involves moments for all pairs of variables, and all subsets of variables included in a constraint. The basic LP we consider is weaker than this SDP and does not contain the positive semidefiniteness constraint.

Combining Raghavendra’s result with known constructions of integrality gaps for Unique Games by Raghavendra and Steurer [RS09a], and by Khot and Saket [KS09], one can obtain a result qualitatively similar to ours, for the mixed hierarchy. In particular, a $(c,s)$ integrality gap for the basic SDP implies a $(c-\varepsilon,s+\varepsilon)$ integrality gap for $\Omega((\log\log n)^{1/4})$ levels of the mixed hierarchy.

Note however, that the above result is incomparable to our result, since it starts with stronger hypothesis (a basic SDP gap) and yields a gap for the mixed hierarchy as opposed to the Sherali-Adams hierarchy. While the above can also be used to derive lower bounds for linear extended formulations, one needs to start with an SDP gap instance to derive an LP lower bound. The basic SDP is known to be provably stronger than the basic LP for several problems including various 2-CSPs. Also, for the worst case $f$ for $q=2$, the integrality gap of the basic SDP is $O(2^{k}/k)$ [CMM07b], while that of the basic LP is $2^{k-1}$.

A recent result by Khot and Saket [KS15] shows a connection between the integrality gaps for the basic LP and those for the basic SDP. They prove that, assuming the UGC, a $(c,s)$ integrality gap instance for the basic LP implies an NP-hardness of distinguishing instances $\Phi$ with $\operatorname{\operatorname{OPT}}(\Phi)\geq\Omega\left(\frac{c}{k^{3}\cdot\log(q)}\right)$ from instances with $\operatorname{\operatorname{OPT}}(\Phi)\leq 4s$. Their result also shows that a $(c,s)$ integrality gap instance for the basic LP can be used to produce a $\left(\Omega\left(\frac{c}{k^{3}\cdot\log(q)}\right),4s\right)$ integrality gap instance for the basic SDP, and hence for $\Omega({(\log\log n)}^{1/4})$ levels of the mixed hierarchy.

Other related work.

The power of the basic LP for solving valued CSPs to optimality has been studied in several previous works. These works consider the problem of minimizing the penalty for unsatisfied constraints, where the penalties take values in ${\mathbb{Q}}\cup\left\{\infty\right\}$. Also, they study the problem not only in terms of single predicate $f$, but rather in terms of the constraint language generated by a given set of (valued) predicates.

It was shown by Thapper and Živný [Tv13] that when the penalties are finite-valued, if the problem of finiding the optimum solution cannot be solved by the basic LP, then it is NP-hard. Kolmogorov, Thapper and Živný [KTv15] give a characterization of CSPs where the problem of minimizing the penalty for unsatisfied constraints can be solved exactly by the basic LP. Also, a recent result by Thapper and Živný [Tv16] shows the valued CSP problem for a constraint language can be solved to optimality by a bounded number of levels of the Sherali-Adams hierarchy if and only if it can be solved by a relaxation obtained by augmenting the basic LP with contraints implied by three levels of the Sherali-Adams hierarchy. However, the above works only consider the case when the LP gives an exact solution, and do not focus on approximation.

The techniques from [CMM09] used in our result, were also extended by Lee [Lee15] to prove a hardness for the Graph Pricing problem. Kenkre et al. [KPPS15] also applied these to show the optimality of a simple LP-based algorithm for Digraph Ordering.

Our results

Our main result is the following.

Theorem 2.1.1**.**

Let $f:[q]^{k}\to\{0,1\}$ be any predicate. Let $\Phi_{0}$ be a $(c,s)$ integrality gap instance for basic LP relaxation of MAX k-CSP ($f$). Then for every $\varepsilon>0$, there exists $c_{\varepsilon}>0$ such that for infinitely many $N\in{\mathbb{N}}$, there exist $(c-\varepsilon,s+\varepsilon)$ integrality gap instances of size $N$ for the LP relaxation given by $c_{\varepsilon}\cdot\frac{\log N}{\log\log N}$ levels of the Sherali-Adams hierarchy.

Combining the above with the connection between Sherali-Adams gaps and extended formulations by [CLRS13, KMR17] yields the following corollary:

Corollary 2.1.2**.**

Let $f:[q]\to\{0,1\}$ be any predicate. Let $\Phi_{0}$ be a $(c,s)$ integrality gap instance for basic LP relaxation of MAX k-CSP ($f$). Then for every $\varepsilon>0$, there exists $c^{\prime}_{\varepsilon}>0$ such that for infinitely $N\in{\mathbb{N}}$, there exist $(c-\varepsilon,s+\varepsilon)$ integrality gap instances of size $N$, for every linear extended formulation of size at most $N^{c^{\prime}_{\varepsilon}\cdot\frac{\log N}{\log\log N}}$.

As an application of our methods, we also simplify and strengthen the approximation resistance results for LPs proved by Khot et al. [KTW14]. They studied predicates $f:\{0,1\}^{k}\to\{0,1\}$ and provided a necessary and sufficient condition for the predicate to be strongly approximation resistant for the Sherali-Adams LP hierarchy. We say a predicate is strongly approximation resistant if for all $\varepsilon>0$, it is hard to distinguish instances $\Phi$ for which $\left\lvert\operatorname{\operatorname{OPT}}(\Phi)-\mathchoice{\underset{x\in\{0,1\}^{k}}{\mathbb{E}}\left[f(x)\right]}{{\mathbb{E}}_{x\in\{0,1\}^{k}}[f(x)]}{{\mathbb{E}}_{x\in\{0,1\}^{k}}[f(x)]}{{\mathbb{E}}_{x\in\{0,1\}^{k}}[f(x)]}\right\rvert\leq\varepsilon$ from instances with $\operatorname{\operatorname{OPT}}(\Phi)\geq 1-\varepsilon$. In the context of the Sherali-Adams hierarchy, they showed that when this condition is satisfied, there exist instances $\Phi$ satisfying $\left\lvert\operatorname{\operatorname{OPT}}(\Phi)-\mathchoice{\underset{x\in\{0,1\}^{k}}{\mathbb{E}}\left[f(x)\right]}{{\mathbb{E}}_{x\in\{0,1\}^{k}}[f(x)]}{{\mathbb{E}}_{x\in\{0,1\}^{k}}[f(x)]}{{\mathbb{E}}_{x\in\{0,1\}^{k}}[f(x)]}\right\rvert\leq\varepsilon$ and $\operatorname{\operatorname{FRAC}}(\Phi)\geq 1-\varepsilon$, where $\operatorname{\operatorname{FRAC}}(\Phi)$ is the value of the relaxation given by $O_{\varepsilon}(\log\log n)$ levels of the Sherali-Adams hierarchy. We strengthen their result (and provide a simpler proof) to prove the following.

Theorem 2.1.3**.**

Let $f:\{0,1\}^{k}\to\{0,1\}$ be any predicate satisfying the condition for strong approximation resistance for LPs, given by [KTW14]. Then for every $\varepsilon>0$, there exists $c_{\varepsilon}>0$ such that infinitely many $N\in{\mathbb{N}}$, there exists an instance $\Phi$ of $\textsf{MAX k-CSP}(f)$ of size $N$, satisfying

[TABLE]

where $\operatorname{\operatorname{FRAC}}(\Phi)$ is the value of the relaxation given by $c_{\varepsilon}\cdot\frac{\log N}{\log\log N}$ levels of the Sherali-Adams hierarchy.

As before, the above theorem also yields a corollary for extended formulations.

2.2 Preliminaries of CSPs and Their Relaxations

2.2.1 Constraint Satisfaction Problems

We use $[n]$ to denote the set $\left\{1,\ldots,n\right\}$. The only exception is $[q]$, where we overload this notation to denote the set $\left\{0,\ldots,q-1\right\}$, which corresponds to the alphabet for the Constraint Satisfaction Problem under consideration.

Definition 2.2.1**.**

Let $[q]$ denote the set $\left\{0,\ldots,q-1\right\}$. For a predicate $f:[q]^{k}\rightarrow\{0,1\}$, an instance $\Phi$ of MAX k-CSPq $(f)$ consists of a set of variables $\{x_{1},\ldots,x_{n}\}$ and a set of constraints $C_{1},\ldots,C_{m}$. Each constraint $C_{i}$ is over a $k$-tuple of variables $(x_{i_{1}},\ldots,x_{i_{k}})$ and is of the form

[TABLE]

for some $b_{i_{1}},\ldots,b_{i_{k}}\in[q]$, where the addition is modulo $q$. For an assignment $\sigma:\{x_{1},\ldots,x_{n}\}\mapsto[q]$, let ${\sf sat}(\sigma)$ denote the fraction of constraints satisfied by $\sigma$. The maximum fraction of constraints that can be simultaneously satisfied is denoted by $\operatorname{\operatorname{OPT}}(\Phi)$, i.e.

[TABLE]

For a constraint $C$ of the above form, we use $x_{C}$ to denote the tuple of variables $(x_{i_{1}},\ldots,x_{i_{k}})$ and $b_{C}$ to denote the tuple $(b_{i_{1}},\ldots,b_{i_{k}})$. We then write the constraint as $f(x_{C}+b_{C})$. We also denote by $S_{C}$ the set of indices $\{i_{1},\ldots,i_{k}\}$ of the variables participating in the constraint $C$.

2.2.2 The LP Relaxations for Constraint Satisfaction Problems

Below we present various LP relaxations for the MAX k-CSPq $(f)$ problem that are relevant in this chapter. Note that an integer solution to the problem can be given by an assignment $\sigma:[n]\to[q]$.

We start with the level-$t$ Sherali-Adams relaxation. The intuition behind it is the following: instead of looking for an assignment to satisfy maximum number of constraints, we look for distribution on assignments that maximize expected satisfied constraint. Note that this, in itself, does not relax the problem as we can sample an assignment form the optimal distribution to get an optimal assignment (the optimal distribution may be supported on one assignment). The relaxation happens by looking at the collection of “local distributions” with consistency constraints as described below. A feasible solution to our relaxation consists of local distributions for all sets $S\subseteq[n],1\leq\left\lvert S\right\rvert\leq t$, where $t$ is the level of Sherali-Adams relaxation. We define $\{0,1\}$-valued variables $x_{(S,\alpha)}$ for each $S\subseteq[n],1\leq|S|\leq t$ and $\alpha\in[q]^{S}$, to be the probability of $S$ getting assigned $\alpha$ in any local distribution of a feasible solution. In case of optimal distribution with unique support, we get $x_{(S,\alpha)}=1$ if $\sigma(S)=\alpha$ and 0 otherwise. We also introduce a variable $x_{(\emptyset,\emptyset)}$, which equals to 1. One of the consistency constraint is that the variables $\{x_{(S,\alpha)}\}_{\alpha\in[q]^{S}}$ give a probability distribution over assignments to $S$, we denote it by $\mathcal{D}_{S}$. A further consistency constraint between these local distributions by requiring that for $T\subseteq S$, the distribution over assignments to $S$, when marginalized to $T$, denoted as $\mathcal{D}_{S|T}$ is precisely the distribution over assignments to $T$ i.e., $\mathcal{D}_{S|T}=\mathcal{D}_{T}$. The relaxation is shown in Fig.˜2.1. Note that second and third constraint imply that for each local distribution, probabilities sum to $1$.

The basic LP relaxation is a reduced form of the above relaxation where only those variables $x_{(S,\alpha)}$ are included for which $S=S_{C}$ is the set of CSP variables for some constraint $C$. The consistency constraints are included only for singleton subsets of the sets $S_{C}$. Note that the all the constraints for the basic LP are implied by the relaxation obtained by level $k$ of the Sherali-Adams hierarchy.

For an LP/SDP relaxation of MAX k-CSPq, and for a given instance $\Phi$ of the problem, we denote by $\operatorname{\operatorname{FRAC}}(\Phi)$ the LP/SDP (fractional) optimum. A relaxation is said to have a $(c,s)$-integrality gap if there exists a CSP instance $\Phi$ such that $\operatorname{\operatorname{FRAC}}(\Phi)\geq c$ and $\operatorname{\operatorname{OPT}}(\Phi)<s$.

The main theorem we prove concerning the hardness of MAX k-CSPq in two different LP relaxations above: See 2.1.1

Proof Overview and Techniques

The Gap Instance.

The construction of our gap instances is inspired by the construction by Khot et al. [KTW14]. They gave a generic construction to prove integrality gaps for any approximation resistant predicate (starting from certificates of hardness in form of certain “vanishing measures”), and we use similar ideas to give a construction which can start from a basic LP integrality gap instance as a certificate, to produce a gap instance for many levels. This construction is discussed in Section˜2.6.

Given an integrality gap instance $\Phi_{0}$ on $n_{0}$ variables, we treat this as a “template” (as in Raghavendra [Rag08]) and generate a random instance using this. Concretely, we generate a new instance $\Phi$ on $n_{0}$ sets of $n$ variables each. To generate a constraint, we sample a random constraint $C_{0}\in\Phi_{0}$, and pick a variable randomly from each of the sets corresponding to variables in $C_{0}$. Thus, the instances generated are $n_{0}$-partite random hypergraphs, with each edge being generated according to a specified “type” (indices of sets to chose vertices from).

Note that previous instances of gap constructions for LP and SDP hierarchies were (hyper)graphs generated according to the model ${\cal G}_{n,p}$ with the signs of the literals chosen independently at random. However, proving an LP/SDP lower bound using such instances implies a strong result: it in fact proves that the predicate $f$ is useless for the corresponding relaxation, in the sense defined by [AH13]. Assuming the UGC, uselessness only holds for a limited class of predicates $f$ (when $f^{-1}(1)$ supports a balanced pairwise independent distribution on ${[q]}^{k}$), as shown in [AH13]. Thus, proving an SDP lower bound for predicates which are not expected to be useless requires a new construction of instances, which cannot be generated uniformly at random. Our construction provides such a generalization, and may be useful in proving new SDP lower bounds. The properties of random ${\cal G}_{n,p}$ hypergraphs easily carry over to our instances, and we collect these properties in Section˜2.3.1.

The above construction ensures that if the instance $\Phi_{0}$ does not have an assignment satisfying more than an $s$ fraction of the constraints, then $\operatorname{\operatorname{OPT}}(\Phi)\leq s+\varepsilon$ with high probability. Also, it is well-known that providing a good LP solution to the relaxation given by $t$ levels of the Sherali-Adams hierarchy is equivalent to providing distributions $\mathcal{D}_{S}$ on $[q]^{S}$ for all sets of variables $S$ with $\left\lvert S\right\rvert\leq t$, such that the distributions are consistent restricted to subsets i.e., for all $S$ with $\left\lvert S\right\rvert\leq t$ and all $T\subseteq S$, we have $\mathcal{D}_{S|T}=\mathcal{D}_{T}$. Thus, in our case, we need to produce such consistent local distributions such that the expected probability that a random constraint $C\in\Phi$ is satisfied by the local distribution on the set of variables involved in $C$ (which we denote as $S_{C}$) is at least $c-\varepsilon$.

Local Distributions from Local Structure.

Most works on integrality gaps for CSPs utilize the local structure of random hypergraphs to produce such distributions. Since the girth of a sparse random hypergraph is $\Omega(\log n)$, any induced subgraph on $o(\log n)$ vertices is simply a forest. In case the induced (hyper)graph $G_{S}$ on a set $S$ is a tree, there is an easy distribution to consider: simply choose an arbitrary root and propagate down the tree by sampling each child conditioned on its parent. It is also easy to see that for $T\subseteq S$, if the induced (hyper)graph $G_{T}$ is a subtree of $G_{S}$, then the distributions $\mathcal{D}_{S}$ and $\mathcal{D}_{T}$ produced as above are consistent.

The extension of this idea to forests requires some care. One can consider extending the distribution to forests by propagating independently on each tree in the forest. However, if for $T\subseteq S$ $G_{T}$ is a forest while $G_{S}$ is a tree, then a pair of vertices disconnected in $G_{T}$ will have no correlation in $\mathcal{D}_{T}$ but may be correlated in $\mathcal{D}_{S}$. This was handled, for example, in [KTW14] by adding noise to the propagation and using a large ball $B(S)$ around $S$ to define $\mathcal{D}_{S}$. Then, if two vertices of $T$ are disconnected in $B(T)$ but connected in $B(S)$, then they must be at a large distance from each other. Thus, because of the noise, the correlation between them (which is zero in $\mathcal{D}_{T}$) will be very small in $\mathcal{D}_{S}$. However, correcting approximate consistency to exact consistency incurs a cost which is exponential in the number of levels (i.e., the sizes of the sets), which is what limits the results in [KTW14, dlVKM07] to $O(\log\log n)$ levels. This also makes the proof more involved since it requires a careful control of the errors in consistency.

Consistent Partitioning Schemes.

We resolve the above consistency issue by first partitioning the given set $S$ into a set of clusters, each of which have diameter $\Delta_{H}=o(\log n)$ in the underlying hypergraph $H$. Since each cluster has bounded diameter, it becomes a tree when we add all the missing paths between any two vertices in the cluster. We then propagate independently on each cluster (augmented with the missing paths). This preserves the correlation between any two vertices in the same cluster, even if the path between them was not originally present in $G_{S}$.

Of course, the above plan requires that the partition obtained for $T\subseteq S$, is consistent with the restriction to $T$ of partition obtained for the set $S$. In fact, we construct distributions over partitions $\left\{\mathcal{P}_{S}\right\}_{\left\lvert S\right\rvert\leq t}$, which satisfy the consistency property $\mathcal{P}_{S|T}=\mathcal{P}_{T}$. These distributions over partitions, which we call consistent partitioning schemes, are constructed in Section˜2.5.

In addition to being consistent, we require that the partitioning scheme cuts only a few edges in expectation, since these contribute to a loss in the LP objective. We remark that such low-diameter decompositions (known as separating and padded decompositions) have been used extensively in the theory metric embeddings (see e.g., [KLMN05] and the references therein). The only additional requirement in our application is consistency.

We obtain the decompositions by proving the (easy) hypergraph extensions the results of Charikar, Makarychev and Makarychev [CMM07a], who exhibit a metric which is similar to the shortest path metric on graphs at small distances, and has the property that its restriction to any subset of size at most $n^{\varepsilon^{\prime}}$ (for an appropriate $\varepsilon^{\prime}<1$) is $\ell_{2}$ embeddable. This is proved in Section˜2.3.1. A variant of this metric was used by Charikar, Makarychev and Makarychev [CMM09] to prove lower bounds for MAX-CUT, for $n^{\varepsilon^{\prime}}$ levels of the Sherali-Adams hierarchy. They used the embedding to construct a “local SDP solution” for any $n^{\varepsilon^{\prime}}$ variables (with value $1-\varepsilon^{\prime}$) and produced the distributions required for Sherali-Adams by rounding the SDP solutions (which gives value $1-O(\sqrt{\varepsilon^{\prime}})$). However, rounding an SDP solution with a high value does not always produce a good integral solution for other CSPs.

Instead, we use these metrics in Section˜2.5 to construct the consistent partitioning schemes as described above, by applying a result of Charikar et al. [CCG*+*98] giving separating decompositions for finite subsets of $\ell_{2}$. We remark that it is the consistency requirement of the partitioning procedure that limits our results to $O\left(\frac{\log n}{\log\log n}\right)$ levels. The separation probability in the decomposition procedure grows with the dimension of the $\ell_{2}$ embedding, while (to the best of our knowledge) dimension reduction procedures seem to break consistency.

2.3 Hypergraphs

An instance $\Phi$ of MAX k-CSP defines a natural associated hypergraph $H=(V,E)$ with $V$ being the set of variables in $\Phi$ and $E$ containing one $k$-hyperedge for every constraint $C\in\Phi$. We remind the reader of the familiar notions of degree, paths, and cycles for the case of ($k$-uniform) hypergraphs:

Definition 2.3.1**.**

Let $H=(V,E)$ be a hypergraph.

For a vertex $v\in V$, the degree of the vertex $v$ is defined to be the number of distinct hyperedges containing it.

• -

A simple path $P$ is a finite alternate sequence of distinct vertices and distinct edges starting and ending at vertices, i.e., $P=v_{1},e_{1},v_{2},\ldots,v_{l},e_{l},v_{l+1}$, where $v_{i}\in V~\forall i\in[l+1]$ and $e_{i}\in E~\forall i\in[l]$. Furthermore, $e_{i}$ contains $v_{i},v_{i+1}$ for each $i$. Here $l$ is called the length of the path $P$. All paths discussed in this chapter will be simple paths.

• -

A sequence $\mathcal{C}=(v_{1},e_{1},v_{2},\ldots,v_{l},e_{l},v_{1})$ is called a cycle of length $l$ if the initial segment $v_{1},e_{1},\ldots,v_{l}$ is a (simple) path, $e_{l+1}\neq e_{i}$ for all $i\in[l]$, and $v_{1}\in e_{l}$. For a path $P$ (or cycle $\mathcal{C}$), we use $\operatorname{\operatorname{V}}(P)$ (or $\operatorname{\operatorname{V}}(\mathcal{C})$) to denote the set of all the vertices that occur in the edges, i.e., the set $\{v{\;\;:\;\;}(\exists i\in[h])(v\in e_{i})\}$, where $e_{1},\ldots,e_{h}$ are the hyperedges included in $P$ (or $\mathcal{C}$).

• -

For a given hypergraph $H$, the length of the smallest cycle in $H$ is called the girth of $H$.

To observe the difference the notions of cycle in graphs and hypergraphs, it is instructive to consider the following example: let $u,v$ be two distinct vertices in a $k$-uniform hypergraph for $k\geq 3$, and let $e_{1},e_{2}$ be two distinct hyperedges both containing $u$ and $v$. Then $u,e_{1},v,e_{2},u$ is a cycle of length $2$, which cannot occur in a graph.

We shall also need the following notion of the closure of a set $S\subseteq V$ in a given hypergraph $H$, defined by [CMM09] for the case of graphs. A stronger notion of closure was also considered by [BCK15].

Definition 2.3.2**.**

For a given hypergraph $H$ and $R\in{\mathbb{N}}$, and a set $S\subseteq\operatorname{\operatorname{V}}(H)$, we denote by $\operatorname{cl}_{R}(S)$ the $R$-closure* of $S$ obtained by adding all the vertices in all the paths of length at most $R$ connecting two vertices of $S$:*

[TABLE]

For ease of notation, we use $\operatorname{cl}(S)$ to denote $\operatorname{cl}_{1}(S)$.

2.3.1 Properties of Random Hypergraphs

In this section we collect various properties of the hypergraphs corresponding to our integrality gap instances. The gap instances we generate contain several disjoint collections of variables. Each constraint in the instance has a specified “type”, which specifies which of the collections each of the participating $k$ variables much be sampled from. The constraint is generated by randomly sampling each of the $k$ variables, from the collections specified by its type. This is captured by the generative model described below.

In the model below and in the construction of the gap instance, the parameter $n_{0}$ should be thought of as constant, while the parameters $n$ and $m$ should be though of a growing to infinity. We will choose $m=\gamma\cdot n$ for $\gamma=O_{k,q}(1)$.

Definition 2.3.3**.**

Let $n_{0},k\in{\mathbb{N}}$ with $k\geq 2$. Let $m,n>0$ and let $\Gamma$ be a distribution on $\left[n_{0}\right]^{k}$. We define a distribution ${\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$ on $k$-uniform $n_{0}$-partite hypergraphs with $m$ edges and $N=n_{0}\cdot n$ vertices, divided in $n_{0}$ sets $X_{1},\ldots,X_{n_{0}}$ of size $n$ each. A random hypergraph $H\sim{\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$ is generated by sampling $m$ random hyperedges independently as follows:

Sample a random type $(i_{1},\ldots,i_{k})\in[n_{0}]^{k}$ from the distribution $\Gamma$.

• -

For all $j\in[k]$, sample $v_{i_{j}}$ independently and uniformly in $X_{i_{j}}$.

• -

Add the edge $e_{i}=\left\{v_{i_{1}},\ldots,v_{i_{k}}\right\}$ to $H$.

Note that as specified above, the model may generate a multi-hypergraph. However, the number of such repeated edges is likely to be small, and we will bound these, and in fact the number of cycles of size $o(\log n)$ in ˜2.4.2.

We will study the following metrics (similar to the ones defined in [CMM07a]):

Definition 2.3.4**.**

Given a hypergraph $H$ with vertex set $V$, we define two metrics $d^{H}_{\mu}(\cdot,\cdot),\rho^{H}_{\mu}(\cdot,\cdot)$ on $V$ as

[TABLE]

for $u\neq v$, where $d_{H}(\cdot,\cdot)$ denotes the shortest path distance in $H$.

We primarily need the fact the local $\ell_{2}$-embeddability of the metric $\rho_{\mu}$. The following theorem captures various properties of random hypergraphs required for our construction. The proof of the theorem heavily uses results proved in [ABLT06] and [CMM09].

Theorem 2.3.5**.**

Let $H^{\prime}\sim{\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$ with $m=\gamma\cdot n$ edges and let $\varepsilon>0$. Then for large enough $n$, with high probability (at least $1-\varepsilon$, over the choice of $H^{\prime}$), there exists $\delta>0$, constant $c=c(k,\gamma,n_{0},\varepsilon)$, $\theta=\theta(k,\gamma,n_{0},\varepsilon)$ and a subhypergraph $H\subset H^{\prime}$ with $V(H)=V(H^{\prime})$ satisfying the following:

H has girth $\mathtt{g}\geq\delta\cdot\log n$.

• -

$\left\lvert\operatorname{\operatorname{E}}(H^{\prime})\setminus\operatorname{\operatorname{E}}(H)\right\rvert\leq\varepsilon\cdot m$.

• -

For all $t\leq n^{\theta}$, for $\mu\geq c\cdot\frac{\log t+\log\log n}{\log n}$, for all $S\subseteq\operatorname{\operatorname{V}}(H^{\prime})$ with $\left\lvert S\right\rvert\leq t$, the metric $\rho^{H}_{\mu}$ restricted to $S$ is isometrically embeddable into the unit sphere in $\ell_{2}$,

This theorem is proved in the next section. We note for the reader that ideas and methods involved in the proof of the above theorem are largely disconnected from those of the ˜2.1.1.

2.4 Local $\ell_{2}$-embeddability of the Metric

$\rho_{\mu}^{H}$

The goal of this section is to prove the following result about the local $\ell_{2}$-embeddability of the metric $\rho_{\mu}^{H}$. See 2.3.5 To prove the above theorem, we will use the local structure of random hypergraphs. We first prove that with high probability for random hypergraphs (sampled from ${\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$) a few hyperedges can be removed to obtain a hypergraph whose girth is $\Omega(\log n)$ and the degree is bounded. The following lemma shows a possible trade-off between the degree of the hypergraph vs the number of hyperedges required to be removed.

Lemma 2.4.1**.**

Let $H^{\prime}\sim{\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$ be a random hypergraph with $m=\gamma\cdot n$ hyperedges. Then for any $\varepsilon>0$, with probability $1-\varepsilon$, there exists a sub-hypergraph $H$ with $V(H)=V(H^{\prime})$ such that $\forall u\in V(H)$, $\deg_{H}(u)\leq 100\cdot\log\left(\frac{n_{0}}{\varepsilon}\right)\cdot k\cdot\gamma$ and $\left\lvert\operatorname{\operatorname{E}}(H^{\prime})\setminus\operatorname{\operatorname{E}}(H)\right\rvert\leq\varepsilon\cdot m$.

• **Proof:  ** By linearity of expectation, the expected degree of any vertex $v$ in $H^{\prime}$ is at most $k\cdot\gamma$. Let $D=100\cdot\log\left(\frac{n_{0}}{\varepsilon}\right)\cdot k\cdot\gamma$, and let $S$ be the set of all vertices $u$ with $\deg_{H^{\prime}}(u)>D$. Let $E_{S}$ be the set of all hyperedges with at least one vertex in $S$. We shall take $E(H)=E(H^{\prime})\setminus E_{S}$. Note that for any $u\in V(H^{\prime})$, $\mathchoice{{\mathbb{P}}\left[u\in S\right]}{{\mathbb{P}}[u\in S]}{{\mathbb{P}}[u\in S]}{{\mathbb{P}}[u\in S]}=\mathchoice{{\mathbb{P}}\left[\deg_{H^{\prime}}(u)\geq D\right]}{{\mathbb{P}}[\deg_{H^{\prime}}(u)\geq D]}{{\mathbb{P}}[\deg_{H^{\prime}}(u)\geq D]}{{\mathbb{P}}[\deg_{H^{\prime}}(u)\geq D]}\leq\exp(-D/4)$ by a Chernoff-Hoeffding bound. We use this to bound the expected number of edges deleted.

[TABLE]

The penultimate inequality uses the independence of the hyperedges in the generation process, which gives $\mathchoice{{\mathbb{E}}\left[\deg_{H^{\prime}}(u)\mid\deg_{H^{\prime}}(u)\geq D\right]}{{\mathbb{E}}[\deg_{H^{\prime}}(u)\mid\deg_{H^{\prime}}(u)\geq D]}{{\mathbb{E}}[\deg_{H^{\prime}}(u)\mid\deg_{H^{\prime}}(u)\geq D]}{{\mathbb{E}}[\deg_{H^{\prime}}(u)\mid\deg_{H^{\prime}}(u)\geq D]}~\leq~D+\mathchoice{{\mathbb{E}}\left[\deg_{H^{\prime}}(u)\right]}{{\mathbb{E}}[\deg_{H^{\prime}}(u)]}{{\mathbb{E}}[\deg_{H^{\prime}}(u)]}{{\mathbb{E}}[\deg_{H^{\prime}}(u)]}$. From our choice of the parameter $D$, we get that $\mathchoice{{\mathbb{E}}\left[E_{S}\right]}{{\mathbb{E}}[E_{S}]}{{\mathbb{E}}[E_{S}]}{{\mathbb{E}}[E_{S}]}\leq\varepsilon^{2}\cdot\gamma\cdot n=\varepsilon^{2}\cdot m$. Thus, the number of edges deleted is at most $\varepsilon\cdot m$ with probability at least $1-\varepsilon$.

The following lemma shows that the expected number of small cycles in random hypergraph is small.

Lemma 2.4.2**.**

Let $H\sim{\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$ be a random hypergraph and for $l\geq 2$, let $Z_{l}(H)$ denote the number of cycles of length at most $l$ in $H$. For $m,n$ and $k$ such that $k^{2}\cdot(m/n)>1$, we have

[TABLE]

• **Proof:  ** Let the vertices of $H$ correspond to the set $[n_{0}]\times[n]$. Suppose we contract the set of $[n_{0}]\times\{j\}$ vertices into a single vertex $j\in[n]$ to get a random multi-hypergraph $H^{\prime}$ on vertex set $[n]$. An equivalent way to view the sampling to $H^{\prime}$ is: for each $i\in[m]$, the $i$-th hyperedge $e_{i}$ of $H^{\prime}$ is sampled by independently sampling $k$ vertices (with replacement) uniformly at random from $[n]$. Note that the sampling of $H^{\prime}$ is independent of $\Gamma$ in the definition of ${\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$. Clearly, a cycle of length at most $l$ in $H$ produces a cycle of length at most $l$ in $H^{\prime}$. Hence, it suffices to bound the expected number of cycles in $H^{\prime}$

Given any pair $(u^{\prime},v^{\prime})$ of vertices of $H^{\prime}$, for $u^{\prime}\neq v^{\prime}$, the probability of the pair $(u^{\prime},v^{\prime})$ belonging together in some hyperedge of $H^{\prime}$ is at most $\frac{mk^{2}}{n^{2}}$. Consider a given $h$-tuple of vertices ${\bf u}=(u_{i_{1}},\cdots u_{i_{h}})$. Note that we require that hyperedges participating in a cycle be distinct. So, the probability that ${\bf u}$ is part of a cycle in $H^{\prime}$, i.e., there exists distinct hyperedges $e_{j}\in H^{\prime}$ for $j\in[h]$ such that $u_{i_{j}},u_{i_{j+1}}\in e_{j}$ for $j\in[h-1]$, and $u_{i_{1}},u_{i_{h}}\in e_{h}$ is at most $\left(\frac{mk^{2}}{n^{2}}\right)^{h}$. As a result, expected number of cycles of length $h$ in $H^{\prime}$ is bounded above by:

[TABLE]

From the geometric form of the bound, it follows that expected number of cycles of length at most $l$ in $H^{\prime}$ is at most $\frac{\left(k^{2}\cdot\frac{m}{n}\right)^{l+1}}{\left(k^{2}\cdot\frac{m}{n}\right)-1}<\left(k^{2}\cdot\frac{m}{n}\right)^{2l}\,.$

Using the above lemma, it is easy to show that one can remove all small cycles in a random hypergraph by deleting only a few hyperedges.

Corollary 2.4.3**.**

Let $H\sim{\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$ be a random hypergraph with $m=\gamma\cdot n$ for $\gamma>1$ and $k\geq 2$. Then, there exists $\delta=\delta(\gamma)>0$ such that with probability $1-n^{-1/6}$, all cycles of length at most $\delta\cdot\log n$ in $H$ can be removed by deleting at most $n^{2/3}$ hyperedges.

• **Proof:  ** As above, let $Z_{l}$ denote the number of cycles of length at most $l$. With the choice of $m,n,$ and $k$, we have $k^{2}\cdot\frac{m}{n}\geq 2$. By ˜2.4.2, $\mathchoice{{\mathbb{E}}\left[Z_{l}\right]}{{\mathbb{E}}[Z_{l}]}{{\mathbb{E}}[Z_{l}]}{{\mathbb{E}}[Z_{l}]}\leq\left(k^{2}\cdot\frac{m}{n}\right)^{2l}$. Since $m=\gamma\cdot n$, there exists a $\mathtt{g}=\delta\cdot\log n$ such that $\mathchoice{{\mathbb{E}}\left[Z_{l}\right]}{{\mathbb{E}}[Z_{l}]}{{\mathbb{E}}[Z_{l}]}{{\mathbb{E}}[Z_{l}]}\leq\sqrt{n}$. By Markov’s inequality, $\mathchoice{{\mathbb{P}}\left[Z_{l}\geq n^{2/3}\right]}{{\mathbb{P}}[Z_{l}\geq n^{2/3}]}{{\mathbb{P}}[Z_{l}\geq n^{2/3}]}{{\mathbb{P}}[Z_{l}\geq n^{2/3}]}\leq n^{-1/6}$. Thus, with probability $1-n^{-1/6}$, one can remove all cycles of length at most $\delta\cdot\log n$ by deleting at most $n^{2/3}$ edges.

One can also extend the analysis in [ABLT06] to show that the hypergraphs are locally sparse, i.e., the number of hyperedges contained in a small set of vertices is small. For a hypergraph $H$ and a set $S\subseteq\operatorname{\operatorname{V}}(H)$, we use $E(S)$ to denote the edges contained in the set $S$.

Definition 2.4.4**.**

We say that $S\subseteq V(H)$ is $\eta$-sparse if $\left\lvert E(S)\right\rvert\leq\frac{\left\lvert S\right\rvert}{k-1-\eta}$. We call a $k$-uniform hypergraph $H$ on $N$ vertices to be $(\tau,\eta)$-sparse* if all subsets $S\subset\operatorname{\operatorname{V}}(H),\left\lvert S\right\rvert\leq\tau\cdot\left\lvert\operatorname{\operatorname{V}}(H)\right\rvert$, $S$ is $\eta$-sparse. We call $H$ to be $\eta$-sparse if it is $(1,\eta)$-sparse, i.e., all subsets of vertices of $H$ are sparse.*

We note here that while this notion of sparsity is a generalization of that considered in [ABLT06], it is also identical to the notions of expansion considered in works in proof complexity (see e.g., [BS01]) and later in works on integrality gaps [AAT05, BGMT12, BCK15]. We prove that random hypergraphs generated with our model are locally sparse:

Lemma 2.4.5**.**

Let $\eta<1/4$ and $m=\gamma\cdot n$ for $\gamma>1$. Then for $\tau\leq\frac{1}{n_{0}}\cdot\left(\frac{1}{e\cdot k^{3k}\cdot\gamma}\right)^{1/\eta}$ the following holds:

[TABLE]

We note that we will require the sparsity $\eta$ to be $O_{k,\gamma}(1/\log n)$. This gives sparsity only for sublinear size sets, as compared to sets of size $\Omega(n)$ in previous works where $\eta$ is a constant. For the proof of the lemma, we follow an approach similar to that of ˜2.4.2: we collapse the vertices of $H$ of the form $[n_{0}]\times\{j\}$ to vertex $j\in[n]$ to construct $H^{\prime}$, and thus reducing the problem to random multi-hypergraph form a random multipartite hypergraph. The rest proof of the lemma is along the lines of several known proofs [AAT05, BGMT12].

• **Proof:  ** As in the proof of ˜2.4.2, given a random hypergraph $H$, we construct a hypergraph $H^{\prime}$ ( by contracting all the vertices in $[n_{0}]\times\{j\}$ to $j\in[n]$ ).

Consider a subset of vertices $S\subseteq\operatorname{\operatorname{V}}(H)$ and let $S^{\prime}\subseteq\operatorname{\operatorname{V}}(H^{\prime})$ be the corresponding contracted set in $H^{\prime}$. Since each edge in $H$ corresponds to an edge in $H^{\prime}$ (counting multiplicities), we have

[TABLE]

Thus, it suffices to show that $H^{\prime}$ is $(\tau^{\prime},\eta)$-sparse for $\tau^{\prime}=\tau\cdot n_{0}$, since $\left\lvert S^{\prime}\right\rvert\leq\tau\cdot N=(\tau\cdot n_{0})\cdot n$. Given any multiset in $[n]^{k}$, the probability that it corresponds to an edge in $H^{\prime}$ is at most $(k!)\cdot(m/n^{k})$. Thus, the probability that there exists a set $T$ of size at most $\tau^{\prime}\cdot n$, containing at least $\left\lvert T\right\rvert/(k-1-\eta)$ edges (counting multiplicities) is at most

[TABLE]

where $r=\frac{h}{k-1-\eta}$. Note that we also need to consider $h=1$ as edges in $H^{\prime}$ correspond to multisets of size $k$, and so may not have all distinct vertices. Simplifying the above using $\binom{a}{b}\leq\left(\frac{a\cdot e}{b}\right)^{b}$ and $k!\leq k^{k}$ gives

[TABLE]

Let $\theta=\eta/(2k)$. We divide the above summation in two parts and first consider

[TABLE]

for $\tau^{\prime}\leq\left(e\cdot k^{3k}\cdot\gamma\right)^{-1/\eta}$. Considering the first half of the summation, we get

[TABLE]

Combining the two bounds gives that the probability is at most  $3k^{3}\cdot\gamma^{1/k}\cdot n^{-\theta/2}$, which equals the desired bound.

Charikar et al. [CMM07a] prove an analogue of ˜2.3.5 for metrics defined on locally-sparse graphs. In fact, they use a consequence of sparsity, which they call $\ell$-path decomposability. To this end, we define the incidence graph111This is the same notion as the constraint-variable graph considered in various works on lower bounds for CSPs. associated with a hypergraph, on which we will apply their result.

Definition 2.4.6**.**

Let $H=(V(H),E(H))$ be a $k$-uniform hypergraph. We define its incidence graph as the bipartite graph $G_{H}$ defined on vertex sets $V(H)$ and $E(H)$, and edge set $\mathcal{E}$ defined as

[TABLE]

Note that for any $u,v\in V(H)$, we have $d_{G_{H}}(u,v)=2\cdot d_{H}(u,v)$. We prove that for a locally sparse hypergraph $H$, its incidence graph $G_{H}$ is also locally sparse.

Lemma 2.4.7**.**

Let $H$ be a $k$-uniform $(\tau,\eta)$-sparse hypergraph on $N$ vertices with $m=\gamma\cdot n$ hyperedges. Then the incidence graph $G_{H}$ is $(\tau^{\prime},\eta^{\prime})$ sparse for $\tau^{\prime}=\nicefrac{{\tau}}{{k\cdot(1+\gamma)}}$ and $\eta^{\prime}=\nicefrac{{\eta}}{{(1+\eta)}}$.

• **Proof:  ** Let $\tau^{\prime}=\nicefrac{{\tau}}{{k\cdot(1+\gamma)}}$ and let $G_{H}$ be the incidence graph with $N+m=(1+\gamma)\cdot N$ vertices. Let $G^{\prime}$ be is the densest subgraph of $G_{H}$, among all subgraphs of size at most $\tau^{\prime}\cdot(N+m)$. Let the vertex set of $G^{\prime}$ be $V^{\prime}\cup E^{\prime}$ where $V^{\prime}\subseteq V(H)$ and $E^{\prime}\subseteq E(H)$, and let the edge-set be $\mathcal{E}^{\prime}$. There cannot be any isolated vertices in $G^{\prime}$ since removing those will only increase the density.

Let $S\subseteq V(H)$ be the set of all vertices contained in all edges in $E^{\prime}$ i.e., $S~\stackrel{{\scriptstyle\mathrm{def}}}{{=}}~\left\{v\in V(H)~\mid~\exists e\in E^{\prime}~\text{s.t.}~v\in e\right\}$. Note that $V^{\prime}\subseteq S$, since there are no isolated vertices, and $E^{\prime}\subseteq E(S)$, where $E(S)$ denotes the set of hyperedges contained in $S$.

By our choice of parameters, $\left\lvert S\right\rvert\leq k\cdot\left\lvert E^{\prime}\right\rvert\leq k\cdot\tau^{\prime}\cdot(N+m)\leq\tau\cdot N$. Thus, using the sparsity of $H$, we have

[TABLE]

Also, since each hyperedge of $E^{\prime}$ can include at most $k$ vertices in $S$, and since each edge in $\mathcal{E}^{\prime}$ is incident on a vertex in $V^{\prime}$, we have

[TABLE]

Combining the two inequalities gives

[TABLE]

Hence, we get that $\left\lvert\mathcal{E}^{\prime}\right\rvert\leq\frac{\left\lvert V^{\prime}\right\rvert+\left\lvert E^{\prime}\right\rvert}{(1-\eta^{\prime})}$ for $\eta^{\prime}=\frac{\eta}{(1+\eta)}$.

Charikar et al. [CMM07a] defined the following structural property of a graph.

Definition 2.4.8** ([CMM07a]).**

A graph $G$ is $\ell$-path decomposable* if every $2$-connected subgraph $G^{\prime}$ of $G$, such that $G^{\prime}$ is not an edge, contains a path of length $\ell$ such that every vertex of the path has degree at most $2$ in $G^{\prime}$.*

The above property was also implicitly used by Arora et al. [ABLT06], who proved the following (see Lemma 2.12 in [ABLT06]):

Lemma 2.4.9**.**

Let $\ell>0$ be an integer and $0<\eta<\frac{1}{3\ell-1}<1$. Let $G$ be a $\eta$-sparse graph with girth $\mathtt{g}>\ell$. Then $G$ is $\ell$-path decomposable.

Recall that we defined the metrics $d_{\mu}$ and $\rho_{\mu}$ on $H$ as (for $u\neq v$) :

[TABLE]

For a graph $G$, we define the following two metrics, for $u\neq v$:

[TABLE]

We note that if $H$ is a hypergraph and $G_{H}$ is its incidence graph, then the metrics $d_{\mu}^{G_{H}}$ and $\rho_{\mu}^{G_{H}}$ restricted to $V(H)$, coincide with the metrics $d_{\mu}$ and $\rho_{\mu}$ defined on $H$. Charikar et al. proved the following theorem (see Theorem $5.2$) in [CMM09].

Theorem 2.4.10** ([CMM09]).**

Let $G$ be a graph on $n^{\prime}$ vertices with maximum degree $D$. Let $t<\sqrt{n^{\prime}}$ and $\ell>0$ be such that for $t^{\prime}=D^{\ell+1}\cdot t$, every subgraph of $G$ on at most $t^{\prime}$ vertices is $\ell$-path decomposable. Also, let $\mu$, $t$ and $\ell$ satisfy the relation ${(1-\mu)}^{\ell/9}\leq\frac{\mu}{2(t+1)}$. Then for every subset $S$ of at most $t$ vertices there exists a mapping $\psi_{S}$ from $S$ to unit sphere in $\ell_{2}$ such that all $u,v\in S$:

[TABLE]

We use this theorem to prove the main theorem of the section.

• **Proof of ˜2.3.5:  ** Let $H^{\prime}\sim{\cal H}_{k}\left(m,n,n_{0},\Gamma\right)$ with $m=\gamma\cdot n$ hyperedges and $N=n_{0}\cdot n$ vertices. Given $\varepsilon>0$, from ˜2.4.1 we have that with high probability at least $1-\varepsilon/2$, there exists $H_{1}$ such that the maximum degree of $H_{1}$ is at most $D=100\cdot\log\left(\frac{2n_{0}}{\varepsilon}\right)\cdot k\cdot\gamma$ with $\left\lvert\operatorname{\operatorname{E}}(H^{\prime})\setminus\operatorname{\operatorname{E}}(H_{1})\right\rvert\leq(\varepsilon/2)\cdot m$.

Using Corollary˜2.4.3 we also have that there exists $\delta>0$, such that with probability at least $1-\varepsilon/4$ (for large enough $n$) $H^{\prime}$ has a sub-hypergraph $H_{2}$ with $\mathtt{g}\geq\delta\cdot\log n$ and $\left\lvert\operatorname{\operatorname{E}}(H^{\prime})\setminus\operatorname{\operatorname{E}}(H_{2})\right\rvert\leq(\varepsilon/4)\cdot m$. By ˜2.4.5, there exists $\eta=\Omega_{n_{0},k,\gamma,\varepsilon}(1/(\log n))$ such that $H^{\prime}$ is $(\tau,\eta)$-sparse with probability at least $1-\varepsilon/4$, for $\tau\geq n^{-1/4}$.

Hence with probability $1-\varepsilon$, we have that $H=(V(H^{\prime}),E(H_{1})\cap E(H_{2}))$ satisfies:

• –

Degree of $H$ is bounded above by $D$.

• –

$H$ is $(\tau,\eta)$-sparse (for $\tau\geq n^{-1/4}$ and $\eta=\Omega_{n_{0},k,\gamma,\varepsilon}(1/(\log n)$).

• –

Girth of $H$ is at least $\mathtt{g}>\delta\cdot\log n$.

• –

$\left\lvert\operatorname{\operatorname{E}}(H^{\prime})\setminus\operatorname{\operatorname{E}}(H)\right\rvert\leq\varepsilon\cdot m$.

We now show that the metric $\rho^{H}_{\mu}$ is locally $\ell_{2}$ embeddable.

Let $G=G_{H}$ be the incidence graph for the hypergraph $H$. Note that $N\leq\left\lvert\operatorname{\operatorname{V}}(G)\right\rvert\leq N\cdot(1+\gamma)$ and degree of $G$ is also bounded by $D$. Since a cycle in $G$ is also a cycle in $H$, the girth of $G$ is at also least $\mathtt{g}\geq\delta\cdot\log n$.

By ˜2.4.7, we have $G$ is $(\frac{\tau}{k(1+\gamma)},\frac{\eta}{1+\eta})$-sparse. By Lemma˜2.4.9, any subgraph of $G$ on at most $\frac{\tau}{k(1+\gamma)}\cdot(N+m)$ vertices is $\ell$-path decomposable for any $\ell\leq\min\{\mathtt{g},1/(4\eta)\}$. Since $D=100\cdot k\gamma\cdot\log(2n_{0}/\varepsilon)$, there exists $\ell_{0}=\Omega_{k,\gamma,n_{0},\varepsilon}(\log n)$ such that $D^{\ell_{0}+1}\leq n^{1/6}$. We choose $\ell=\min\left\{\mathtt{g},1/(4\eta),\ell_{0}\right\}$.

Let $\mu_{0}$ be the smallest $\mu$ such that $\exp\left(-\mu\ell/9\right)\leq\frac{\mu}{2(t+1)}$ (note that $\frac{1}{\mu}\cdot\exp\left(-\mu\ell/9\right)$ is decreasing in $\mu$). Since we must have $\mu\geq 1/\ell$, there exists a $\mu_{0}$ satisfying

[TABLE]

From our choice of $\ell$, there exist constants $c=c(k,\gamma,n_{0},\varepsilon)$ and $\theta=\theta(k,\gamma,n_{0},\varepsilon)<1/2$ such that $\mu_{0}\leq c\cdot\frac{\log t+\log\log n}{\log n}<1$ when $t\leq n^{\theta}$. Then, for any $\mu\in[\mu_{0},1)$, we have $(1-\mu)^{\ell/9}\leq\exp(-\mu\ell/9)\leq\frac{\mu}{2(t+1)}$.

We can now apply Theorem˜2.4.10 to construct the embedding. Given any subset $S$ of $\operatorname{\operatorname{V}}(H)$ of size at most $t\leq n^{\theta}$, note that $S$ is also a subset of $\operatorname{\operatorname{V}}(G)$. Moreover, we have $t\leq n^{\theta}\leq(N+m)^{1/2}$. Also, we have $t\cdot D^{\ell+1}\leq n^{1/2}\cdot n^{1/6}=n^{2/3}\leq\frac{\tau}{k(\gamma+1)}\cdot(N+m)$. Thus, any subgraph of $G$ on $t\cdot D^{\ell+1}$ vertices is $\ell$-path decomposable.

Thus, when $\mu\geq\mu_{0}$, by Theorem˜2.4.10 there exists a mapping $\psi_{S}$ from $S$ to the unit sphere, such that for all $u,v\in S$, we have

[TABLE]

where the last equality uses the fact that for all $u,v\in\operatorname{\operatorname{V}}(H)$, $\rho^{H}_{\mu}(u,v)=\rho^{G}_{\mu}(u,v)$ since $d_{G}(u,v)=2\cdot d_{H}(u,v)$.

2.5 Decompositions of Hypergraphs from Local Geometry

We will construct the Sherali-Adams solution by partitioning the given subset of vertices in to trees, and then creating a natural distribution over satisfying assignments on trees. We define below the kind of partitions we need.

Definition 2.5.1**.**

Let $X$ be a finite set. For a set $S$, let $\mathcal{P}_{S}$ denote a distribution over partitions of $S$. For $T\subseteq S$, let $\mathcal{P}_{S|T}$ be the distribution over partitions of $T$ obtained by restricting the partitions in $\mathcal{P}_{S}$ to the set $T$. We say that a collection of distributions $\left\{\mathcal{P}_{S}\right\}_{\left\lvert S\right\rvert\leq t}$ forms a consistent partitioning scheme of order $t$, if

[TABLE]

In addition to being consistent as described above, we also require the distributions to have small probability of cutting the hyperedges for the hypergraphs corresponding to our CSP instances. We define this property below.

Definition 2.5.2**.**

Let $H=(V,E)$ be a $k$-uniform hypergraph. Let $\left\{\mathcal{P}_{S}\right\}_{\left\lvert S\right\rvert\leq t}$ be a consistent partitioning scheme of order $t$ for the vertex set $V$, with $t\geq k$. We say the scheme $\left\{\mathcal{P}_{S}\right\}_{\left\lvert S\right\rvert\leq t}$ is $\varepsilon$-sparse for $H$ if

[TABLE]

In this section, we will prove that the hypergraphs arising from random CSP instances admit sparse and consistent partitioning schemes. Recall that for a hypergraph $H$, we define (Definition˜2.3.4) the metrics $d^{H}_{\mu}$ and $\rho^{H}_{\mu}$ as:

[TABLE]

Lemma 2.5.3**.**

Let $H=(V,E)$ be $k$-uniform hypergraph and let $d_{\mu}$ be the metric as defined above. Let $H$ be such that for all sets $S\subseteq V$ with $\left\lvert S\right\rvert\leq t$, the metric induced on $\rho_{\mu}$ on $S$ is isometrically embeddable into $\ell_{2}$. Then, there exists $\varepsilon\leq 10k\cdot\sqrt{\mu\cdot t}$ and $\Delta_{H}=O(1/\mu)$ such that $H$ admits an $\varepsilon$-sparse consistent partitioning scheme of order $t$, with each partition consisting of clusters of diameter at most $\Delta_{H}$ in $H$.

We use the following result of Charikar et al. [CCG*+*98] which shows that low-dimensional metrics have good separating decompositions with bounded diameter, i.e., decompositions which have a small probability of separating points at a small distance.

Theorem 2.5.4** ([CCG*+*98]).**

Let $W$ be a finite collection of points in ${\mathbb{R}}^{d}$ and let $\Delta>0$ be given. Then there exists a distribution $\mathcal{P}$ over partitions of $W$ such that

$\forall P\in Supp(\mathcal{P})$, each cluster in $P$ has $\ell_{2}$ diameter at most $\Delta$.

• -

For all $x,y\in W$

[TABLE]

We also need the observation that the partitions produced by the above theorem are consistent, assuming the set $S$ considered above lie in a fixed bounded set (using a trivial modification of the procedure in [CCG*+*98]). For the sequel, we use $B(x,\delta)$ to denote the $\ell_{2}$ ball around $x$ of radius $\delta$ and $B_{H}(u,r)$ to denote a ball of radius $r$ around a vertex $u\in V(H)$. Thus,

[TABLE]

The balls $B(S,\delta)$ and $B_{H}(S,r)$ are defined similarly.

Claim 2.5.5**.**

Let $S$ and $T$ be sets such that $T\subseteq S$. Let $W_{S}=\left\{w_{u}\right\}_{u\in S}$ and $W_{T}=\left\{w_{u}^{\prime}\right\}_{u\in T}$ be $\ell_{2}$-embeddings of $S$ and $T$ satisfying $\phi(W_{T})\subseteq W_{S}\subseteq B(0,R_{0})\subset{\mathbb{R}}^{d}$, for some unitary transformation $\phi$ and $R_{0}>0$. Let $\mathcal{P}_{S}$ and $\mathcal{P}_{T}$ be distributions over partitions of $S$ and $T$ respectively, induced by partitions on $W_{S}$ and $W_{T}$ as given by Theorem˜2.5.4. Then

[TABLE]

• **Proof:  ** The claim follows simply by considering (a trivial modification of) the algorithm of [CCG*+*98]. For a given set $W$ and a parameter $\Delta$, they produce a partition using the following procedure:

• –

Let $W^{\prime}=W$.

• –

Repeat until $W^{\prime}=\emptyset$

Pick a random point $x$ in $B(W,\Delta/2)$ according to the Haar measure. Let $C_{x}=B(x,\Delta/2)\cap W^{\prime}$.

• *

If $C_{x}\neq\emptyset$, set $W^{\prime}=W^{\prime}\setminus C_{x}$. Output $C_{x}$ as a cluster in the partition.

The work [CCG*+*98] shows that the above procedure produces a distribution over partitions satisfying the conditions in Theorem˜2.5.4. We simply modify the procedure to sample a random point $x$ in $B(0,R_{0}+\Delta/2)$ instead of $B(S,\Delta/2)$. This does not affect the separation probability of any two points, since the only non-empty clusters are still produced by the points in $B(S,\Delta/2)$.

Let $P$ be a partition of $S$ produced by the above procedure when applied to the point set $W_{S}$, and let $P^{\prime}$ be a random partition produced when applied to the point set $\phi(W_{T})$. It is easy to see from the above procedure that the distribution $\mathcal{P}_{T}$ is invariant under a unitary transformation of $W_{T}$. By coupling the random choice of a point in $B(0,R_{0}+\Delta/2)$ chosen at each step in the procedures applied to $W_{S}$ and $\phi(W_{T})\subseteq W_{S}$, we get that $P(T)=P^{\prime}$ i.e., the partition $P$ restricted to $T$ equals $P^{\prime}$. Thus, we get $\mathcal{P}_{S|T}=\mathcal{P}_{T}$.

We can use the above to prove Lemma˜2.5.3.

• **Proof of Lemma˜2.5.3:  ** Given a set $S$, let $W_{S}$ be an $\ell_{2}$ embedding of the metric $\rho_{\mu}$ restricted to $S$. Since, $|S|\leq t$, we can assume $W_{S}\in{\mathbb{R}}^{t}$. We apply partitioning procedure of Charikar et al. from Theorem˜2.5.4 with $\Delta=1/2$. From the definition of the metric $\rho^{H}_{\mu}$, we get that there exists a $\Delta_{H}=O(1/\mu)$ such that $\rho^{H}_{u,v}\leq 1/2~\implies~d_{H}(u,v)\leq\Delta_{H}$. Moreover, for $u,v$ contained in an edge $e$, we have that $\rho_{\mu}(u,v)\leq\sqrt{5\mu}$ and hence the probability that $u$ and $v$ are separated is at most $10\sqrt{\mu\cdot t}$. Thus, the probability that any vertex in $e$ is separated from $u$ is at most $10k\cdot\sqrt{\mu\cdot t}$.

Finally, for any $S\subseteq T$, if $W_{S}$ and $W_{T}$ denote the corresponding $\ell_{2}$ embeddings, by the rigidity of $\ell_{2}$ we have that for $\phi(W_{T})\subseteq W_{S}$ for some unitary transformation $\phi$. Thus, by ˜2.5.5, we get that this is a consistent partitioning scheme of order $t$.

2.6 The Sherali-Adams Integrality Gaps Construction

2.6.1 Integrality Gaps from the Basic LP

Recall that the basic LP relaxation for MAX k-CSPq ($f$) as given in Fig.˜2.2. In this section, we will prove˜2.1.1. We recall the statement below. See 2.1.1 Let $\Phi_{0}$ be a $(c,s)$ integrality gap instance for the basic LP relaxation for MAX k-CSPq ($f$) with $n_{0}$ variables and $m_{0}$ constraints. We use it to construct a new integrality gap instance $\Phi$. The construction is similar to the gap instances constructed by Khot et al. [KTW14] discussed in the next section. However, we describe this construction first since it’s simpler. The procedure for constructing the instance $\Phi$ is described in Fig.˜2.3.

Soundness

We first prove that no assignment satisfies more than $s+\varepsilon$ fraction of constraints for the above instance.

Lemma 2.6.1**.**

For every $\varepsilon>0$, there exists $\gamma=\gamma(\varepsilon,n_{0},q)$ such that for an instance $\Phi$ generated by choosing at least $\gamma\cdot n$ constraints independently at random as above, we have with probability $1-\exp\left(-\Omega(n)\right)$, $\operatorname{\operatorname{OPT}}(\Phi)<s+\varepsilon$.

• **Proof:  ** Fix an assignment $\sigma\in[q]^{N}$. We will first consider $\mathchoice{{\mathbb{E}}\left[{\sf sat}_{\Phi}\left(\sigma\right)\right]}{{\mathbb{E}}[{\sf sat}_{\Phi}\left(\sigma\right)]}{{\mathbb{E}}[{\sf sat}_{\Phi}\left(\sigma\right)]}{{\mathbb{E}}[{\sf sat}_{\Phi}\left(\sigma\right)]}$ for a randomly generated $\Phi$ as above.

[TABLE]

where for each $i\in[n_{0}]$, $Z_{i}$ is an independent random variable with the distribution

[TABLE]

and $Z_{C_{0}}$ denotes the collection of variables in the constraint $C_{0}$, i.e., $Z_{C_{0}}=\left\{Z_{i}\right\}_{i\in S_{C_{0}}}$. Thus, the random variables $Z_{1},\ldots,Z_{n_{0}}$ define a random assignment to the variables in $\Phi_{0}$, which gives, for any $\sigma$

[TABLE]

Consider a randomly added constraint $C$ to the instance $\Phi$. We have that

[TABLE]

for any fixed $\sigma$ over random choice of the constraint $C$. Thus, for an instance $\Phi$ with $m$ independently and randomly generated constraints, we have

[TABLE]

Taking a union bound over all assignments, we get

[TABLE]

which is at most $\exp\left(-\Omega(n)\right)$ for $m=O(((\log q)/\varepsilon^{2})\cdot n\cdot n_{0})$.

Completeness

To prove the completeness, we first observe that the instance $\Phi$ as constructed above is also a gap instance for the basic LP. We will then “boost” this hardness to many levels of the Sherali-Adams hierarchy.

Lemma 2.6.2**.**

For every $\varepsilon>0$, there exists $\gamma=\gamma(\varepsilon)$ such that for an instance $\Phi$ generated by choosing at least $\gamma\cdot n$ constraints independently at random as above, with probability $1-\exp\left(-\Omega(n)\right)$ there exist distributions $\overline{\cal D}_{S_{C}}$ over $[q]^{S_{C}}$ for each $C\in\Phi$, and distributions $\overline{\cal D}_{i}$ over $[q]$ for each variable $x_{i}\in[n\cdot n_{0}]$, satisfying

For all $C\in\Phi$ and all $i\in S_{C}$, $\overline{\cal D}_{S_{C}|\{i\}}=\mathcal{D}_{i}$.

• -

The distributions satisfy $\operatorname*{\mathbb{E}}_{C\in\Phi}\mathchoice{\underset{\alpha\sim\overline{\cal D}_{S_{C}}}{\mathbb{E}}\left[f(\alpha+b_{C})\right]}{{\mathbb{E}}_{\alpha\sim\overline{\cal D}_{S_{C}}}[f(\alpha+b_{C})]}{{\mathbb{E}}_{\alpha\sim\overline{\cal D}_{S_{C}}}[f(\alpha+b_{C})]}{{\mathbb{E}}_{\alpha\sim\overline{\cal D}_{S_{C}}}[f(\alpha+b_{C})]}~\geq~c-\frac{\varepsilon}{10}$.

• **Proof:  ** For each $C_{0}\in\Phi_{0}$ and each $j\in[n_{0}]$, let $\mathcal{D}^{(0)}_{S_{C_{0}}}$ and $\mathcal{D}^{(0)}_{j}$ denote the basic LP solution satisfying

[TABLE]

Each constraint $C\in\Phi$ is sampled according to some constraint $C_{0}\in\Phi_{0}$, and we take $\overline{\cal D}_{S_{C}}\stackrel{{\scriptstyle\mathrm{def}}}{{=}}\mathcal{D}^{(0)}_{S_{C_{0}}}$ for the corresponding contraint $C_{0}\in\Phi_{0}$. Also, each variable $x_{i}$ for $i\in[n_{0}\cdot n]$, belongs to one of the sets $X_{j}$ for $j\in[n_{0}]$, and we take $\overline{\cal D}_{i}\stackrel{{\scriptstyle\mathrm{def}}}{{=}}\mathcal{D}^{(0)}_{j}$ for the corresponding $j\in[n_{0}]$.

The consistency of the distributions follows immediately from the construction of the instance $\Phi$. Let $C\in\Phi$ be any constraint and let $C_{0}$ be the corresponding constraint in $\Phi_{0}$. If $S_{C_{0}}=(j_{1},\ldots,j_{k})$, then $S_{C}=(i_{1},\ldots,i_{k})$ where each $i_{r}\in\{j_{r}\}\times[n]$ for all $r\in[k]$. Thus, for any $r\in[k]$,

[TABLE]

To bound the objective value, we again consider its expectation over a randomly generated instance $\Phi$. Let $C$ be a random constraint added to $\Phi$. Then, if we define $\overline{\cal D}_{S_{C}}$ as above for this constraint, we have

[TABLE]

Thus, the expected contribution of each constraint is at least $c$. The probability that the average of $m$ constraints deviates by at least $\varepsilon/10$ from the expectation, is at most $\exp\left(-\Omega(\varepsilon^{2}\cdot m)\right)$. There exists $\gamma=O(1/\varepsilon^{2})$ such that for $m\geq\gamma\cdot n$, the probability is at most $\exp(-\Omega(n))$.

To construct local distributions for the Sherali-Adams hierarchy, we will consider (a slight modification) the hypergraph $H$ corresponding to the instance $\Phi$. We first show that distributions on hyperedges of this hypergraph can be consistently propagated in a tree, provided they agree on intersecting vertices.

For a set $U\subseteq\operatorname{\operatorname{V}}(H)$ in a hypergraph $H$, recall that $\operatorname{cl}(U)$ includes all paths of lengths at most 1 between any two vertices in $U$. Thus, $E(\operatorname{cl}(U))=\left\{e\in E~\mid~\left\lvert e\cap U\right\rvert\geq 2\right\}$. Note that Lemma˜2.6.2 implies that hyperedges forming a tree in $H$ satisfy the hypothesis of Lemma˜2.6.3 below.

Lemma 2.6.3**.**

Let $H=(V,E)$ be a $k$-uniform hypergraph. Let $U\subseteq V$ and let the set of hyperedges $E(\operatorname{cl}(U))$ form a tree. For each $e\in E(\operatorname{cl}(U))$, let $\overline{\cal D}_{e}$ be a distribution on $[q]^{e}$ such that for any $u\in U$ and $e_{1},e_{2}\in E(\operatorname{cl}(U))$ such that $e_{1}\cap e_{2}=\{u\}$, we have $\overline{\cal D}_{e_{1}|u}=\overline{\cal D}_{e_{2}|u}=\overline{\cal D}_{u}$. Then,

there exists a distribution $\overline{\cal D}_{U}$ on $[q]^{U}$ such that $\overline{\cal D}_{U|e\cap U}=\overline{\cal D}_{e|e\cap U}$ for all $e\in E(U)$.

• -

If $U^{\prime}\subseteq U$ is such that the hyperedges in $E(\operatorname{cl}(U^{\prime}))$ form a subtree of $E(\operatorname{cl}(U))$, then $\overline{\cal D}_{U|U^{\prime}}=\overline{\cal D}_{U^{\prime}}$.

• **Proof:  ** We define the distribution by starting with an arbitrary hyperedge and traversing the tree in an arbitrary order. Let $e_{1},\ldots,e_{r}$ be a traversal of the hyperedges in $E(\operatorname{cl}(U))$ such that for all $i$, $\left\lvert\left(\cup_{j<i}e_{j}\right)\cap e_{i}\right\rvert=1$. Let $U_{0}=\cup_{j<i}e_{j}$ be the set of vertices for which we have already sampled an assignment and let $e_{i}$ be the next hyperedge in the traversal, with $u$ being the unique vertex in $e_{i}\cap U_{0}$. We sample an assignment to the vertices in $e$, conditioned on the value for the vertex $u$. Formally, we extend the distribution $\overline{\cal D}_{U_{0}}$ to $U_{0}\cup e$ by taking, for any $\alpha\in[q]^{U_{0}\cup e}$

[TABLE]

The above process defines a distribution $\overline{\cal D}_{\operatorname{cl}(U)}$ on $\operatorname{cl}(U)$, with

[TABLE]

In the above expression, we use $\deg(u)$ to denote the degree of vertex $u$ in tree formed by the hyperedges in $E(\operatorname{cl}(U))$ i.e., $\deg(u)=\left\lvert\left\{e\in E(\operatorname{cl}(U))~|~u\in e\right\}\right\rvert$. We then define the distribution $\overline{\cal D}_{U}$ as the marginalized distribution $\overline{\cal D}_{\operatorname{cl}(U)|U}$ i.e.,

[TABLE]

Note that the distribution $\overline{\cal D}_{\operatorname{cl}(U)}$ and hence also the distribution $\overline{\cal D}_{U}$ are independent of the order in which we traverse the hyperedges in $E(\operatorname{cl}(U))$. Also, since the above process samples each hyperedge according to the distribution $\overline{\cal D}_{e}$, we have that for any $e\in E(U)$, $\overline{\cal D}_{\operatorname{cl}(U)|e}=\overline{\cal D}_{e}$. Thus, also for any $e\in E(U)$, $\overline{\cal D}_{U|e\cap U}=\overline{\cal D}_{e|e\cap U}$.

Let $U^{\prime}\subseteq U$ be any set such that $E(\operatorname{cl}(U^{\prime}))$ forms a subtree of $E(\operatorname{cl}(U))$. Then there exists a traversal $e_{1},\ldots,e_{r}$, and $i\in[r]$ such that $e_{j}\in E(\operatorname{cl}(U^{\prime}))~\forall j\leq i$ and $e_{j}\notin E(\operatorname{cl}(U^{\prime}))~\forall j>i$. However, the distribution defined by the partial traversal $e_{1},\ldots,e_{i}$ is precisely $\overline{\cal D}_{\operatorname{cl}(U^{\prime})}$. Thus, we get that $\overline{\cal D}_{\operatorname{cl}(U)|\operatorname{cl}(U^{\prime})}=\overline{\cal D}_{\operatorname{cl}(U^{\prime})}$ which implies $\overline{\cal D}_{U|U^{\prime}}=\overline{\cal D}_{U^{\prime}}$.

We can now prove the completeness for our construction using consistent decompositions.

Lemma 2.6.4**.**

Let $\varepsilon>0$ and let $\Phi$ be a random instance of MAX k-CSPq ($f$) generated by choosing $\gamma\cdot n$ constraints independently at random as above. Then, there is a $t=\Omega_{\varepsilon,k,n_{0}}\left(\frac{\log n}{\log\log n}\right)$, such that with probability $1-\varepsilon$ over the choice of $\Phi$, there exist distributions $\left\{\mathcal{D}_{S}\right\}_{\left\lvert S\right\rvert\leq t}$ satisfying:

For all $S\subseteq V$ with $\left\lvert S\right\rvert\leq t$, $\mathcal{D}_{S}$ is a distribution on $[q]^{S}$.

• -

For all $T\subseteq S\subseteq V$ with $\left\lvert S\right\rvert\leq t$, $\mathcal{D}_{S|T}=\mathcal{D}_{T}$.

• -

The distributions satisfy

[TABLE]

• **Proof:  ** By ˜2.3.5, we know that there exists $\delta$ such that with probability $1-\varepsilon/4$, after removing a set of constraints $C_{B}$ of size at most $(\varepsilon/4)\cdot m$, we can assume that the remaining instance has girth at least $\mathtt{g}=\delta\cdot\log n$. Also, there exists $\theta,c>0$ such that for all $t\leq n^{\theta}$, the metric $\rho^{H}_{\mu}$ restricted to any set $S$ of size at most $t$ embeds isometrically into the unit sphere in $\ell_{2}$, for all $\mu\geq c\cdot\frac{\log t+\log\log n}{\log n}$.

We choose $\mu=2c\cdot\frac{\log\log n}{\log n}$ and $t=\frac{\varepsilon^{2}}{400k^{2}}\cdot\frac{1}{\mu}$ so that

[TABLE]

Thus, by Lemma˜2.5.3, $H$ admits an $(\varepsilon/2)$-sparse partitioning scheme of order $t$ with each cluster in the partition having diameter at most $\Delta_{H}=O(1/\mu)$. Let $\left\{\mathcal{P}_{S}\right\}_{\left\lvert S\right\rvert\leq t}$ denote this partitioning scheme.

Given a set $S$, the distribution $\mathcal{D}_{S}$ is a convex combination of several distributions $\mathcal{D}_{S,P}$, corresponding to different partitions $P$ sampled from $\mathcal{P}_{S}$. We describe the distribution $\mathcal{D}_{S}$ by giving the procedure to sample an $\alpha\in[q]^{S}$. Given the set $S$ with $\left\lvert S\right\rvert\leq t$:

Sample a partition $P=(U_{1},\ldots,U_{r})$ from the distribution $\mathcal{P}_{S}$.

• -

For each set $U_{i}$, consider the set $\mathcal{C}\left(U_{i}\right)$ obtained by including the vertices contained in all the hyperedges in the shortest path between all $u,v\in U_{i}$. Note that since $U_{i}$ has diameter at most $\Delta_{H}$ in $H$, $\mathcal{C}\left(U_{i}\right)$ is connected and in fact $\mathcal{C}\left(U\right)=\operatorname{cl}_{\Delta_{H}}(U)$. Also, since each vertex in an included path is within distance at most $\Delta_{H}/2$ of an end-point, and $U_{i}$ has diameter at most $\Delta_{H}$, we know that the diameter of $\mathcal{C}\left(U_{i}\right)$ is at most $2\cdot\Delta_{H}$. Hence, $\mathcal{C}\left(U_{i}\right)$ is a tree. Finally, we must have $\operatorname{cl}(\mathcal{C}\left(U_{i}\right))~=~\mathcal{C}\left(U_{i}\right)$ since any additional path of length 1 would create a cycle of length at most $2\cdot\Delta_{H}+1$.

Thus, by Lemma˜2.6.2 and Lemma˜2.6.3 (with probability at least $1-\varepsilon/4$) there exists a distribution $\overline{\cal D}_{\mathcal{C}\left(U_{i}\right)}$ for each $U_{i}$, satisfying $\overline{\cal D}_{\mathcal{C}\left(U_{i}\right)|e}=\overline{\cal D}_{e}$ for all $e\in E\left(\mathcal{C}\left(U_{i}\right)\right)$. Here, $\overline{\cal D}_{e}$ are the distributions given by Lemma˜2.6.2, which form a solution to the basic LP for $\Phi$, with value at least $c-\varepsilon/4$. For each $U_{i}$, define the distribution

[TABLE]

• -

Sample $\alpha\in[q]^{S}$ according to the distribution

[TABLE]

Thus, we have

[TABLE]

where the distributions $\mathcal{D}^{\prime}_{U_{i}}$ are defined as above.

We first prove the distributions are consistent on intersections i.e., $\mathcal{D}_{S|T}=\mathcal{D}_{T}$ for any $T\subseteq S$. Note that by Lemma˜2.5.3, the distributions $\mathcal{P}_{S}$ and $\mathcal{P}_{T}$ satisfy $\mathcal{P}_{S|T}=\mathcal{P}_{T}$. Each partition $(U_{1},\ldots,U_{r})$ also produces a partition $T$. For ease of notation, we assume that the first (say) $r^{\prime}$ clusters have non-empty intersection with $S$. Let $V_{i}=U_{i}\cap T$ for $1\leq i\leq r^{\prime}$ ($V_{i}=\emptyset$ for $i>r^{\prime}$). Then, we have

[TABLE]

The second to last equality above uses the fact that $\mathcal{C}\left(V_{i}\right)$ is a subtree of $\mathcal{C}\left(U_{i}\right)$ and thus $\overline{\cal D}_{\mathcal{C}\left(U_{i}\right)|\mathcal{C}\left(V_{i}\right)}=\overline{\cal D}_{\mathcal{C}\left(V_{i}\right)}$ by Lemma˜2.6.3. The last equality uses the fact that $\mathcal{P}_{S|T}=\mathcal{P}_{T}$ by Lemma˜2.5.3.

We now argue that the LP solution corresponding to the above distributions $\left\{\mathcal{D}_{S}\right\}_{\left\lvert S\right\rvert\leq t}$ has value at least $c-\varepsilon$. Recall that the value of the LP solution is given by

[TABLE]

Consider any constraint $C$ in $\Phi$, with the corresponding set of variables $S_{C}$ and the corresponding hyperedge $e$. When defining the distribution $\mathcal{D}_{S_{C}}$, we will partition $S_{C}$ according to the distribution $\mathcal{P}_{S_{C}}$. By Lemma˜2.5.3 and our choice of parameters

[TABLE]

For a constraint set which is not in the deleted set $C_{B}$, if the hyperedge $e$ corresponding to the constraint $C$ is not split by a partition $P$ sampled according to $\mathcal{P}_{S_{C}}$, then by Lemma˜2.6.3 $\mathcal{D}_{S_{C},P}=\overline{\cal D}_{S_{C}}$. Here, $\overline{\cal D}_{S_{C}}$ is the distribution given by Lemma˜2.6.2. Since $f$ is Boolean, we have that for $C\notin C_{B}$,

[TABLE]

Using Lemma˜2.6.2 again, we get

[TABLE]

where the penultimate inequality uses the fact that the fraction of constraints in the initially deleted set $C_{B}$ is at most $\varepsilon/4$ (for large enough $n$).

2.6.2 Integrality Gaps for Resistant Predicates

Let $f:\{0,1\}^{k}\to\{0,1\}$ be a boolean predicate and let $\rho(f)=\frac{f^{-1}(1)}{2^{k}}$ be the fractions of satisfying assignments to $f$. Then $f$ is approximation resistant if it is hard to distinguish the MAX-CSP instances on $f$ between which are at least $1-\operatorname{o}(1)$ satisfiable vs which are at most $\rho(f)+\operatorname{o}(1)$ satisfiable.

In [KTW14] the authors introduce the notion of vanishing measure (on a polytope defined by $f$) and use it to characterize a variant of approximation resistance, called strong approximation resistance, assuming the Unique Games conjecture. They also show gave a weaker notion of vanishing measures, which they used to characterize strong approximation resistance for LP hierarchies. In particular, they proved that when the condition in their characterization is satisfied, there exists a $(1-\operatorname{o}(1),\rho(f)+\operatorname{o}(1))$ integrality gap for $\operatorname{\operatorname{O}}(\log\log n)$ levels of Sherali-Adams hierarchy for predicates $f$. Here, we show that using ˜2.1.1, their result can be simplified and strengthened222The LP integrality gap result of Khot et al. is in fact slightly stronger than stated above. They show that LP value is at least $1-o(1)$ while there is no integer solution achieving a value outside the range $[\rho(f)-o(1),\rho(f)+o(1)]$. It is easy to see that the same also holds for the instance constructed here. to $O\left(\frac{\log n}{\log\log n}\right)$ levels.

Let us first recall some useful notation defined by Khot et al. [KTW14] before we define the notion of vanishing measure:

Definition 2.6.5**.**

For a predicate $f:\{0,1\}^{k}\to\{0,1\}$, let $\mathcal{C}(f)$ be the convex polytope of first moments (biases) of distributions supported on satisfying assignments of $f$ i.e.,

[TABLE]

For a measure $\Lambda$ on $\mathcal{C}(f)$, $S\subseteq[k]$, $b\in\{0,1\}^{S}$ and permutation $\pi:S\to S$, let $\Lambda_{S,\pi,b}$ denote the induced measure on ${\mathbb{R}}^{S}$ by considering vectors with coordinates $\left\{{(-1)}^{b_{\pi(i)}}\cdot\zeta_{\pi(i)}\right\}_{i\in S}$, where $\zeta\sim\Lambda$.

We recall below the definition of vanishing measure for LPs from [KTW14] (see Definition $1.3$):

Definition 2.6.6**.**

A measure $\Lambda$ on $\mathcal{C}(f)$ is called vanishing (for LPs) if for every $1\leq t\leq k$, the following signed measure

[TABLE]

is identically [math]. We say $f$ has a vanishing measure if there exists a vanishing measure $\Lambda$ on $\mathcal{C}(f)$.

In particular, they prove the following theorem:

Theorem 2.6.7**.**

Let $f:\{0,1\}^{k}\to\{0,1\}$ be a $k$-ary boolean predicate that has a vanishing measure. Then for every $\varepsilon>0$, there is a constant $c_{\varepsilon}>0$ such that for infinitely may $N\in{\mathbb{N}}$, there exists an instance $\Phi$ of $\textsf{MAX k-CSP}(f)$ on $N$ variables satisfying the following:

$\operatorname{\operatorname{OPT}}(\Phi)~\leq~\rho(f)+\varepsilon$.

• -

The optimum for the LP relaxation given by $c_{\varepsilon}\cdot\log\log N$ levels of Sherali-Adams hierarchy has $\operatorname{\operatorname{FRAC}}(\Phi)\geq 1-\operatorname{\operatorname{O}}(k\cdot\sqrt{\varepsilon})$.

Combining this with our ˜2.1.1 already gives us the following stronger result:

Corollary 2.6.8**.**

Let $f:\{0,1\}^{k}\to\{0,1\}$ be a $k$-ary boolean predicate that has a vanishing measure. Then for every $\varepsilon>0$, there is a constant $c_{\varepsilon}>0$ such that for infinitely may $N\in{\mathbb{N}}$, there exists an instance $\Phi$ of $\textsf{MAX k-CSP}(f)$ on $N$ variables satisfying the following:

All integral assignment of $\Phi$ satisfies at most $\rho(f)+\varepsilon$ fraction of constraints.

• -

The LP relaxation given by $c_{\varepsilon}\cdot\frac{\log N}{\log\log N}$ levels of Sherali-Adams hierarchy has $\operatorname{\operatorname{FRAC}}(\Phi)\geq 1-O(k\sqrt{\varepsilon})$.

However, note that to apply ˜2.1.1, one only needs a gap for the basic LP, which is much weaker requirement than the $O(\log\log N)$-level gap given by Theorem˜2.6.7. We observe below that the gap for the basic LP follows very simply from the construction by Khot et al. [KTW14]. One can then directly use this gap for applying ˜2.1.1 instead of going through Theorem˜2.6.7.

Khot et al. [KTW14] use the probabilistic construction given in Fig.˜2.4, for a given $\varepsilon>0$. The construction actually requires $\Lambda$ to be a vanishing measure over the polytope $\mathcal{C}_{\delta}(f)\stackrel{{\scriptstyle\mathrm{def}}}{{=}}(1-\delta)\cdot\mathcal{C}(f)$, for $\delta=\sqrt{\varepsilon}$. However, since $\mathcal{C}_{\delta}(f)$ is simply a scaling of $\mathcal{C}(f)$, a vanishing measure over $\mathcal{C}(f)$ also gives a vanishing measure over $\mathcal{C}_{\delta}(f)$. Note that each $\zeta_{0}\in\mathcal{C}(f)$ corresponds to a distribution $\nu_{0}$ supported in $f^{-1}(1)$. For each $\zeta\in\mathcal{C}_{\delta}$, let $\zeta_{0}=\frac{1}{1-\delta}\cdot\zeta$ be the point in $\mathcal{C}(f)$ with distribution $\nu_{0}$. Then the distribution $\nu=(1-\delta)\cdot\nu_{0}+\delta\cdot U_{k}$ (where $U_{k}$ denotes the uniform distribution on $\{0,1\}^{k}$) satisfies $\forall i\in[k]\mathchoice{\underset{\alpha\sim\nu}{\mathbb{E}}\left[(-1)^{\alpha_{i}}\right]}{{\mathbb{E}}_{\alpha\sim\nu}[(-1)^{\alpha_{i}}]}{{\mathbb{E}}_{\alpha\sim\nu}[(-1)^{\alpha_{i}}]}{{\mathbb{E}}_{\alpha\sim\nu}[(-1)^{\alpha_{i}}]}=\zeta_{i}$.

They show for a sufficiently large constant $\gamma$, an instance $\Phi$ with $m=\gamma\cdot n$ constraints satisfies with high probability, that for all assignments $\sigma$, $\left\lvert{\sf sat}_{\Phi}(\sigma)-\rho(f)\right\rvert\leq\varepsilon$ (see Lemma $4.4$ in [KTW14]). The proof is similar to that of Lemma˜2.6.1.

Additionally, we need the following claim from [KTW14] (see Claim $4.7$ there), which allows one to “round” coordinates of the vectors $\zeta\in\mathcal{C}_{\delta}(f)$ to the end-points of the intervals $I_{0},\ldots,I_{n_{0}}$. This ensures that any two variables in the same collection $X_{i}$ have the same bias. The proof of the claim follows simply from a hybrid argument.

Claim 2.6.9**.**

Let $\zeta\in\mathcal{C}_{\delta}(f)$ and let $\nu$ be the corresponding distribution supported in $f^{-1}(1)$ such that for all $i\in[k]$, we have $\zeta_{i}=\mathchoice{\underset{\alpha\sim\nu}{\mathbb{E}}\left[(-1)^{\alpha_{i}}\right]}{{\mathbb{E}}_{\alpha\sim\nu}[(-1)^{\alpha_{i}}]}{{\mathbb{E}}_{\alpha\sim\nu}[(-1)^{\alpha_{i}}]}{{\mathbb{E}}_{\alpha\sim\nu}[(-1)^{\alpha_{i}}]}$. Let $t_{1},\ldots,t_{k}\in[0,1]$ be such that for all $i\in[k]$, $\left\lvert t_{i}-\left\lvert\zeta_{i}\right\rvert\right\rvert\leq\varepsilon$ for $\varepsilon<\delta/2$. Then there exists a distribution $\nu^{\prime}$ on $\{0,1\}^{k}$ such that

[TABLE]

• **Proof:  ** Let $r_{j}=\operatorname{\operatorname{sgn}}(\zeta_{j})\cdot t_{j}$ be the desired bias of the $j^{th}$ coordinate. Then, $\left\lvert\zeta(j)-r_{j}\right\rvert\leq\varepsilon$ for all $j\in[k]$ We construct a sequence of distributions $\nu_{0},\ldots,\nu_{k}$ such that $\nu_{0}=\nu$ and $\nu_{k}=\nu^{\prime}$. In $\overline{\nu}_{j}$, the biases are $(r_{1},\ldots,r_{j},\zeta_{j+1},\ldots,\zeta_{k})$.

The biases in $\nu_{0}$ satisfy the above by definition. We obtain $\overline{\nu}_{j}$ from $\overline{\nu}_{j-1}$ as,

[TABLE]

where $D_{j}$ is the distribution in which all bits, except for the $j^{th}$ one, are set independently according to their biases in $\overline{\nu}_{j-1}$. For the $j^{th}$ bit, we set it to $\operatorname{\operatorname{sgn}}(r_{j}-\zeta_{j})$ (if $r_{j}-\zeta(j)=0$, we can simply proceed with $\overline{\nu}_{j}=\overline{\nu}_{j-1}$). The biases for all except for the $j^{th}$ bit are unchanged. For the $j^{th}$ bit, the bias now becomes $r_{j}$ if

[TABLE]

Since $\zeta\in\mathcal{C}_{\delta}(f)$, we know that $\left\lvert\operatorname{\operatorname{sgn}}(r_{j}-\zeta(j))-r_{j}\right\rvert\geq\delta/2$. Also, $\left\lvert r_{j}-\zeta(j))\right\rvert\leq\varepsilon$ by assumption. Thus, we can choose $\tau_{j}=O(\varepsilon/\delta)$ which gives that $\mathchoice{\lVert\overline{\nu}_{j}-\overline{\nu}_{j-1}\rVert}{\lVert\overline{\nu}_{j}-\overline{\nu}_{j-1}\rVert}{\lVert\overline{\nu}_{j}-\overline{\nu}_{j-1}\rVert}{\lVert\overline{\nu}_{j}-\overline{\nu}_{j-1}\rVert}_{1}=O(\varepsilon/\delta)$. The final bound then follows by triangle inequality.

We can now use the above to give a simplified proof of Corollary˜2.6.8.

• **Proof ofCorollary˜2.6.8:  ** Here we exhibit a solution of the basic LP Fig.˜2.2 for the instance given in Fig.˜2.4. For each variable $y_{j}$ coming from the set $X_{j}$ for $j\in\{0,1,\ldots,n_{0}\}$, we set the bias $t_{j}$ of the variable to be the rightmost point of the interval $I_{j}$ i.e., set $x_{(y_{j},-1)}=\frac{1}{2}\cdot\left(1-\frac{i}{n_{0}}\right)$ and $x_{(y_{j},1)}=\frac{1}{2}\cdot\left(1+\frac{i}{n_{0}}\right)$.

For each constraint $C$ of the form $f(y_{i_{1}}+b_{1},\ldots,y_{i_{k}}+b_{k})$, let $\zeta(C)\in\mathcal{C}_{\delta}(f)$ be the point used to generate it, and let $\nu(C)$ denote the corresponding distribution on $\{0,1\}^{k}$. By ˜2.6.9, there exists a distribution $\nu^{\prime}(C)$ such that $\mathchoice{\lVert\nu(C)-\nu^{\prime}(C)\rVert}{\lVert\nu(C)-\nu^{\prime}(C)\rVert}{\lVert\nu(C)-\nu^{\prime}(C)\rVert}{\lVert\nu(C)-\nu^{\prime}(C)\rVert}_{1}=O(k\varepsilon/\delta)$ and such that the biases of the literals satisfy $\mathchoice{\underset{\alpha\sim\nu^{\prime}(C)}{\mathbb{E}}\left[(-1)^{\alpha_{j}}\right]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[(-1)^{\alpha_{j}}]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[(-1)^{\alpha_{j}}]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[(-1)^{\alpha_{j}}]}=\operatorname{\operatorname{sgn}}(\zeta_{j})\cdot t_{i_{j}}$, where $t_{i_{j}}$ denotes the bias for the interval to which $y_{i_{j}}$ belongs. When $t_{i_{j}}\neq 0$, we negate a variable only when $\operatorname{\operatorname{sgn}}(\zeta_{j})<0$. Thus, we have $\mathchoice{\underset{\alpha\sim\nu^{\prime}(C)}{\mathbb{E}}\left[(-1)^{\alpha_{j}+b_{j}}\right]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[(-1)^{\alpha_{j}+b_{j}}]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[(-1)^{\alpha_{j}+b_{j}}]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[(-1)^{\alpha_{j}+b_{j}}]}=t_{i_{j}}$, which is consistent with the bias given by the singleton variables $x_{(y_{i_{j}},1)}$ and $x_{(y_{i_{j}},-1)}$. We thus define the local distribution on the set $S_{C}$ as $\mathcal{D}_{S_{C}}(\alpha)=(\nu^{\prime}(C))(\alpha+b_{C})$.

For all $C\in\Phi$, since $\zeta(C)\in\mathcal{C}_{\delta}(f)$, we have that $\mathchoice{\underset{\alpha\sim\nu(C)}{\mathbb{E}}\left[f(\alpha)\right]}{{\mathbb{E}}_{\alpha\sim\nu(C)}[f(\alpha)]}{{\mathbb{E}}_{\alpha\sim\nu(C)}[f(\alpha)]}{{\mathbb{E}}_{\alpha\sim\nu(C)}[f(\alpha)]}\geq 1-\delta$. Also, since $\mathchoice{\lVert\nu(C)-\nu^{\prime}(C)\rVert}{\lVert\nu(C)-\nu^{\prime}(C)\rVert}{\lVert\nu(C)-\nu^{\prime}(C)\rVert}{\lVert\nu(C)-\nu^{\prime}(C)\rVert}_{1}=O(k\varepsilon/\delta)$, we get that $\mathchoice{\underset{\alpha\sim\nu^{\prime}(C)}{\mathbb{E}}\left[f(\alpha)\right]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[f(\alpha)]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[f(\alpha)]}{{\mathbb{E}}_{\alpha\sim\nu^{\prime}(C)}[f(\alpha)]}\geq 1-\delta-O(k\varepsilon/\delta)$. Thus, we have for all $C\in\Phi$, $\mathchoice{\underset{\alpha\sim\mathcal{D}_{S_{C}}}{\mathbb{E}}\left[f(\alpha+b_{C})\right]}{{\mathbb{E}}_{\alpha\sim\mathcal{D}_{S_{C}}}[f(\alpha+b_{C})]}{{\mathbb{E}}_{\alpha\sim\mathcal{D}_{S_{C}}}[f(\alpha+b_{C})]}{{\mathbb{E}}_{\alpha\sim\mathcal{D}_{S_{C}}}[f(\alpha+b_{C})]}\geq 1-\delta-O(k\varepsilon/\delta)$. Taking $\delta=\sqrt{\varepsilon}$ proves the claim.

2.6.3 Lower-bounds for LP Extended Formulations

A connection between LP integrality gaps for the Sheral-Adams hierarchy, and lower bounds on the size of LP extended formulations, was first established by Chan et al. [CLRS13] and later improved by Kothari et al. [KMR17]. In [KMR17], the authors proved the following:

Theorem 2.6.10** ([KMR17], Theorem$1.2$).**

There exist constants $0<h<H$ such that the following holds. Consider a function $f:\mathbb{N}\to\mathbb{N}$. Suppose that the $f(n)$-level Sherali-Adams relaxation for a CSP cannot achieve a $(c,s)$-approximation on instances on $n$ variables. Then, no LP extended formulation (of the original LP) of size at most $n^{h\cdot f(n)}$ can achieve a $(c,s)$-approximation for the CSP on $n^{H}$ variables.

Combining ˜2.1.1 with Theorem˜2.6.10 yields (with $f(N)=c_{\varepsilon}\cdot\frac{\log N}{\log\log N}$): See 2.1.2

Chapter 3 Optimizing Polynomials Over Sphere

3.1 Introduction and Context

In this chapter, we study the problem of optimizing homogeneous polynomials over the unit sphere. Formally, given an $n$-variate degree-$d$ homogeneous polynomial $f$, the goal is to compute

[TABLE]

When $f$ is a homogeneous polynomial of degree 2, this problem is equivalent computing the spectral norm of an associated symmetric matrix $M_{f}$. For higher degree $d$, it defines a natural higher-order analogue of the eigenvalue problem for matrices. The problem also provides an important testing ground for the development of new spectral and semidefinite programming (SDP) techniques, and techniques developed in the context of this problem have had applications to various other constrained settings [HLZ10, Lau09, Las09].

Besides being a natural and fundamental problem in its own right, it has connections to widely studied questions in many other areas. In quantum information theory [BH13, BKS14], the problem of computing the optimal success probability of a protocol for Quantum Merlin-Arthur games can be thought of as optimizing certain classes of polynomials over the unit sphere. The problem of estimating the $2\rightarrow 4$ norm of an operator, which is equivalent to optimizing certain homogeneous degree-4 polynomials over the sphere, is known to be closely related to the Small Set Expansion Hypothesis (SSEH) and the Unique Games Conjecture (UGC) [BBH*+*12, BKS14]. The polynomial optimization problem is also very relevant for natural extensions of spectral problems, such as low-rank decomposition and PCA, to the case of tensors [BKS15, GM15, MR14, HSS15]. Frieze and Kannan [FK08] (see also [BV09]) also established a connection between the problem of approximating the spectral norm of a tensor (or equivalently, computing $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ for a polynomial $f$), and finding planted cliques in random graphs.

The problem of polynomial optimization has been studied111In certain cases, the problem studied is not to maximize $\left\lvert f\right\rvert$, but just $f(x)$. While the two problems are equivalent for homogeneous polynomials of odd degree, some subtle issues arise when considering polynomials of even degree. We compare the two notions in the full version [BGG*+*17]. over various compact sets [Las09, DK08], and is natural to ask how well polynomial time algorithms can approximate the optimum value over a given compact set (see [DK08] for a survey). While the maximum of a degree-$d$ polynomial over the simplex admits a PTAS for every fixed $d$ [dKLP06], the problem of optimizing even a degree $3$ polynomial over the hypercube does not admit any approximation better than $2^{{(\log n)}^{1-\varepsilon}}$ (for arbitrary $\varepsilon>0$) assuming NP cannot be solved in time $2^{{(\log n)}^{O(1)}}$ [HV04].

The approximability of polynomial optimization on the sphere is poorly understood in comparison. It is known that the maximum of a degree-$d$ polynomial can be approximated within a factor of $n^{d/2-1}$ in polynomial time [HLZ10, So11a]. On the hardness side, Nesterov [Nes03] gave a reduction from Maximum Independent Set to optimizing a homogeneous cubic polynomial over $\mathbb{S}^{n-1}$. Formally, given a graph $G$, there exists a homogeneous cubic polynomial $f(G)$ such that $\sqrt{1-\frac{1}{\alpha(G)}}=\max_{\|x\|=1}f(x)$. Combined with the hardness of Maximum Independent Set [Hås96a], this rules out an FPTAS for optimization over the unit sphere. Assuming the Exponential Time Hypothesis, Barak et al. [BBH*+*12] proved that computing $2\rightarrow 4$ norm of a matrix, a special case when $f$ is a degree-$4$ homogeneous polynomial, is hard to approximate within a factor $\exp(\log^{1/2-\epsilon}(n))$ for any $\epsilon>0$.

Optimization over $\mathbb{S}^{n-1}$ has been given much attention in the optimization community, where for a fixed number of variables $n$ and degree $d$ of the polynomial, it is known that the estimates produced by $q$ levels a certain hierarchy of SDPs (Sum of Squares) get arbitrarily close to the true optimal solution as $q$ increases (see [Las09] for various applications). We refer the reader to the recent work of Doherty and Wehner [DW12] and de Klerk, Laurent, and Sun [dKLS14] and references therein for more information on convergence results. These algorithms run in time $n^{O(q)}$, which is polynomial for constant $q$. Unfortunately, known convergence results often give a non-trivial bound only when the $q$ is linear in $n$.

In computer science, much attention has been given to the sub-exponential runtime regime (i.e. $q\ll n$) since many of the target applications such as SSE, QMA and refuting random CSPs are of considerable interest in this regime. In addition to the polytime $n^{d/2-1}$-approximation for general polynomials [HLZ10, So11a], approximation guarantees have been proved for several special cases including $2\rightarrow q$ norms [BBH*+*12], polynomials with non-negative coefficients [BKS14], some polynomials that arise in quantum information theory [BKS17, BH13], and random polynomials [RRS16, BGL16]. Hence, there is considerable interest in tightly characterizing the approximation guarantee achievable using sub-exponential time.

In this chapter, we develop general techniques to design and analyze algorithms for polynomial optimization over the sphere. The sphere constraint is one of the simplest constraints for polynomial optimization and thus is a good test-bed for techniques. Indeed, we believe these techniques will also be useful in understanding polynomial optimization for other constrained settings.

In addition to giving an analysis the problem for arbitrary polynomials, these techniques can also be adapted to take advantage of the structure of the input polynomial, yielding better approximations for several special cases such as polynomials with non-negative coefficients, and sparse polynomials. Previous polynomial time algorithms for polynomial optimization work by reducing the problem to diameter estimation in convex bodies [So11a] and seem unable to utilize structural information about the (class of) input polynomials. Development of a method which can use such information was stated as an open problem by Khot and Naor [KN08] (in the context of $\ell_{\infty}$ optimization).

Our approximation guarantees are with respect to the optimum of the well-studied Lasserre/sum-of-squares (SoS) semidefinite programming relaxation. Such SDPs are the most natural tool to bound the optima of polynomial optimization problems, and our results shed light on the efficacy of higher levels of the SoS hierarchy to deliver better approximations to the optimum. We discuss the SoS connection in Section˜3.1.2, but first turn to stating our approximation guarantees.

3.1.1 Our Algorithmic Results

For a homogeneous polynomial $h$ of even degree $q$, a matrix $M_{h}\in\Re^{{[n]}^{q/2}\times{[n]}^{q/2}}$ is called a matrix representation of $h$ if ${(x^{\otimes q/2})}^{T}\cdot M_{h}\cdot x^{\otimes q/2}=h(x)~\forall x\in{\mathbb{R}}^{n}$. Next we define the quantity,

[TABLE]

Let $h_{\max}$ denote $\sup_{\|x\|=1}h(x)$. Clearly, $h_{\max}\leq\Lambda\left(h\right)$, i.e. $\Lambda\left(h\right)$ is a relaxation of $h_{\max}$. However, this does not imply that $\Lambda\left(h\right)$ is a relaxation of $\mathchoice{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}$, since it can be the case that $h_{\max}\neq\mathchoice{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}$. To remedy this, one can instead consider $\sqrt{\Lambda\left(h^{2}\right)}$ which is a relaxation of $\mathchoice{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}$, since $(h^{2})_{\max}=\mathchoice{\lVert h^{2}\rVert_{\infty}}{\lVert h^{2}\rVert_{\infty}}{\lVert h^{2}\rVert_{\infty}}{\lVert h^{2}\rVert_{\infty}}$. More generally, for a degree-$d$ homogeneous polynomial $f$ and an integer $q$ divisible by $2d$, we have the upper estimate

[TABLE]

The following result shows that $\Lambda\left(f^{q/d}\right)^{d/q}$ approximates $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ within polynomial factors, and also gives an algorithm to approximate $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ with respect to the upper bound $\Lambda\left(f^{q/d}\right)^{d/q}$. In the statements below and the rest of this section, $O_{d}(\cdot)$ and $\Omega_{d}(\cdot)$ notations hide $2^{O(d)}$ factors. Our algorithmic results are as follows:

Theorem 3.1.1**.**

Let $f$ be an $n$-variate homogeneous polynomial of degree-$d$, and let $q\leq n$ be an integer divisible by $2d$. Then,

[TABLE]

(where $\Lambda_{C}\left(\cdot\right)$ is a related efficiently computable quantity)

Furthermore, there is a deterministic algorithm that runs in $n^{O(q)}$ time and returns $x$ such that

[TABLE]

where $c(n,d,q)$ is ${(n/q)}^{d/2-1}$, ${(n/q)}^{d/4-1/2}$ and $\sqrt{m/q}$ respectively, for each of the above cases (the inequality uses $\Lambda_{C}\left(\cdot\right)$ in the case of polynomials with non-negative coefficients).

Remark 3.1.2**.**

Interestingly, our deterministic algorithms only involve computing the maximum eigenvectors of $n^{O(q)}$ different matrices in $\Re^{n\times n}$. We actually don’t require computing $\Lambda\left(f^{q/d}\right)^{d/q}$, even though this quantity can also be computed in $n^{O(q)}$ time by the sum-of-squares SDP (see Section˜3.1.2). The quantity $\Lambda\left(f^{q/d}\right)^{d/q}$ is only used in the analysis.

Remark 3.1.3**.**

If $m=n^{\,\rho\cdot d}$ for $\rho<1/3$, then for all $q\leq n^{1-\rho}$, the $\sqrt{m/q}$-approximation for sparse polynomials is better than the ${(n/q)}^{d/2-1}$ arbitrary polynomial approximation.

Remark 3.1.4**.**

In cases where $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}=f_{\max}$ (such as when $d$ is odd or $f$ has non-negative coefficients), the above result holds whenever $q$ is even and divisible by $d$, instead of $2d$.

A key technical ingredient en route establishing the above results is a method to reduce the problem for arbitrary polynomials to a list of multilinear polynomial problems (over the same variable set). We believe this to be of independent interest, and describe its context and abstract its consequence (Theorem˜3.1.5) next.

Let $M_{g}$ be a matrix representation of a degree-$q$ homogeneous polynomial $g$, and let $K=(I,J)\in[n]^{q/2}\times[n]^{q/2}$ have all distinct elements. Observe that there are $q!$ distinct entries of $M_{g}$ including $K$ across which, one can arbitrarily assign values and maintain the property of representing $g$, as long as the sum across all $q!$ entries remains the same (specifically, this is the set of all permutations of $K$). In general for $K^{\prime}=(I^{\prime},J^{\prime})\in[n]^{q/2}\times[n]^{q/2}$, we define the orbit of $K^{\prime}$ denoted by $\mathscr{O}{\,\!\left({K^{\prime}}\right)}$, as the set of permutations of $K^{\prime}$, i.e. the number of entries to which ’mass’ from $M_{g}[I^{\prime},J^{\prime}]$ can be moved while still representing $g$.

As $q$ increases, the orbit sizes of the entries increase, and to show better bounds on $\Lambda\left(f^{q/d}\right)$, one must exploit these additional "degrees of freedom" in representations of $f^{q/d}$. However, a big obstacle is that the orbit sizes of different entries can range anywhere from $1$ to $q!$,  two extremal examples being $((1,\dots 1),(1,\dots 1))$ and $((1,\dots q/2),(q/2+1,\dots q))$. This makes it hard to exploit the additional freedom afforded by growing $q$. Observe that if $g$ were multilinear, all matrix entries corresponding to non-zero coefficients have a span of $q!$ and indeed it turns out to be easier to analyze the approximation factor in the multilinear case as a function of $q$ since the representations of $g$ can be highly symmetrized. However, we are still faced with the problem of $f^{q/d}$ being highly non-multilinear. The natural symmetrization strategies that work well for multilinear polynomials fail on general polynomials, which motivates the following result:

Theorem 3.1.5** (Informal).**

For even $q$, let $g(x)$ be a degree-$q$ homogeneous polynomial. Then there exist multilinear polynomials $g_{1}(x),\dots,g_{m}(x)$ of degree at most $q$, such that

[TABLE]

and $m=q^{O(q)}$.

By combining Theorem˜3.1.5 (or an appropriate generalization) with the appropriate analysis of the multilinear polynomials induced by $f^{q/d}$, we obtain the aforementioned results for various classes of polynomials.

Weak decoupling lemmas.

A common approach for reducing to the multilinear case is through more general “decoupling” or “polarization” lemmas, which also have a variety of applications in functional analysis and probability [DlPG12]. However, such methods increase the number of variables to $nq$, which would completely nullify any advantage obtained from the increased degrees of freedom. This is because the approximation obtained would be of the form $(\#vars/q)^{d/2-1}=n^{d/2-1}$.

Our proof of Theorem˜3.1.5 (and its generalizations) requires only a decoupling with somewhat weaker properties than given by the above lemmas. However, we need it to be very efficient in the number of variables. In analogy with “weak regularity lemmas” in combinatorics, which trade structural control for complexity of the approximating object, we call these results “weak decoupling lemmas” (see Section˜3.2.1). They provide a milder form of decoupling but only increase the number of variables to $2n$ (independently of $q$).

We believe these could be more generally applicable; in particular to other constrained settings of polynomial optimization as well as in the design of sub-exponential algorithms. Our techniques might also be able to yield a full tradeoff between the number of variables and quality of decoupling.

3.1.2 Connection to sum-of-squares hierarchy

The Sum of Squares Hierarchy (SoS) is one of the canonical and well-studied approaches to attack polynomial optimization problems. Algorithms based on this framework are parametrized by the degree or level $q$ of the SoS relaxation. For the case of optimization of a homogeneous polynomial $h$ of even degree $q$ (with some matrix representation $M_{h}$) over the unit sphere, the level $q$ SoS relaxes the non-convex program of maximizing $(x^{\otimes q/2})^{T}\cdot M_{h}\cdot x^{\otimes q/2}=h(x)$ over $x\in{\mathbb{R}}^{n}$ with $\|x\|_{2}=1$, to the semidefinite program of maximizing $\mathop{\mathsf{Tr}}\!\left({M^{T}_{h}X}\right)$ over all positive semidefinite matrices $X\in{\mathbb{R}}^{[n]^{q/2}\times[n]^{q/2}}$ with $\mathop{\mathsf{Tr}}\!\left({X}\right)=1$. (This is a relaxation because $X=x^{\otimes q/2}(x^{\otimes q/2})^{T}$ is psd with $\mathop{\mathsf{Tr}}\!\left({X}\right)=\|x\|_{2}^{q}$.)

It is well known (see for instance [Lau09]) that the quantity $\Lambda\left(h\right)$ from Eq.˜3.2 is the dual value of this SoS relaxation. Further, strong duality holds for the case of optimization on the sphere and therefore $\Lambda\left(h\right)$ equals the optimum of the SoS SDP and can be computed in time $n^{O(q)}$. (See the full version [BGG*+*17] for more detailed SoS preliminaries.) In light of this, our results from Theorem˜3.1.1 can also be viewed as a convergence analysis of the SoS hierarchy for optimization over the sphere, as a function of the number of levels $q$. Such results are of significant interest in the optimization community, and have been studied for example in [DW12, dKLS14] (see Section˜3.1.3 for a comparison of results).

SoS Lower Bounds. While the approximation factors in our upper bounds of Theorem˜3.1.1 are modest, there is evidence to suggest that this is inherent.

When $h$ is a degree-$q$ polynomial with random i.i.d $\pm 1$ coefficients, it was shown in [BGL16] that there is a constant $c$ such that w.h.p. \Bigl{(}\frac{n}{q^{c+o(1)}}\Bigr{)}^{q/4}\leq\Lambda\left(h\right)\leq\Bigl{(}\frac{n}{q^{c-o(1)}}\Bigr{)}^{q/4}. On the other hand, $\mathchoice{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}\leq O(\sqrt{nq\log q})$ w.h.p. Thus, the ratio between $\Lambda\left(h\right)$ and $\mathchoice{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}$ can be as large as $\Omega_{q}(n^{q/4-1/2})$.

Hopkins et al. [HKP*+*17] recently proved that degree-$d$ polynomials with random coefficients achieve a degree-$q$ SoS gap of roughly ${(n/q^{O(1)})}^{d/4-1/2}$ (provided $q>n^{\epsilon}$ for some constant $\epsilon>0$). This is also a lower bound on the ratio between $\Lambda\left(f^{q/d}\right)^{d/q}$ and $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ for the case of arbitrary polynomials. Note that this lower bound is roughly square root of our upper bound from Theorem˜3.1.1. Curiously, our upper bound for the case of polynomials with non-negative coefficients essentially matches this lower bound for random polynomials.

Non-Negative Coefficient Polynomials. In this chapter, we give a new lower bound construction for the case of non-negative polynomials, To the best of our knowledge, the only previous lower bound for this problem, was known through Nesterov’s reduction [DK08], which only rules out a PTAS. We give the following polynomially large lower bound. The gap applies for random polynomials associated with a novel distribution of $4$-uniform hypergraphs, and is analyzed using subgraph counts in a random graph.

Theorem 3.1.6**.**

There exists an $n$ variate degree-4 homogeneous polynomial $f$ with non-negative coefficients such that

[TABLE]

For larger degree $t$, we prove an $n^{\Omega(t)}$ gap between $\mathchoice{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}$ and a quantity $\mathchoice{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}$ that is closely related to $\Lambda\left(h\right)$. Specifically, $\mathchoice{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}$ is defined by replacing the largest eigenvalue of matrix representations $M_{h}$ of $h$ in Eq.˜3.2 by the spectral norm $\mathchoice{\lVert M_{h}\rVert_{2}}{\lVert M_{h}\rVert_{2}}{\lVert M_{h}\rVert_{2}}{\lVert M_{h}\rVert_{2}}$. (See [BGG*+*17] for a formal definition.) Note that $\mathchoice{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}{\lVert h\rVert_{\mathrm{sp}}}\geq\max\{\Lambda\left(h\right),\Lambda\left(-h\right)\}$. Like $\Lambda\left(\cdot\right)$, $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ suggests a natural hierarchy of relaxations for the problem of approximating $\mathchoice{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}$, obtained by computing $\|h^{q/t}\|_{sp}^{t/q}$ as the $q$-th level of the hierarchy.

We prove a lower bound of $n^{q/24}/\left(q\cdot\log n\right)^{O(q)}$ on $\|f^{q/4}\|_{sp}$ where $f$ is as in Theorem˜3.1.6. This not only gives $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ gaps for the degree-$q$ optimization problem on polynomials with non-negative coefficients, but also an $n^{1/6}/{(q\log n)}^{O(1)}$ gap on higher levels of the aforementioned $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ hierarchy for optimizing degree-$4$ polynomials with non-negative coefficients. Formally we show:

Theorem 3.1.7**.**

Let $g:=f^{q/4}$ where $f$ is the degree-$4$ polynomial as in Theorem˜3.1.6. Then

[TABLE]

Our lower bound on $\|f^{q/4}\|_{sp}$ is based on a general tool that allows one to “lift” level-$4$ $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ gaps, that meet one additional condition, to higher levels. While we derive final results only for the weaker relaxation $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$, the underlying structural result can be used to lift SoS lower bounds (i.e. gaps for $\Lambda\left(\cdot\right)$) as well, provided the SoS solution matrix $X$ satisfies PSD-ness of two other matrices of appropriately related shapes to $X$ (see full version [BGG*+*17]) — this inspired us to name our tool “Tetris theorem.” Recently, the insightful pseudo-calibration approach [BHK*+*16] has provided a recipe to give higher level SoS lower bounds for certain average-case problems. We believe our lifting result might similarly be useful in the context of worst-case problems, where in order to get higher degree lower bounds, it suffices to give lower bounds for constant degree SoS with some additional structural properties.

3.1.3 Related Previous and Recent Works

Polynomial optimization is a vast area with several previous results. Below, we collect the results most relevant for comparison with the ones in this chapter, grouped by the class of polynomials. Please see the excellent monographs [Lau09, Las09] for more.

Arbitrary Polynomials. For general homogeneous polynomials of degree-$d$, a polytime $O_{d}\left(n^{d/2-1}\right)$-approximation was given by He et al. [HLZ10], which was improved to $O_{d}\left({(n/\log n)}^{d/2-1}\right)$ by So [So11a]. The convergence of SDP hierarchies for polynomial optimization was analyzed by Doherty and Wehner [DW12]. However, their result only applies to relaxations given by $\Omega(n)$ levels of the SoS hierarchy (Theorem 7.1 in [DW12]). Thus, our results can be seen as giving an interpolation between the polynomial time algorithms obtained by [HLZ10, So11a] and the exponential time algorithms given by $\Omega(n)$ levels of SoS, although the bounds obtained by [DW12] are tighter (by a factor of $2^{O(d)}$) for $q=\Omega(n)$ levels.

For the case of arbitrary polynomials, we believe a tradeoff between running time and approximation quality similar to ours can also be obtained by considering the tradeoffs for the results of Brieden et al. [BGK*+*01] used by So [So11a]. However, to the best of our knowledge, this is not published. In particular, So uses the techniques of Khot and Naor [KN08] to reduce degree-$d$ polynomial optimization to $d-2$ instances of the problem of optimizing the $\ell_{2}$ diameter of a convex body. This is solved by [BGK*+*01], who give an $O({(n/k)}^{1/2})$ approximation in time $2^{k}\cdot n^{O(1)}$. We believe this can be combined with proof of So, to yield a $O_{d}\left({(n/q)}^{d/2-1}\right)$ approximation in time $2^{q}$. We note here that the method of Khot and Naor [KN08] cannot be improved further (up to polylog) for the case $d=3$ (see Section˜3.6). Our results for the case of arbitrary polynomials show that similar bounds can also be obtained by a very generic algorithm given by the SoS hierarchy. Moreover, the general techniques developed here are versatile and demonstrably applicable to various other cases (like polynomials with non-negative coefficients, sparse polynomials, worst-case sparse PCA) where no alternate proofs are available. The techniques of [KN08, So11a] are oblivious to the structure in the polynomials, and it appears to be unlikely that similar results can be obtained by using diameter estimation techniques.

Polynomials with Non-negative Coefficients. The case of polynomials with non-negative coefficients was considered by Barak, Kelner, and Steurer [BKS14] who proved that the relaxation obtained by $\Omega(d^{3}\cdot\log n/\varepsilon^{2})$ levels of the SoS hierarchy provides an $\varepsilon\cdot\|f\|_{BKS}$ additive approximation to the quantity $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$. Here, the parameter we denote by $\mathchoice{\lVert f\rVert_{BKS}}{\lVert f\rVert_{BKS}}{\lVert f\rVert_{BKS}}{\lVert f\rVert_{BKS}}$ corresponds to a relaxation for $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ that is weaker than the one given by $\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}$.222Specifically, $\mathchoice{\lVert f\rVert_{BKS}}{\lVert f\rVert_{BKS}}{\lVert f\rVert_{BKS}}{\lVert f\rVert_{BKS}}$ minimizes the spectral norm over a smaller set of matrix representations of $f$ than $\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}$ which allows all matrix representations. Their results can be phrased as showing that a relaxation obtained by $q$ levels of the SoS hierarchy gives an approximation ratio of

[TABLE]

Motivated by connections to quantum information theory, they were interested in the special case where $\|f\|_{BKS}/\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ is bounded by a constant. However, this result does not imply strong multiplicative approximations outside this special case since, in general, $\|f\|_{BKS}$ and $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ can be far apart. In particular, we are able to establish that there exist polynomials $f$ with non-neg. coefficients such that $\|f\|_{BKS}/\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}\geq n^{d/24}$. Moreover, we conjecture that the worst-case gap between $\|f\|_{BKS}$ and $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ for polynomials with non-neg. coefficients are as large as $\widetilde{\Omega}_{d}((n/d)^{d/4-1/2})$ (note that the conjectured $(n/d)^{d/4-1/2}$ gap for non-negative coefficient polynomials is realizable using arbitrary polynomials, i.e. it was established in [BGL16] that polynomials with i.i.d. $\pm 1$ coefficients achieve this gap w.h.p.).

Our results show that $q$ levels of SOS gives an $(n/q)^{d/4-1/2}$ approximation to $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ which has a better dependence on $q$ and consequently, converges to a constant factor approximation after $\Omega(n)$ levels.

$2$-to-$4$ norm. It was proved in [BKS14] that for any matrix $A$, $q$ levels of the SoS hierarchy approximates $\|A\|^{4}_{2\rightarrow 4}=\mathchoice{\lVert\|Ax\|_{4}^{4}\rVert_{\infty}}{\lVert\|Ax\|_{4}^{4}\rVert_{\infty}}{\lVert\|Ax\|_{4}^{4}\rVert_{\infty}}{\lVert\|Ax\|_{4}^{4}\rVert_{\infty}}$ (i.e. the fourth power of the $2$-to-$4$-norm) within a factor of

[TABLE]

Brandao and Harrow [BH15] also gave a nets based algorithm with runtime $2^{q}$ that achieves the same approximation as above. Here again, the cases of interest were those matrices for which $\|A\|^{2}_{2\rightarrow 2}\|A\|^{2}_{2\rightarrow\infty}$ and $\|A\|^{4}_{2\rightarrow 4}$ are at most constant apart.

We would like to bring attention to an open problem in this line of work. It is not hard to show that for an $m\times n$ matrix $A$ with i.i.d. Gaussian entries, $\|A\|^{2}_{2\rightarrow 2}=\Theta(m+n)$, $\|A\|^{2}_{2\rightarrow\infty}=\Theta(n)$, and $\|A\|^{2}_{2\rightarrow 4}=\Theta(m+n^{2})$ which implies the worst case approximation factor achieved above is $\Omega(n/\sqrt{q})$ when we take $m=\Omega(n^{2})$.

Our result for arbitrary polynomials of degree-$4$, achieves an approximation factor of $O(n/q)$ after $q$ levels of SoS which implies that the current best known approximation $2$-to-$4$ norm is oblivious to the structure of the $2$-to-$4$ polynomial and seems to suggest that this problem can be better understood for arbitrary tall matrices. For instance, can one get a $\sqrt{m}/q$ approximation for $(m\times n)$ matrices (note that [BH15] already implies a $\sqrt{m/q}$-approximation for all $m$, and our result implies a $\sqrt{m}/q$-approximation when $m=\Omega(n^{2})$).

Random Polynomials. For the case when $f$ is a degree-$d$ homogeneous polynomial with i.i.d. random $\pm 1$ coefficients [BGL16, RRS16] showed that degree-$q$ SoS certifies an upper bound on $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ that is with high probability at most $\widetilde{O}((n/q)^{d/4-1/2})\cdot\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$. Curiously, this matches our approximation guarantee for the case of arbitrary polynomials with non-negative coefficients. This problem was also studied for the case of sparse random polynomials in [RRS16] motivated by applications to refuting random CSPs.

3.2 Overview of Proofs and Techniques

In the interest of clarity, we shall present all techniques for the special case where $f$ is an arbitrary degree-$4$ homogeneous polynomial. We shall further assume that $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}=f_{\max}$ just so that $\Lambda\left(f\right)$ is a relaxation of $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$. Summarily, the goal of this section is to give an overview of an $O(n/q)$-approximation of $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$, i.e.,

[TABLE]

Many of the high level ideas remain the same when considering higher degree polynomials and special classes like polynomials with non-negative coefficients, or sparse polynomials.

3.2.1 Warmup: $(n^{2}/q^{2})$-Approximation

We begin with seeing how to analyze constant levels of the $\Lambda\left(\cdot\right)$ relaxation and will then move onto higher levels in the next section. The level-$4$ relaxation actually achieves an $n$-approximation, however we will start with $n^{2}$ as a warmup and cover the $n$-approximation a few sections later.

$n^{2}$-Approximation using level-$4$ relaxation

We shall establish that $\Lambda\left(f\right)\leq O(n^{2})\cdot\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$. Let $M_{f}$ be the SoS-symmetric representation of $f$, let $x_{i_{1}}x_{i_{2}}x_{i_{3}}x_{i_{4}}$ be the monomial whose coefficient in $f$ has the maximum magnitude, and let $B$ be the magnitude of this coefficient. Now by Gershgorin circle theorem, we have $\Lambda\left(f\right)\leq\|M_{f}\|_{2}\leq n^{2}\cdot B$.

It remains to establish $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}=\Omega(B)$. To this end, define the decoupled polynomial $\mathcal{F}(x,y,z,t):={(x\otimes y)}^{T}\cdot M_{f}\cdot(z\otimes t)$ and define the decoupled two-norm as

[TABLE]

It is well known that $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}=\Theta(\mathchoice{\lVert\mathcal{F}\rVert_{\infty}}{\lVert\mathcal{F}\rVert_{\infty}}{\lVert\mathcal{F}\rVert_{\infty}}{\lVert\mathcal{F}\rVert_{\infty}})$ (see Lemma˜3.2.1). Thus, we have,

[TABLE]

In order to better analyze $\Lambda\left(f^{q/4}\right)^{4/q}$ we will need to introduce some new techniques.

$(n^{2}/q^{2})$-Approximation Assuming Theorem˜3.1.5

We will next show that $\Lambda\left(f^{q/4}\right)^{4/q}\leq O(n^{2}/q^{2})\cdot\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ (for $q$ divisible by $4$). In fact, one can show something stronger, namely that for every homogeneous polynomial $g$ of degree-$q$, $\Lambda\left(g\right)\leq 2^{O(q)}\cdot{(n/q)}^{q/2}\cdot\mathchoice{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}$ which clearly implies the above claim (also note that for the target $O(n^{2}/q^{2})$-approximation to $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$, losses of $2^{O(q)}$ in the estimate of $\mathchoice{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}$ are negligible, while factors of the order $q^{\Omega(q)}$ are crucial).

Given the additional freedom in choice of representation (due to the polynomial having higher degree), a first instinct would be to completely symmetrize, i.e., take the SoS-symmetric representation of $g$, and indeed this works for multilinear $g$ (see [BGG*+*17] for details).

However, the above approach of taking the SoS-symmetric representation breaks down when the polynomial is non-multilinear. To circumvent this issue, we employ Theorem˜3.1.5 which on combining with the aforementioned multilinear polynomial result, yields that for every homogeneous polynomial $g$ of degree-$q$, $\Lambda\left(g\right)\leq{(n/q)}^{q/2}\cdot\mathchoice{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}$. The proofs of Theorem˜3.1.5 and it’s generalizations (that will be required for the $n/q$ approximation), are quite non-trivial and are the most technically involved sections of our upper bound results. We shall next give an outline of the proof of Theorem˜3.1.5.

Reduction to Optimization of Multilinear Polynomials

One of the main techniques we develop in this work, is a way of reducing the optimization problem for general polynomials to that of multilinear polynomials, which does not increase the number of variables. While general techniques for reduction to the multilinear case have been widely used in the literature [KN08, HLZ10, So11a] (known commonly as decoupling/polarization techniques), these reduce the problem to optimizing a multilinear polynomial in $n\cdot d$ variables (when the given polynomial $h$ is of degree $d$). Below is one example:

Lemma 3.2.1** ([HLZ10]).**

Let $\mathcal{A}$ be a SoS-symmetric $d$-tensor and let $h(x):=\left\langle\mathcal{A},x^{\otimes d}\right\rangle$. Then $\|h\|_{2}~\geq~2^{-O(d)}\cdot\max_{\|x^{i}\|=1}\left\langle\mathcal{A},x^{1}\otimes\dots\otimes x^{d}\right\rangle$.

Since we are interested in the improvement in approximation obtained by considering $f^{q/4}$ for a large $q$, applying these would yield a multilinear polynomial in $n\cdot q$ variables. For our analysis, this increase in variables exactly cancels the advantage we obtain by considering $f^{q/4}$ instead of $f$ (i.e., the advantage obtained by using $q$ levels of the SoS hierarchy).

We can uniquely represent a homogeneous polynomial $g$ of degree $q$ as

[TABLE]

where each $G_{2\alpha}$ is a multilinear polynomial and $g_{r}(x)\stackrel{{\scriptstyle\mathrm{def}}}{{=}}\sum_{\left\lvert\alpha\right\rvert=r}x^{2\alpha}\cdot G_{2\alpha}(x)$. We reduce the problem to optimizing $\mathchoice{\lVert G_{2\alpha}\rVert_{\infty}}{\lVert G_{2\alpha}\rVert_{\infty}}{\lVert G_{2\alpha}\rVert_{\infty}}{\lVert G_{2\alpha}\rVert_{\infty}}$ for each of the polynomials $G_{2\alpha}$. More formally, we show that

[TABLE]

As a simple and immediate example of its applicability, Eq.˜3.4 provides a simple proof of a polytime constant factor approximation for optimization over the simplex (actually this case is known to admit a PTAS [dKLP06, dKLS15]). Indeed, observe that a simplex optimization problem for a degree-$q/2$ polynomial in the variable vector $y$ can be reduced to a sphere optimization by substituting $y_{i}=x_{i}^{2}$. Now since every variable present in a monomial has even degree in that monomial, each $G_{2\alpha}$ is constant, which implies a constant factor approximation (dependent on $q$) on applying Eq.˜3.4.

Returning to our overview of the proof, note that given representations of each of the polynomials $G_{2\alpha}$, each of the polynomials $g_{r}$ can be represented as a block-diagonal matrix with one block corresponding to each $\alpha$. Combining this with triangle inequality and the fact that the maximum eigenvalue of a block-diagonal matrix is equal to the maximum eigenvalue of one of the blocks, gives the following inequality:

[TABLE]

We can further strengthen Eq.˜3.5 by averaging the “best” representation of $G_{2\alpha}$ over $|\mathscr{O}{\,\!\left({\alpha}\right)}|$ diagonal-blocks which all correspond to $x^{2\alpha}$. We show (see [BGG*+*17])

[TABLE]

Since $|\mathscr{O}{\,\!\left({\alpha}\right)}|$ can be as large as $q^{\Omega(q)}$, the above strengthening is crucial. We then prove the following inequality, which shows that the decomposition in Section˜3.2.1 not only gives a block-diagonal decomposition for matrix representations of $g$, but can in fact be thought of as a “decomposition” of the tensor corresponding to $g$ (with regard to computing $\mathchoice{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}$). We show that

[TABLE]

The above inequality together with Eq.˜3.6, implies Eq.˜3.4.

A new weak decoupling lemma

Recall that the expansion of $g(x)$ in Section˜3.2.1, contains the term $x^{2\alpha}\cdot G_{2\alpha}(x)$. The key part of proving the bound in Eq.˜3.7 is to show the following “weak decoupling” result for $x^{2\alpha}$ and $G_{2\alpha}$.

[TABLE]

The proof of Eq.˜3.7 can then be completed by considering the unit vector $y:=\sqrt{\alpha}/\sqrt{|\alpha|}$, i.e. $y:=\sum_{i\in[n]}\frac{\sqrt{\alpha_{i}}}{\sqrt{|\alpha|}}\cdot e_{i}$. A careful calculation shows that $y^{2\alpha}\geq 2^{-O(q)}/\left|{\mathscr{O}{\,\!\left({\alpha}\right)}}\right|$ which finishes the proof.

The primary difficulty in establishing the above decoupling is the possibility of cancellations. To see this, let $x^{*}$ be the vector realizing $\|G_{2\alpha}\|_{2}$ and substitute $z=(x^{*}+y)$ into $g$. Clearly, $y^{2\alpha}\cdot G_{2\alpha}(x^{*})$ is a term in the expansion of $g(z)$, however there is no guarantee that the other terms in the expansion don’t cancel out this value. To fix this our proof relies on multiple delicate applications of the first-moment method, i.e., we consider a complex vector random variable $Z(x^{*},y)$ that is a function of $x^{*}$ and $y$, and argue about $\mathchoice{{\mathbb{E}}\left[|g(Z)|\right]}{{\mathbb{E}}[|g(Z)|]}{{\mathbb{E}}[|g(Z)|]}{{\mathbb{E}}[|g(Z)|]}$.

The extremal case of $\alpha=0^{n}$. We first consider the extremal case of $\alpha=0^{n}$, where we define $y^{2\alpha}=1$. This amounts to showing that for every homogeneous polynomial $h$ of degree $t$, $\mathchoice{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}{\lVert h\rVert_{\infty}}\geq\mathchoice{\lVert h_{m}\rVert_{\infty}}{\lVert h_{m}\rVert_{\infty}}{\lVert h_{m}\rVert_{\infty}}{\lVert h_{m}\rVert_{\infty}}\cdot 2^{-O(t)}$ where $h_{m}$ is the restriction of $h$ to its multilinear monomials.

Given the optimizer $x^{*}$ of $\mathchoice{\lVert h_{m}\rVert_{\infty}}{\lVert h_{m}\rVert_{\infty}}{\lVert h_{m}\rVert_{\infty}}{\lVert h_{m}\rVert_{\infty}}$, let $z$ be a random vector such that each $Z_{i}=x_{i}^{*}$ with probability $p$ and $Z_{i}=0$ otherwise. Then, $\mathchoice{{\mathbb{E}}\left[h(Z)\right]}{{\mathbb{E}}[h(Z)]}{{\mathbb{E}}[h(Z)]}{{\mathbb{E}}[h(Z)]}$ is a univariate degree-$t$ polynomial in $p$ with the coefficient of $p^{t}$ equal to $h_{m}(x^{*})$. An application of Chebyshev’s extremal polynomial inequality then gives that there exists a value of the probability $p$ such that

[TABLE]

For the case of general $\alpha$, we first pass to the complex version of $\mathchoice{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}$ defined as

[TABLE]

We use another averaging argument together with an application of the polarization lemma (Lemma˜3.2.1) to show that we do not lose much by considering ${\lVert g\rVert^{c}_{\infty}}$. In particular, $\mathchoice{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}~\leq~{\lVert g\rVert^{c}_{\infty}}~\leq~2^{O(q)}\cdot\mathchoice{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}{\lVert g\rVert_{\infty}}$.

The extremal case of $g=g_{r}$. In this case, the problem reduces to showing that for all $\alpha\in{\mathbb{N}}_{r}^{n}$ and for all $y\in\mathbb{S}^{n-1}$,

[TABLE]

Fix any $\alpha\in{\mathbb{N}}_{r}^{n}$, and let $\omega\in\mathbb{C}^{n}$ be a complex vector random variable, such that $\omega_{i}$ is an independent and uniformly random $(2\alpha_{i}+1)$-th root of unity. Let $\Xi$ be a random $(q-2r+1)$-th root of unity, and let $x^{*}$ be the optimizer of $\|G_{2\alpha}\|_{2}$. Let $Z:=\omega\circ y+\Xi\cdot x^{*}$, where $\omega\circ y$ denotes the coordinate-wise product. Observe that for any $\alpha^{\prime},\gamma$ such that $|\alpha^{\prime}|=r,~|\gamma|=q-2r,~\gamma\leq\mathds{1}$,

[TABLE]

By linearity, this implies $\mathchoice{{\mathbb{E}}\left[\prod_{i}\omega_{i}\cdot\Xi\cdot g_{r}(Z)\right]}{{\mathbb{E}}[\prod_{i}\omega_{i}\cdot\Xi\cdot g_{r}(Z)]}{{\mathbb{E}}[\prod_{i}\omega_{i}\cdot\Xi\cdot g_{r}(Z)]}{{\mathbb{E}}[\prod_{i}\omega_{i}\cdot\Xi\cdot g_{r}(Z)]}=y^{2\alpha}\cdot G_{2\alpha}(x^{*})$. The claim then follows by noting that

[TABLE]

The general case. The two special cases considered here correspond to the cases when we need to extract a specific $g_{r}$ (for $r=0$), and when we need to extract a fixed $\alpha$ from a given $g_{r}$. The argument for the general case uses a combination of the arguments for both these cases. Moreover, to get an $O(n/q)$ approximation, we also need versions of such decoupling lemmas for folded polynomials to take advantage of “easy substructures” as described next.

3.2.2 Exploiting Easy Substructures via Folding and Improved Approximations

To obtain the desired $n/q$-approximation to $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$, we need to use the fact that the problem of optimizing quadratic polynomials can be solved in polynomial time, and moreover that SoS captures this. More generally, in this section we consider the problem of getting improved approximations when a polynomial contains “easy substructures”. It is not hard to obtain improved guarantees when considering constant levels of SoS. The second main technical contribution of our work is in giving sufficient conditions under which higher levels of SoS improve on the approximation of constant levels of SoS, when considering the optimization problem over polynomials containing “easy substructures”.

As a warmup, we shall begin with seeing how to exploit easy substructures at constant levels by considering the example of degree-$4$ polynomials that trivially “contain” quadratics.

$n$-Approximation using Degree-$4$ SoS

Given a degree-$4$ homogeneous polynomial $f$ (assume $f$ is multilinear for simplicity), we consider a degree-$(2,2)$ folded polynomial $h$, whose unfolding yields $f$, chosen so that $\max_{\|y\|=1}\mathchoice{\lVert h(y)\rVert_{\infty}}{\lVert h(y)\rVert_{\infty}}{\lVert h(y)\rVert_{\infty}}{\lVert h(y)\rVert_{\infty}}=\Theta(\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}})$ (recall that an evaluation of a folded polynomial returns a polynomial, i.e., for a fixed $y$, $h(y)$ is a quadratic polynomial in the indeterminate $x$). Such a $h$ always exists and is not hard to find based on the SoS-symmetric representation of $f$. Also recall,

[TABLE]

where each $\mkern 1.2mu\overline{\mkern-1.2muh\mkern-1.2mu}\mkern 1.2mu_{\beta}$ is a quadratic polynomial (the aforementioned phrase “easy substructures” is referencing the folds: $\mkern 1.2mu\overline{\mkern-1.2muh\mkern-1.2mu}\mkern 1.2mu_{\beta}$ which are easy to optimize). Now by assumption we have,

[TABLE]

We then apply the block-matrix generalization of Gershgorin circle theorem to the SoS-symmetric matrix representation of $f$ to show that

[TABLE]

where the last step uses the fact that $\mkern 1.2mu\overline{\mkern-1.2muh\mkern-1.2mu}\mkern 1.2mu_{\beta}$ is a quadratic, and $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ is a tight relaxation of $\mathchoice{\lVert\cdot\rVert_{\infty}}{\lVert\cdot\rVert_{\infty}}{\lVert\cdot\rVert_{\infty}}{\lVert\cdot\rVert_{\infty}}$ for quadratics. This yields the desired $n$-approximation.

$n/q$-approximation using Degree-$q$ SoS

Following the cue of the $n^{2}/q^{2}$-approximation, we derive the desired $n/q$ bound by proving a folded-polynomial analogue of every claim in the previous section (including the multilinear reduction tools), a notable difference being that when we consider a power $f^{q/4}$ of $f$, we need to consider degree-$(q-2q/4,2q/4)$ folded polynomials, since we want to use the fact that any product of $q/4$ quadratic polynomials is “easy” for SoS (in contrast to Section˜3.2.2 where we only used the fact quadratic polynomials are easy for SoS). We now state an abstraction of the general approach we use to leverage the tractability of the folds.

Conditions for Exploiting “Easy Substructures” at Higher Levels of SoS. Let $d:=d_{1}+d_{2}$ and $f:=\mathsf{U}(h)$ where $h$ is a degree-$(d_{1},d_{2})$ folded polynomial that satisfies

[TABLE]

Implicit in our work (see [BGG*+*17]), is the following theorem we believe to be of independent interest:

Theorem 3.2.2**.**

Let $h,f$ be as above, and let

[TABLE]

Then for any $q$ divisible by $2d$,

[TABLE]

In other words, if degree-$d_{2}$ SoS gives a good approximation for every polynomial in the subspace spanned by the folds of $h$, then higher levels of SoS give an improving approximation that exploits this. In this work, we only apply the above with $\Gamma=1$, where exact optimization is easy for the space spanned by the folds.

While we focused on general polynomials for the overview, let us remark that in the case of polynomials with non-negative coefficients, the approximation factor in Theorem˜3.2.2 becomes $O_{d}\left(\delta\cdot{(n/q)}^{d_{1}/4}\right)$.

3.2.3 Lower Bounds for Polynomials with Non-negative Coefficients

Degree-4 Lower Bound for Polynomials with Non-Negative Coefficients

We discuss some of the important ideas from the proof of Theorem˜3.1.6. The lower bound proved by a subset of the authors in [BGL16] proves a large ratio $\frac{\Lambda\left(f\right)}{\|f\|_{2}}$ by considering a random polynomial $f$ where each coefficient of $f$ is an independent (Gaussian) random variable with bounded variance. The most natural adaptation of the above strategy to degree-$4$ polynomials with non-negative coefficients is to consider a random polynomial $f$ where each coefficient $f_{\alpha}$ is independently sampled such that $f_{\alpha}=1$ with probability $p$ and $f_{\alpha}=0$ with probability $1-p$. However, this construction fails for every choice of $p$. If we let $\mathsf{A}\in\mathbb{R}^{{[n]}^{2}\times{[n]}^{2}}$ be the natural matrix representation of $f$ (i.e., each coefficient $f_{\alpha}$ is distributed uniformly among the corresponding entries of $\mathsf{A}$), the Perron-Frobenius theorem shows that $\mathchoice{\lVert\mathsf{A}\rVert_{2}}{\lVert\mathsf{A}\rVert_{2}}{\lVert\mathsf{A}\rVert_{2}}{\lVert\mathsf{A}\rVert_{2}}$ is less than the maximum row sum $\max(\tilde{O}(n^{2}p),1)$ of $\mathsf{M}$, which is also an upper bound on $\Lambda\left(f\right)$. However, we can match this bound by (within constant factors) choosing $x=(\frac{1}{\sqrt{n}},\dots,\frac{1}{\sqrt{n}})$ when $p\geq 1/n^{2}$. Also, when $p<1/n^{2}$, we can take any $\alpha$ with $f_{\alpha}=1$ and set $x_{i}=1/2$ for all $i$ with $\alpha_{i}>0$, which achieves a value of $1/16$.

We introduce another natural distribution of random non-negative polynomials that bypasses this problem. Let $G=(V,E)$ be a random graph drawn from the distribution $G_{n,p}$ (where we choose $p=n^{-1/3}$ and $V=[n]$). Let $\mathcal{C}\subseteq\binom{V}{4}$ be the set of $4$-cliques in $G$. The polynomial $f$ is defined as

[TABLE]

Instead of trying $\Theta(n^{4})$ $p$-biased random bits, we use $\Theta(n^{2})$ of them. This limited independence bypasses the problem above, since the rows of $\mathsf{A}$ now have significantly different row sums: $\Theta(n^{2}p)$ rows that correspond to an edge of $G$ will have row sum $\Theta(n^{2}p^{5})$, and all other rows will be zeros. Since these $n^{2}p$ rows (edges) are chosen independently of $\binom{[n]}{2}$, they still reveal little information that can be exploited to find an $n$-dimensional vector $x$ with large $f(x)$. However, the proof requires a careful analysis of the trace method (to bound the spectral norm of an “error” matrix).

It is simple to prove that $\|f\|_{sp}\geq\Omega\left(\sqrt{n^{2}p^{5}}\right)=\Omega(n^{1/6})$ by considering the Frobenius norm of the $n^{2}p\times n^{2}p$ principal submatrix, over any matrix representation (indeed, $\mathsf{A}$ is the minimizer). To prove $\Lambda\left(f\right)\geq\tilde{\Omega}(n^{1/6})$, we construct a moment matrix $\mathsf{M}$ that is SoS-symmetric, positive semidefinite, and has a large $\langle\mathsf{M},\mathsf{A}\rangle$ (see the dual form of $\Lambda\left(f\right)$ in [BGG*+*17]). It turns out that the $n^{2}p\times n^{2}p$ submatrix of $\mathsf{A}$ shares spectral properties of the adjacency matrix of a random graph $G_{n^{2}p,p^{4}}$, and taking $\mathsf{M}:=c_{1}\mathsf{A}+c_{2}\mathsf{I}$ for some identity-like matrix $\mathsf{I}$ proves $\Lambda\left(f\right)\geq\tilde{\Omega}(n^{1/6})$. An application of the trace method is needed to bound $c_{2}$.

To upper bound $\|f\|_{2}$, we first observe that $\|f\|_{2}$ is the same as the following natural combinatorial problem up to an $O(\log^{4}n)$ factor: find four sets $S_{1},S_{2},S_{3},S_{4}\subseteq V$ that maximize

[TABLE]

where $|\mathcal{C}_{G}(S_{1},S_{2},S_{3},S_{4})|$ is the number of $4$-cliques $\{v_{1},\dots,v_{4}\}$ of $G$ with $v_{i}\in S_{i}$ for $i=1,\dots,4$. The number of copies of a fixed subgraph $H$ in $G_{n,p}$, including its tail bound, has been actively studied in probabilistic combinatorics [Vu01, KV04, JOR04, Cha12, DK12a, DK12b, LZ16], though we are interested in bounding the $4$-clique density of every $4$-tuple of subsets simultaneously. The previous results give a strong enough tail bound for a union bound, to prove that the optimal value of the problem is $O(n^{2}p^{6}\cdot\log^{O(1)}n)$ when $\left\lvert S_{1}\right\rvert=\cdots=\left\lvert S_{4}\right\rvert$, but this strategy inherently does not work when the set sizes become significantly different. However, we give a different analysis for the above asymmetric case, showing that the optimum is still no more than $O(n^{2}p^{6}\cdot\log^{O(1)}n)$.

Lifting Stable Degree-$4$ Lower Bounds

For a degree-$t$ ($t$ even) homogeneous polynomial $f$, note that $\max\{|\Lambda\left(f\right)|,|\Lambda\left(-f\right)|\}$ is a relaxation of $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$. $\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}$ is a slightly weaker (but still quite natural) relaxation of $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ given by

[TABLE]

As in the case of $\Lambda\left(f\right)$, for a (say) degree-$4$ polynomial $f$, ${\lVert f^{q/4}\rVert^{4/q}_{\mathrm{sp}}}$ gives a hierarchy of relaxations for $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$, for increasing values of $q$.

We give an overview of a general method of “lifting” certain “stable” low degree gaps for $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ to gaps for higher levels with at most $q^{O(1)}$ loss in the gap. While we state our techniques for lifting degree-$4$ gaps, all the ideas are readily generalized to higher levels. We start with the observation that the dual of $\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}$ is given by the following “nuclear norm” program. Here $\mathsf{M}_{f}$ the canonical matrix representation of $f$, and $\mathchoice{\lVert\mathsf{X}\rVert_{S_{1}}}{\lVert\mathsf{X}\rVert_{S_{1}}}{\lVert\mathsf{X}\rVert_{S_{1}}}{\lVert\mathsf{X}\rVert_{S_{1}}}$ is the Schatten $1$-norm (nuclear norm) of $X$, which is the sum of its singular values.

[TABLE]

Now let $\mathsf{X}$ be a solution realizing a gap of $\delta$ between $\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}$ and $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$. We shall next see how assuming reasonable conditions on $\mathsf{X}$ and $M_{f}$, one can show that $\|f^{q/4}\|_{sp}/\|f^{q/4}\|_{2}$ is at least $\delta^{q/4}/q^{O(q)}$.

In order to give a gap for the program corresponding to $\mathchoice{\lVert f^{q/4}\rVert_{\mathrm{sp}}}{\lVert f^{q/4}\rVert_{\mathrm{sp}}}{\lVert f^{q/4}\rVert_{\mathrm{sp}}}{\lVert f^{q/4}\rVert_{\mathrm{sp}}}$, a natural choice for a solution is the symmetrized version of the matrix $\mathsf{X}^{\otimes q/4}$ normalized by its Schatten-$1$ norm i.e., for $Y=\mathsf{X}^{\otimes q/4}$, we take

[TABLE]

To get a good gap, we are now left with showing that $\mathchoice{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}$ is not too large. Note that symmetrization can drastically change the spectrum of a matrix as for different permutations $\pi$, the matrices $Y^{\pi}[K]\stackrel{{\scriptstyle\mathrm{def}}}{{=}}Y[\pi(K)]$ can have very different ranks (while $\mathchoice{\lVert Y\rVert_{F}}{\lVert Y\rVert_{F}}{\lVert Y\rVert_{F}}{\lVert Y\rVert_{F}}=\mathchoice{\lVert Y^{\pi}\rVert_{F}}{\lVert Y^{\pi}\rVert_{F}}{\lVert Y^{\pi}\rVert_{F}}{\lVert Y^{\pi}\rVert_{F}}$). In particular, symmetrization can increase the maximum eigenvalue of a matrix by polynomial factors, and thus one must carefully count the number of such large eigenvalues in order to get a good upper bound on $\mathchoice{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}$. Such an upper bound is a consequence of a structural result about $Y^{S}$ that we believe to be of independent interest.

To state the result, we will first need some notation. For a matrix $M\in\Re^{{[n]}^{2}\times{[n]}^{2}}$ let $T\in\mathbb{R}^{{[n]}^{4}}$ denote the tensor given by, $T[i_{1},i_{2},i_{3},i_{4}]=M[(i_{1},i_{2}),(i_{3},i_{4})]$. Also for any non-negative integers $x,y$ satisfying $x+y=4$, let $M_{x,y}\in\Re^{{[n]}^{x}\times{[n]}^{y}}$ denote the (rectangular) matrix given by, $M[(i_{1},\dots,i_{x}),(j_{1},\dots j_{y})]=T[i_{1},\dots,i_{x},j_{1},\dots j_{y}]$. Let $M\in\Re^{{[n]}^{2}\times{[n]}^{2}}$ be a degree-$4$ SoS-Symmetric matrix, let $M_{A}:=M_{1,3}\otimes M_{4,0}\otimes M_{1,3}$, let $M_{B}:=M_{1,3}\otimes M_{3,1}$, let $M_{C}:=M$ and let $M_{D}:=\mathop{\mathsf{Vec}}\!\left({M}\right)\mathop{\mathsf{Vec}}\!\left({M}\right)^{T}=M_{0,4}\otimes M_{4,0}$.

We show (see [BGG*+*17]) that ${(M^{\otimes q/4})}^{S}$ can be written as the sum of $2^{O(q)}$ terms of the form:

[TABLE]

where $12a+8b+4c+8d=q$, $P$ is a matrix with spectral norm $1$ and $C(a,b,c,d)=2^{O(q)}$. This implies that controlling the spectrum of $M_{A},M_{B},M$ and $M_{D}$ is sufficient to control on the spectrum of ${(M^{\otimes q/4})}^{S}$.

Using this result with $M:=\mathsf{X}$, we are able to establish that if $\mathsf{X}$ satisfies the additional condition of $\mathchoice{\lVert\mathsf{X}_{1,3}\rVert_{S_{1}}}{\lVert\mathsf{X}_{1,3}\rVert_{S_{1}}}{\lVert\mathsf{X}_{1,3}\rVert_{S_{1}}}{\lVert\mathsf{X}_{1,3}\rVert_{S_{1}}}\leq 1$ (note that we already know $\mathchoice{\lVert X\rVert_{S_{1}}}{\lVert X\rVert_{S_{1}}}{\lVert X\rVert_{S_{1}}}{\lVert X\rVert_{S_{1}}}\leq 1$), then $\mathchoice{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}{\lVert Y^{S}\rVert_{S_{1}}}=2^{O(q)}$. Thus $\mathsf{Z}$ realizes a $\left\langle M_{f}^{\otimes q/4},Y^{S}\right\rangle/2^{O(q)}$ gap for $\mathchoice{\lVert f^{q/4}\rVert_{\mathrm{sp}}}{\lVert f^{q/4}\rVert_{\mathrm{sp}}}{\lVert f^{q/4}\rVert_{\mathrm{sp}}}{\lVert f^{q/4}\rVert_{\mathrm{sp}}}$. On composing this result with the degree-$4$ gap from the previous section, we obtain an $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ gap of $n^{q/24}/\left(q\cdot\log n\right)^{O(q)}$ for degree-$q$ polynomials with non-neg. coefficients. We also show the $q$-th level $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ gap for degree-$4$ polynomial with non-neg. coefficients is $\tilde{\Omega}(n^{1/6})/q^{O(1)}$.

Even though we only derive results for the weaker relaxation $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$, the structural result above can be used to lift “stable” low-degree SoS lower bounds as well (i.e., gaps for $\Lambda\left(\cdot\right)$), albeit with a stricter notion of stability (see [BGG*+*17]). However, the problem of finding such stable SoS lower bounds remains open.

There are by now quite a few results giving near-tight lower bounds on the performance of higher level SoS relaxations for average-case problems [BHK*+*16, KMOW17b, HKP*+*17]. However, there are few examples in the literature of matching SoS upper/lower bounds on worst-case problems. We believe our lifting result might be especially useful in such contexts, where in order to get higher degree lower bounds, it suffices to give stable lower bounds for constant degree SoS.

3.3 Preliminaries on the SoS Hierarchy for Polynomials

3.3.1 Pseudoexpectations and Moment Matrices

Let ${\Re[x]}_{\leq q}$ be the vector space of polynomials with real coefficients in variables $x=(x_{1},\ldots,x_{n})$, of degree at most $q$. For an even integer $q$, the degree-$q$ pseudo-expectation operator is a linear operator $\widetilde{\mathbb{E}}:{\Re[x]}_{\leq q}\mapsto\Re$ such that

$\mathchoice{{\widetilde{\mathbb{E}}}\left[1\right]}{{\widetilde{\mathbb{E}}}[1]}{{\widetilde{\mathbb{E}}}[1]}{{\widetilde{\mathbb{E}}}[1]}=1$ for the constant polynomial $1$. 2. 2.

$\mathchoice{{\widetilde{\mathbb{E}}}\left[p_{1}+p_{2}\right]}{{\widetilde{\mathbb{E}}}[p_{1}+p_{2}]}{{\widetilde{\mathbb{E}}}[p_{1}+p_{2}]}{{\widetilde{\mathbb{E}}}[p_{1}+p_{2}]}=\mathchoice{{\widetilde{\mathbb{E}}}\left[p_{1}\right]}{{\widetilde{\mathbb{E}}}[p_{1}]}{{\widetilde{\mathbb{E}}}[p_{1}]}{{\widetilde{\mathbb{E}}}[p_{1}]}+\mathchoice{{\widetilde{\mathbb{E}}}\left[p_{2}\right]}{{\widetilde{\mathbb{E}}}[p_{2}]}{{\widetilde{\mathbb{E}}}[p_{2}]}{{\widetilde{\mathbb{E}}}[p_{2}]}$ for any polynomials $p_{1},p_{2}\in{\Re[x]}_{\leq q}$. 3. 3.

$\mathchoice{{\widetilde{\mathbb{E}}}\left[p^{2}\right]}{{\widetilde{\mathbb{E}}}[p^{2}]}{{\widetilde{\mathbb{E}}}[p^{2}]}{{\widetilde{\mathbb{E}}}[p^{2}]}\geq 0$ for any polynomial $p\in{\Re[x]}_{\leq q/2}$.

The pseudo-expectation operator $\widetilde{\mathbb{E}}$ can be described by a moment matrix $\widehat{\mathsf{M}}\in\mathbb{R}^{{\mathbb{N}}_{\!\leq q/2}^{n}\times{\mathbb{N}}_{\!\leq q/2}^{n}}$ defined as $\widehat{\mathsf{M}}[\alpha,\beta]=\mathchoice{{\widetilde{\mathbb{E}}}\left[x^{\alpha+\beta}\right]}{{\widetilde{\mathbb{E}}}[x^{\alpha+\beta}]}{{\widetilde{\mathbb{E}}}[x^{\alpha+\beta}]}{{\widetilde{\mathbb{E}}}[x^{\alpha+\beta}]}$ for $\alpha,\beta\in{\mathbb{N}}_{\!\leq q/2}^{n}$.

For each fixed $t\leq q/2$, we can also consider the principal minor of $\widehat{\mathsf{M}}$ indexed by $\alpha,\beta\in{\mathbb{N}}_{t}^{n}$. This also defines a matrix $M\in{\mathbb{R}}^{{[n]}^{t}\times{[n]}^{t}}$ with $M[I,J]=\mathchoice{{\widetilde{\mathbb{E}}}\left[x^{\alpha({I})+\alpha({J})}\right]}{{\widetilde{\mathbb{E}}}[x^{\alpha({I})+\alpha({J})}]}{{\widetilde{\mathbb{E}}}[x^{\alpha({I})+\alpha({J})}]}{{\widetilde{\mathbb{E}}}[x^{\alpha({I})+\alpha({J})}]}$. Note that this new matrix $M$ satisfies $M[I,J]=M[K,L]$ whenever $\alpha({I})+\alpha({J})=\alpha({K})+\alpha({L})$. Recall that a matrix in ${\mathbb{R}}^{{[n]}^{t}\times{[n]}^{t}}$ with this symmetry is said to be SoS-symmetric .

We will use the following facts about the operator $\widetilde{\mathbb{E}}$ given by the SoS hierarchy.

Claim 3.3.1** (Pseudo-Cauchy-Schwarz [BKS14]).**

$\mathchoice{{\widetilde{\mathbb{E}}}\left[p_{1}p_{2}\right]}{{\widetilde{\mathbb{E}}}[p_{1}p_{2}]}{{\widetilde{\mathbb{E}}}[p_{1}p_{2}]}{{\widetilde{\mathbb{E}}}[p_{1}p_{2}]}\leq{(\mathchoice{{\widetilde{\mathbb{E}}}\left[p_{1}^{2}\right]}{{\widetilde{\mathbb{E}}}[p_{1}^{2}]}{{\widetilde{\mathbb{E}}}[p_{1}^{2}]}{{\widetilde{\mathbb{E}}}[p_{1}^{2}]}\mathchoice{{\widetilde{\mathbb{E}}}\left[p_{2}^{2}\right]}{{\widetilde{\mathbb{E}}}[p_{2}^{2}]}{{\widetilde{\mathbb{E}}}[p_{2}^{2}]}{{\widetilde{\mathbb{E}}}[p_{2}^{2}]})}^{1/2}$* for any $p_{1},p_{2}$ of degree at most $q/2$.*

Constrained Pseudoexpectations

For a system of polynomial constraints $C=\left\{f_{1}=0,\ldots,f_{m}=0,g_{1}\geq 0,\ldots,g_{r}\geq 0\right\}$, we say $\widetilde{\mathbb{E}}_{C}$ is a pseudoexpectation operator respecting $C$, if in addition to the above conditions, it also satisfies

$\mathchoice{{\widetilde{\mathbb{E}}_{C}}\left[p\cdot f_{i}\right]}{{\widetilde{\mathbb{E}}_{C}}[p\cdot f_{i}]}{{\widetilde{\mathbb{E}}_{C}}[p\cdot f_{i}]}{{\widetilde{\mathbb{E}}_{C}}[p\cdot f_{i}]}=0$, $\forall i\in[m]$ and $\forall p$ such that $\deg(p\cdot f_{i})\leq q$. 2. 2.

$\mathchoice{{\widetilde{\mathbb{E}}_{C}}\left[p^{2}\cdot\prod_{i\in S}g_{i}\right]}{{\widetilde{\mathbb{E}}_{C}}[p^{2}\cdot\prod_{i\in S}g_{i}]}{{\widetilde{\mathbb{E}}_{C}}[p^{2}\cdot\prod_{i\in S}g_{i}]}{{\widetilde{\mathbb{E}}_{C}}[p^{2}\cdot\prod_{i\in S}g_{i}]}\geq 0$, $\forall S\subseteq[r]$ and $\forall p$ such that $\deg(p^{2}\cdot\prod_{i\in S}g_{i})\leq q$.

It is well-known that such constrained pseudoexpectation operators can be described as solutions to semidefinite programs of size $n^{O(q)}$ [BS14, Lau09]. This hierarchy of semidefinite programs for increasing $q$ is known as the SoS hierarchy.

3.3.2 Matrix Representations of Polynomials and Relaxations

For a homogeneous polynomial $f$ of even degree $d$, we say a matrix $M\in\Re^{{[n]}^{d/2}\times{[n]}^{d/2}}$ is a degree-$d$ matrix representation of $f$ if for all $x$, $f(x)={(x^{\otimes d/2})}^{T}\cdot M\cdot x^{\otimes d/2}$. Recall that we consider the semidefinite program for optimizing the quantity $\Lambda\left(f\right)$, which is a relaxation for $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ when $f\geq 0$. Let $\mathsf{M}_{f}\in{\mathbb{R}}^{n^{d/2}\times n^{d/2}}$ denote the unique SoS-symmetric matrix representation of $f$. The Fig.˜3.1 gives the primal and dual forms of the relaxation computing $\Lambda\left(f\right)$. It is easy to check that strong duality holds in this case, since the solution $\mathchoice{{\widetilde{\mathbb{E}}_{C}}\left[x^{\alpha}\right]}{{\widetilde{\mathbb{E}}_{C}}[x^{\alpha}]}{{\widetilde{\mathbb{E}}_{C}}[x^{\alpha}]}{{\widetilde{\mathbb{E}}_{C}}[x^{\alpha}]}={(1/\sqrt{n})}^{\left\lvert\alpha\right\rvert}$ for all $\alpha\in{\mathbb{N}}_{\!\leq d}^{n}$, is strictly feasible and in the relative interior of the domain. Thus, the objective values of the two programs are equal.

We will also consider a weaker relaxation of $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$, which we denote by $\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}$. A somewhat weaker version of this was used as the reference value in the work of [BKS14]. The Fig.˜3.2 gives the primal and dual forms of this relaxation.

We will also need to consider constraint sets $C=\left\{{\lVert x\rVert^{2}_{2}}=1,x^{\beta_{1}}\geq 0,\ldots,x^{\beta_{m}}\geq 0\right\}$. We refer to the non-negativity constraints here as moment non-negativity constraints . When considering the maximum of $\mathchoice{{\widetilde{\mathbb{E}}_{C}}\left[f\right]}{{\widetilde{\mathbb{E}}_{C}}[f]}{{\widetilde{\mathbb{E}}_{C}}[f]}{{\widetilde{\mathbb{E}}_{C}}[f]}$, for constraint sets $C$ containing moments non-negativity constraints in addition to ${\lVert x\rVert^{2}_{2}}=1$, we refer to the optimum value as $\Lambda_{C}\left(f\right)$. Note that the maximum is still taken over degree-$d$ pseudoexpectations. Also, strong duality still holds in this case since $\mathchoice{{\widetilde{\mathbb{E}}_{C}}\left[x^{\alpha}\right]}{{\widetilde{\mathbb{E}}_{C}}[x^{\alpha}]}{{\widetilde{\mathbb{E}}_{C}}[x^{\alpha}]}{{\widetilde{\mathbb{E}}_{C}}[x^{\alpha}]}={(1/\sqrt{n})}^{\left\lvert\alpha\right\rvert}$ is still a strictly feasible solution.

Properties of Relaxations Obtained from Constrained Pseudoexpectations

We use the following claim, which is an easy consequence of the fact that the sum-of-squares algorithm can produce a certificate of optimality (see [OZ13]). In particular, if $\max_{\widetilde{\mathbb{E}}_{C}}\mathchoice{{\widetilde{\mathbb{E}}_{C}}\left[f\right]}{{\widetilde{\mathbb{E}}_{C}}[f]}{{\widetilde{\mathbb{E}}_{C}}[f]}{{\widetilde{\mathbb{E}}_{C}}[f]}=\Lambda_{C}\left(f\right)$ for a degree-$q_{1}$ pseudoexpectation operator respecting $C$ containing ${\lVert x\rVert^{2}_{2}}=1$ and moment non-negativity constraints for $\beta_{1},\ldots,\beta_{m}$, then for every $\lambda>\Lambda_{C}\left(f\right)$, we have that $\lambda-f$ can be certified to be positive by showing that $\lambda-f\in\Sigma^{q_{1}}_{C}$. Here $\Sigma^{(q_{1})}_{C}$ is the set of all expressions of the form

[TABLE]

where each $h_{S}$ is a sum of squares of polynomials and the degree of each term is at most $q_{1}$.

Lemma 3.3.2**.**

Let $\Lambda_{C}\left(f\right)$ denote the maximum of $\mathchoice{{\widetilde{\mathbb{E}}_{C}}\left[f\right]}{{\widetilde{\mathbb{E}}_{C}}[f]}{{\widetilde{\mathbb{E}}_{C}}[f]}{{\widetilde{\mathbb{E}}_{C}}[f]}$ over all degree-$d$ pseudoexpectation operators respecting $C$. Then, for a pseudoexpectation operator of degree $d^{\prime}$ (respecting $C$) and a polynomial $p$ of degree at most $(d^{\prime}-d)/2$, we have that

[TABLE]

• **Proof:  ** As described above, for any $\lambda>\Lambda_{C}\left(f\right)$, we can write $\lambda-f=g$ for $g\in\Sigma^{(d)}_{C}$. Since the degree of each term in $p^{2}\cdot g$ is at most $d^{\prime}$, we have by the properties of pseudoexpectation operators (of degree $d^{\prime}$) that

[TABLE]

The following monotonicity claim for non-negative coefficient polynomials will come in handy in later sections.

Lemma 3.3.3**.**

Let $C$ be a system of polynomial constraints containing $\{\forall\beta\in{\mathbb{N}}_{t}^{n},x^{\beta}\geq 0\}$. Then for any non-negative coefficient polynomials $f$ and $g$ of degree $t$, and such that $f\geq g$ (coefficient-wise, i.e. $f-g$ has non-negative coefficients), we have $\Lambda_{C}\left(f\right)\geq\Lambda_{C}\left(g\right)$.

• **Proof:  ** For any pseudo-expectation operator $\widetilde{\mathbb{E}}_{C}$ respecting $C$, we have $\mathchoice{{\widetilde{\mathbb{E}}_{C}}\left[f-g\right]}{{\widetilde{\mathbb{E}}_{C}}[f-g]}{{\widetilde{\mathbb{E}}_{C}}[f-g]}{{\widetilde{\mathbb{E}}_{C}}[f-g]}\geq 0$ because of the moment non-negativity constraints and by linearity.

So let $\widetilde{\mathbb{E}}_{C}$ be a pseudo-expectation operator realizing $\Lambda_{C}\left(g\right)$. Then we have,

[TABLE]

3.3.3 An additional operation on folded polynomials

We define the following operation (and it’s folded counterpart) which in the case of a multilinear polynomial corresponds (up to scaling) to the sum of a row of the SOS symmetric matrix representation of the polynomial. This will be useful in our result for non-negative polynomials.

Definition 3.3.4** (Collapse).**

Let $f\in{\mathbb{R}}_{d}[x]$ be a polynomial. The $k$- collapse of $f$, denoted as $\mathsf{C}_{k}\!\left({f}\right)$ is the degree $d-k$ polynomial $g$ given by,

[TABLE]

For a degree-$(d_{1},d_{2})$ folded polynomial $f$, we define $\mathsf{C}_{k}\!\left({f}\right)$ similarly as the degree-$(d_{1}-k,d_{2})$ folded polynomial $g$ given by,

[TABLE]

3.4 Intermediate Results for Polynomials in ${\mathbb{R}}_{d}[x]$ and ${\mathbb{R}}^{+}_{d}[x]$

3.4.1 Reduction to Multilinear Polynomials

Lemma 3.4.1**.**

Any homogeneous $n$-variate degree-$d$ polynomial $f(x)$ has a unique representation of the form

[TABLE]

where for any $\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}$, $F_{2\alpha}$ is a homogeneous multilinear degree-$(d-2|\alpha|)$ polynomial.

We would like to approximate $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ by individually approximating $\mathchoice{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}$ for each multilinear polynomial $F_{2\alpha}$. This section will establish the soundness of this goal.

Upper Bounding $\Lambda\left(f\right)$ in terms of $\Lambda\left(F_{2\alpha}\right)$

We first bound $\Lambda\left(f\right)$ in terms of $\max_{\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}}\Lambda\left(F_{2\alpha}\right)$. The basic intuition is that any matrix $M_{f}$ such that $\left(x^{\otimes(d/2)}\right)^{T}\cdot M_{f}\cdot x^{\otimes(d/2)}$ for all $x$ (called a matrix representation of $f$) can be written as a sum of matrices $M_{t,f}$ for each $t\leq d/2$, each of which is block-diagonal matrix with blocks corresponding to matrix representations of the polynomials $M_{F_{2\alpha}}$ for each $\alpha$ with $\left\lvert\alpha\right\rvert=2t$.

Lemma 3.4.2**.**

Consider any homogeneous $n$-variate degree-$d$ polynomial $f(x)$. We have,

[TABLE]

• **Proof:  ** We shall start by constructing an appropriate matrix representation $M_{f}$ of $f$ that will give us the desired upper bound on $\Lambda\left(f\right)$. To this end, for any $\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}$, let $M_{F_{2\alpha}}$ be the matrix representation of $F_{2\alpha}$ realizing $\Lambda\left(F_{2\alpha}\right)$. For any $0\leq t\leq d/2$, we define $M_{(t,f)}$ so that for any $\alpha\in{\mathbb{N}}_{t}^{n}$ and $I\in\mathscr{O}{\,\!\left({\alpha}\right)}$, $M_{(t,f)}[I,I]:=M_{F_{2\alpha}}/|\mathscr{O}{\,\!\left({\alpha}\right)}|$, and $M_{(t,f)}$ is zero everywhere else. Now let $M_{f}:=\sum_{t\in[d/2]}M_{(t,f)}$. As for validity of $M_{f}$ as a representation of $f$ we have,

[TABLE]

Now observe that $M_{(t,f)}$ is a block-diagonal matrix (up to simultaneous permutation of its rows and columns). Thus, we have $\|M_{(t,f)}\|\leq\max_{\alpha\in{\mathbb{N}}_{t}^{n}}\|M_{F_{2\alpha}}\|/|\mathscr{O}{\,\!\left({\alpha}\right)}|$. Thus on applying triangle inequality, we obtain $\|M_{f}\|\leq\max\limits_{\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}}(1+d/2)\,\|M_{F_{2\alpha}}\|/|\mathscr{O}{\,\!\left({\alpha}\right)}|$

Lower Bounding

$\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ in terms of $\mathchoice{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}$ (NNC)

We first bound $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ in terms of $\max_{\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}}\mathchoice{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}$, when every coefficient of $f$ is non-negative. If $x^{*}$ is the optimizer of $F_{2\alpha}$, then it is easy to see that $x^{*}\geq 0$. Setting $y=x^{*}+\frac{\sqrt{\alpha}}{|\alpha|}$ ensures that $\|y\|_{2}\leq 2$ and $f(y)$ is large, since $f(y)$ recovers a significant fraction (up to a $2^{O(d)}\cdot|\mathscr{O}{\,\!\left({\alpha}\right)}|$ factor) of $F_{2\alpha}(x^{*})$.

Lemma 3.4.3**.**

Let $f(x)$ be a homogeneous $n$-variate degree-$d$ polynomial with non-negative coefficients. Consider any $\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}$. Then

[TABLE]

• **Proof:  ** Consider any $0\leq t\leq d/2$, and any $\alpha\in{\mathbb{N}}_{t}^{n}$. Let $x^{*}_{\alpha}:=\mathrm{argmax}\mathchoice{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}$ (note $x^{*}_{\alpha}$ must be non-negative). Let

[TABLE]

and let $x^{*}:=y^{*}/\|y^{*}\|$. The second term is a unit vector since ${\lVert\sqrt{\alpha}\rVert^{2}_{2}}=t$. Thus, $\|y^{*}\|=\Theta(1)$ since $y^{*}$ is the sum of two unit vectors. This implies $f(x^{*})\geq f(y^{*})/2^{O(d)}$. Now we have,

[TABLE]

This completes the proof.

Theorem 3.4.4**.**

Consider any homogeneous $n$-variate degree-$d$ polynomial $f(x)$ with non-negative coefficients. Then

[TABLE]

• **Proof:  ** Combining Lemma˜3.4.2 with Lemma˜3.4.3 yields the claim.

We will next generalize the Theorem˜3.4.4 by proving a more general version of the Lemma˜3.4.3.

Lower Bounding

$\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ in terms of $\mathchoice{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}$ (General Case)

We lower bound $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ in terms of $\mathchoice{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}$ for all polynomials. We will first recollect and establish some polynomial identities that will be used in the proof of the generalized version of the Lemma˜3.4.3 (i.e., Lemma˜3.4.11).

Polynomial Identities

Lemma 3.4.5** (Chebyshev’s Extremal Polynomial Inequality).**

Let $p(x)$ be a univariate degree-$d$ polynomial and let $c_{d}$ be it’s leading coefficient. Then we have, $\max_{x\in[0,1]}|p(x)|\geq 2|c_{d}|/4^{d}$.

Lemma 3.4.6** ([HLZ10]).**

Let $x^{1},x^{2},\dots x^{d}\in\Re^{n}$ be arbitrary, let $\mathcal{A}\in\Re^{{[n]}^{d}}$ be a SoS-symmetric $d$-tensor, and let $\xi_{1},\dots,\xi_{d}$ be independent Rademacher random variables. Then

[TABLE]

This lemma implies:

Lemma 3.4.7** ([HLZ10]).**

Let $\mathcal{A}$ be a SoS-symmetric $d$-tensor and let $f(x):=\left\langle\mathcal{A},x^{\otimes d}\right\rangle$. Then

[TABLE]

Lemma 3.4.8**.**

Let $f$ be an $n$-variate degree-$d$ homogeneous polynomial. Let ${\lVert f\rVert^{c}_{\infty}}:=\max\limits_{\begin{subarray}{c}z\in\mathds{C}^{n}\\ \|z\|=1\end{subarray}}|f(z)|$, then

[TABLE]

• **Proof:  ** Let $\mathcal{A}$ be the SoS-symmetric tensor representing $f$. Let $z^{*}=a^{*}+ib^{*}$ be the complex unit vector realizing $f(z^{*})={\lVert f\rVert^{c}_{\infty}}$. Then we have,

[TABLE]

which implies that there exists $c^{1},\dots,c^{d}\in\{a^{*},b^{*}\}$ such that $|\left\langle\mathcal{A},\bigotimes_{j\in[d]}c^{j}\right\rangle|\geq{\lVert f\rVert^{c}_{\infty}}/2^{O(d)}$. Lastly, applying Lemma˜3.4.7 implies the claim.

Some Probability Facts

Lemma 3.4.9**.**

*Let $X_{1},\ldots X_{k}$ be i.i.d. $\mathrm{Bernoulli}(p)$ random variables. Then for any $t_{1},\ldots,t_{k}\in\mathds{N}$,

$\mathchoice{{\mathbb{E}}\left[X_{1}^{t_{1}}\dots X_{k}^{t_{k}}\right]}{{\mathbb{E}}[X_{1}^{t_{1}}\dots X_{k}^{t_{k}}]}{{\mathbb{E}}[X_{1}^{t_{1}}\dots X_{k}^{t_{k}}]}{{\mathbb{E}}[X_{1}^{t_{1}}\dots X_{k}^{t_{k}}]}=p^{k}$.*

Lemma 3.4.10**.**

Let $\zeta$ be a uniformly random $p$-th root of unity. Then for any $t\in[p-1]$, $\mathchoice{{\mathbb{E}}\left[\zeta^{t}\right]}{{\mathbb{E}}[\zeta^{t}]}{{\mathbb{E}}[\zeta^{t}]}{{\mathbb{E}}[\zeta^{t}]}=0$. Also, clearly $\mathchoice{{\mathbb{E}}\left[\zeta^{p}\right]}{{\mathbb{E}}[\zeta^{p}]}{{\mathbb{E}}[\zeta^{p}]}{{\mathbb{E}}[\zeta^{p}]}=1$.

We finally lower bound $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ in terms of $F_{2\alpha}$. Fix $\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}$ and, let $x^{*}$ be the optimizer of $F_{2\alpha}$. Setting  $y=x^{*}+\frac{\sqrt{\alpha}}{|\alpha|}$  as in the non-negative coefficient case does not work since terms from $F_{2\beta}$ may be negative. We bypass this issue by first lower bounding ${\lVert f\rVert^{c}_{\infty}}$ in terms of $F_{2\alpha}$ and using Lemma˜3.4.8. For ${\lVert f\rVert^{c}_{\infty}}$, we use random roots of unity and Bernoulli random variables, together with Lemma˜3.4.5, to extract nonzero contribution only from the monomials that are multiples of $x^{\alpha}$ times multilinear parts.

Lemma 3.4.11**.**

Let $f(x)$ be a homogeneous $n$-variate degree-$d$ polynomial. Then for any $\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}$,

[TABLE]

• **Proof:  ** Fix any $\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}$, let $t:=|\alpha|$ and let $k:=d-2t$. For any $i\in[n]$, let $\zeta_{i}$ be an independent and uniformly randomly chosen $(2\alpha_{i}+1)$-th root of unity, and let $\Xi$ be an independent and uniformly randomly chosen $(k+1)$-th root of unity.

Let $\mkern 1.2mu\overline{\mkern-1.2mux\mkern-1.2mu}\mkern 1.2mu:=\mathrm{argmax}\mathchoice{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}{\lVert F_{2\alpha}\rVert_{\infty}}$. Let $p\in[0,1]$ be a parameter to be fixed later, let $b_{1},\dots,b_{n}$ be i.i.d. $\mathrm{Bernoulli}(p)$ random variables, let $\zeta:=(\zeta_{1},\dots,\zeta_{n}),~b:=(b_{1},\dots,b_{n})$ and finally let

[TABLE]

Since $\sum_{\ell\in\mathsf{S}(\alpha)}\alpha_{\ell}=t$ and roots of unity have magnitude one, $z$ has length $O(1)$. Now consider any fixed $\gamma\in\{0,1\}_{k}^{n}$. We have,

[TABLE]

Thus we have,

[TABLE]

Lastly we have,

[TABLE]

This completes the proof.

In fact, the proof of Lemma˜3.4.11 yields a more general result:

Lemma 3.4.12** (Weak Decoupling).**

Let $f(x)$ be a homogeneous $n$-variate degree-$d$ polynomial. Then for any $\alpha\in{\mathbb{N}}_{\!\leq d/2}^{n}$ and any unit vector $y$,

[TABLE]

We are finally able to establish the multilinear reduction result that is the focus of this section.

Theorem 3.4.13**.**

Let $f(x)$ be a homogeneous $n$-variate degree-$d$ (for even $d$) polynomial. Then

[TABLE]

• **Proof:  ** Combining Lemma˜3.4.2 with Lemma˜3.4.11 yields the claim.

3.4.2 ${(n/q)}^{d/4}$-Approximation for Non-negative Coefficient Polynomials

Theorem 3.4.14**.**

Consider any homogeneous multilinear $n$-variate degree-$d$ polynomial $f(x)$ with non-negative coefficients. We have,

[TABLE]

• **Proof:  ** Let $\mathsf{M}_{f}$ be the SoS-symmetric matrix representation of $f$. Let $I^{*}=(i_{1},\dots,i_{d/2})\in[n]^{d/2}$ be the multi-index of any row of $\mathsf{M}_{f}$ with maximum row sum. Let $S_{I}$ for $I\in[n]^{d/2}$, denote the sum of the row $I$ of $\mathsf{M}_{f}$. By Perron-Frobenius theorem, $\|\mathsf{M}_{f}\|\leq S_{I^{*}}$. Thus, $\Lambda\left(f\right)\leq S_{I^{*}}$.

We next proceed to bound $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ from below. To this end, let $x^{*}:=y^{*}/\|y^{*}\|$ where,

[TABLE]

Since $f$ is multilinear, $I^{*}$ has all distinct elements, and so the second term in the definition of $y^{*}$ is of unit length. Thus $\|y^{*}\|=\Theta(1)$, which implies that $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}\geq f(x^{*})\geq f(y^{*})/2^{O(d)}$. Now we have,

[TABLE]

This completes the proof.

Theorem 3.4.15**.**

Let $f(x)$ be a homogeneous $n$-variate degree-$d$ polynomial with non-negative coefficients. Then for any even $q$ such that $d$ divides $q$,

[TABLE]

• **Proof:  ** Applying Theorem˜3.4.4 to $f^{q/d}$ and combining this with Theorem˜3.4.14 yields the claim.

3.4.3 ${(n/q)}^{d/2}$-Approximation for General Polynomials

Theorem 3.4.16**.**

Consider any homogeneous multilinear $n$-variate degree-$d$ (for even $d$) polynomial $f(x)$. We have,

[TABLE]

• **Proof:  ** Let $\mathsf{M}_{f}$ be the SoS-symmetric matrix representation of $f$, i.e.

[TABLE]

By the Gershgorin circle theorem, we can bound $\mathchoice{\lVert\mathsf{M}_{f}\rVert_{2}}{\lVert\mathsf{M}_{f}\rVert_{2}}{\lVert\mathsf{M}_{f}\rVert_{2}}{\lVert\mathsf{M}_{f}\rVert_{2}}$, and hence $\Lambda\left(f\right)$ by $n^{d/2}\cdot(\max_{\beta}\left\lvert f_{\beta}\right\rvert/d!)$. Here, we use the multilinearity of $f$. On the other hand for a multilinear polynomial, using $x=\beta/\sqrt{\left\lvert\beta\right\rvert}$ (where $|\beta|=d$ by multilinearity), gives $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}\geq d^{-d/2}\cdot\left\lvert f_{\beta}\right\rvert$. Thus, we easily get

[TABLE]

Theorem 3.4.17**.**

Let $f(x)$ be a homogeneous $n$-variate degree-$d$ polynomial, and assume that $2d$ divides $q$. Then

[TABLE]

• **Proof:  ** Applying Theorem˜3.4.13 to $f^{q/d}$ and combining this with Theorem˜3.4.16 yields the claim.

3.4.4 $\sqrt{m/q}$-Approximation for $m$-sparse polynomials

Lemma 3.4.18**.**

Consider any homogeneous multilinear $n$-variate degree-$d$ (for even $d$) polynomial $f(x)$ with $m$ non-zero coefficients. We have,

[TABLE]

• **Proof:  ** Let $\mathsf{M}_{f}$ be the SoS-symmetric matrix representation of $f$, i.e.,

[TABLE]

Now $\Lambda\left(f\right)\leq\|\mathsf{M}_{f}\|\leq\|\mathsf{M}_{f}\|_{F}$. Thus, we have:

[TABLE]

On the other hand, since $f$ is multilinear, using $x=\beta/\sqrt{\left\lvert\beta\right\rvert}$ (where $|\beta|=d$ by multilinearity), implies $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}\geq d^{-d/2}\cdot\left\lvert f_{\beta}\right\rvert$ for any $\beta$. This implies the claim.

Theorem 3.4.19**.**

Let $f(x)$ be a homogeneous $n$-variate degree-$d$ polynomial with $m$ non-zero coefficients, and assume that $2d$ divides $q$. Then

[TABLE]

• **Proof:  ** Combining Theorem˜3.4.13 and Lemma˜3.4.18, yields that for any degree-$q$ homogeneous polynomial $g$ with sparsity $\mkern 1.2mu\overline{\mkern-1.2mum\mkern-1.2mu}\mkern 1.2mu$, we have

[TABLE]

Lastly, taking $g=f^{q/d}$ and observing that the sparsity of $g$ is at most $\left(\kern-3.00003pt\left(\genfrac{}{}{0.0pt}{}{m}{q/d}\right)\kern-3.00003pt\right)$ implies the claim.

3.5 Improved Approximations via Folding

Recall that we call a folded polynomial multilinear if all its monomials are multilinear. In particular, there’s no restriction on the folds of the polynomial.

Lemma 3.5.1** (Folded Analogue of Lemma˜3.4.1).**

Let ${\!\left({{\mathbb{R}}_{d_{2}}\![x]}\right)\!_{d_{1}}}\![x]\ni f(x):=\sum_{\beta\in{\mathbb{N}}_{d_{1}}^{n}}\mkern 1.2mu\overline{\mkern-1.2muf\mkern-1.2mu}\mkern 1.2mu_{\beta}(x)\cdot x^{\beta}$ be a $(d_{1},d_{2})$-folded polynomial. $f$ can be written as

[TABLE]

where for any $\alpha\in{\mathbb{N}}_{\!\leq d_{1}\!/2}^{n}$, $F_{2\alpha}(x)$ is a multilinear $(d_{1}-2|\alpha|,d_{2})$-folded polynomial.

• **Proof:  ** Simply consider the folded polynomial

[TABLE]

where $\mkern 1.2mu\overline{\mkern-1.2mu(F_{2\alpha})\mkern-1.2mu}\mkern 1.2mu_{\gamma}=\mkern 1.2mu\overline{\mkern-1.2muf\mkern-1.2mu}\mkern 1.2mu_{2\alpha+\gamma}$.

3.5.1 Reduction to Multilinear Folded Polynomials

Here we will prove a generalized version of Lemma˜3.4.2, which is a generalization in two ways; firstly it allows for folds instead of just coefficients, and secondly it allows a more general set of constraints than just the hypersphere since we will need to add some additional non-negativity constraints for the case of non-negative coefficient polynomials (so that $\Lambda_{C}\left(\right)$ satisfies monotonicity over NNC polynomials which will come in handy later).

Recall that $\Lambda_{C}\left(\right)$ is defined in Section˜3.3.2 and that $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ and $\Lambda_{C}\left(f\right)$ for a folded polynomial $f$, are applied to the unfolding of $f$.

Relating $\Lambda_{C}\left(f\right)$ to $\Lambda_{C}\left(F_{2\alpha}\right)$

Lemma 3.5.2** (Folded Analogue of Lemma˜3.4.2).**

Let $C$ be a system of polynomial constraints of the form $\{\|x\|_{2}^{2}=1\}\cup C^{\prime}$ where $C^{\prime}$ is a moment non-negativity constraint set. Let $f\in{\!\left({{\mathbb{R}}_{d_{2}}\![x]}\right)\!_{d_{1}}}\![x]$ be a $(d_{1},d_{2})$-folded polynomial. We have,

[TABLE]

• **Proof:  ** Consider any degree-$(d_{1}+d_{2})$ pseudo-expectation operator $\widetilde{\mathbb{E}}_{C}$. We have,

[TABLE]

3.5.2 Relating Evaluations of $f$ to Evaluations of $F_{2\alpha}$

Here we would like to generalize Lemma˜3.4.3 and Lemma˜3.4.11 to allow folds, however for technical reasons related to decoupling of the domain of the folds from the domain of the monomials of a folded polynomial, we instead generalize claims implicit in the proofs of Lemma˜3.4.3 and Lemma˜3.4.11.

Let $f\in{\!\left({{\mathbb{R}}_{d_{2}}\![x]}\right)\!_{d_{1}}}\![x]$ be a $(d_{1},d_{2})$-folded polynomial. Recall that an evaluation of a folded polynomial treats the folds as coefficients and only substitutes values in the monomials of the folded polynomial. Thus, for any fixed $y\in\Re^{n}$, $f(y)$ (sometimes denoted by $(f(y))(x)$ for contextual clarity) is a degree-$d_{2}$ polynomial in $x$, i.e. $f(y)\in{\mathbb{R}}_{d_{2}}[x]$.

Lemma 3.5.3** (Folded Analogue of Lemma˜3.4.3).**

Let $f\in{\!\left({{\mathbb{R}}^{+}_{d_{2}}\![x]}\right)\!_{d_{1}}}\![x]$ be a $(d_{1},d_{2})$-folded polynomial whose folds have non-negative coefficients. Then for any $\alpha\in{\mathbb{N}}_{\!\leq d_{1}/2}^{n}$ and any $y\geq 0$,

[TABLE]

where the ordering is coefficient-wise.

• **Proof:  ** Identical to the proof of Lemma˜3.4.3.

Lemma 3.5.4** (Folded Analogue of Lemma˜3.4.11).**

Let $f\in{\!\left({{\mathbb{R}}_{d_{2}}\![x]}\right)\!_{d_{1}}}\![x]$ be a $(d_{1},d_{2})$-folded polynomial. Consider any $\alpha\in{\mathbb{N}}_{\!\leq d_{1}/2}^{n}$ and any $y$, and let

[TABLE]

where $\Xi$ is an independent and uniformly randomly chosen $(d_{1}-2|\alpha|+1)$-th root of unity, and for any $i\in[n]$, $\zeta_{i}$ is an independent and uniformly randomly chosen $(2\alpha_{i}+1)$-th root of unity, and $b_{i}$ is an independent $\mathrm{Bernoulli}(p)$ random variable ($p$ is an arbitrary parameter in $[0,1]$). Then

[TABLE]

where $r(p)$ is a univariate polynomial in $p$ with degree less than $d_{1}-2|\alpha|$ (and whose coefficients are in ${\mathbb{R}}_{d_{2}}[x]$).

• **Proof:  ** This follows by going through the proof of Lemma˜3.4.11 for every fixed $x$.

3.5.3 Bounding $\Lambda_{C}\left(\right)$ of Multilinear Folded Polynomials

Here we bound $\Lambda_{C}\left(\right)$ of a multilinear folded polynomial in terms of properties of the polynomial that are inspired by treating the folds as coefficients and generalizing the coefficient-based approximations for regular (non-folded) polynomials from Theorem˜3.4.16 and Theorem˜3.4.14.

General Folds: Bounding $\Lambda\left(\right)$ in terms of $\Lambda\left(\right)$ of the "worst" fold

Here we will give a folded analogue of the proof of Theorem˜3.4.16 wherein we used Gershgorin-Circle theorem to bound SOS value in terms of the max-magnitude-coefficient.

Lemma 3.5.5** (Folded Analogue of Gershgorin Circle Bound on Spectral Radius).**

For even $d_{1},d_{2}$, let $d=d_{1}+d_{2}$, let $f\in{\!\left({{\mathbb{R}}_{d_{2}}\![x]}\right)\!_{d_{1}}}\![x]$ be a multilinear $(d_{1},d_{2})$-folded polynomial. We have,

[TABLE]

• **Proof:  ** Since $\Lambda\left(f\right)\leq\|f\|_{sp}$, it is sufficient to bound $\|f\|_{sp}$.

Let $M_{\mkern 1.2mu\overline{\mkern-1.2muf\mkern-1.2mu}\mkern 1.2mu_{\gamma}}$ be the matrix representation of $\mkern 1.2mu\overline{\mkern-1.2muf\mkern-1.2mu}\mkern 1.2mu_{\gamma}$ realizing $\|\mkern 1.2mu\overline{\mkern-1.2muf\mkern-1.2mu}\mkern 1.2mu_{\gamma}\|_{sp}$. Let $M_{f}$ be an $[n]^{d_{1}/2}\times[n]^{d_{1}/2}$ block matrix with $[n]^{d_{2}/2}\times[n]^{d_{2}/2}$ size blocks, where for any $I,J\in[n]^{d_{1}/2}$ the block of $M_{f}$ at index $(I,J)$ is defined to be $\frac{1}{d_{1}!}\cdot M_{\mkern 1.2mu\overline{\mkern-1.2muf\mkern-1.2mu}\mkern 1.2mu_{\alpha({I})+\alpha({J})}}$. Clearly $M_{f}$ (interpreted as an $[n]^{d/2}\times[n]^{d/2}$) is a matrix representation of the unfolding of $f$ since $f$ is a multilinear folded polynomial. Lastly, applying Block-Gershgorin circle theorem to $M_{f}$ and upper bounding the sum of spectral norms over a block row by $n^{d_{1}/2}$ times the max term implies the claim.

Non-Negative Coefficient Folds: Relating SoS Value to the SoS Value of the $d_{1}/2$-collapse

Observe that in the case of a multilinear degree-$d$ polynomial, the $d/2$-collapse corresponds (up to scaling) to the sum of a row of the SOS symmetric matrix representation of the polynomial. We will next develop a folded analogue of the proof of Theorem˜3.4.14 wherein we employed Perron-Frobenius theorem to bound SOS value in terms of the $d/2$-collapse.

The proof here however, is quite a bit more subtle than in the general case above. This is because one can apply the block-matrix analogue of Gershgorin theorem (due to Feingold et al. [FV*+*62]) to a matrix representation of the folded polynomial (whose spectral norm is an upper bound on $\Lambda\left(\right)$) in the general case. Loosely speaking, this corresponds to bounding $\Lambda\left(f\right)$ in terms of

[TABLE]

where $k=d_{1}/2$. This however is not enough in the nnc case as in order to win the $1/2$ in the exponent, one needs to relate $\Lambda_{C}\left(f\right)$ to

[TABLE]

This however, cannot go through Block-Gershgorin since it is not true that the spectral norm of a non-negative block matrix is upper bounded by the max over rows of the spectral norm of the sum of blocks in that row. It instead, can only be upper bounded by the max over rows of the sum of spectral norms of the blocks in that row.

To get around this issue, we skip the intermediate step of bounding $\Lambda_{C}\left(f\right)$ by the spectral norm of a matrix and instead prove the desired relation directly through the use of pseudoexpectation operators. This involved first finding a pseudo-expectation based proof of Gershgorin/Perron-Frobenius bound on spectral radius that generalizes to folded polynomials in the right way.

Lemma 3.5.6** (Folded analogue of Perron-Frobenius Bound on Spectral Radius).**

For even $d_{1}=2k$, let $f\in{\!\left({{\mathbb{R}}^{+}_{d_{2}}\![x]}\right)\!_{d_{1}}}\![x]$ be a multilinear $(d_{1},d_{2})$-folded polynomial whose folds have non-negative coefficients. Let $C$ be the system of polynomial constraints given by $\{\|x\|_{2}^{2}=1;\forall\beta\in{\mathbb{N}}_{d_{2}}^{n},x^{\beta}\geq 0\}$. We have,

[TABLE]

where

[TABLE]

• **Proof:  ** Consider any pseudo-expectation operator $\widetilde{\mathbb{E}}_{C}$ of degree at least $d_{1}+d_{2}$. Note that since $\widetilde{\mathbb{E}}_{C}$ satisfies $\{\forall\beta\in{\mathbb{N}}_{d_{2}}^{n},x^{\beta}\geq 0\}$, by linearity $\widetilde{\mathbb{E}}_{C}$ must also satisfy $\{h\geq 0\}$ for any $h\in{\mathbb{R}}^{+}_{d_{2}}[x]$ - a fact we will use shortly.

Since $f$ is a multilinear folded polynomial, $\mkern 1.2mu\overline{\mkern-1.2muf\mkern-1.2mu}\mkern 1.2mu_{\alpha}$ is only defined when $0\leq\alpha\leq\mathds{1}$. If $\alpha\not\leq\mathds{1}$, we define $\mkern 1.2mu\overline{\mkern-1.2muf\mkern-1.2mu}\mkern 1.2mu_{\alpha}:=0$ We have,

[TABLE]

We are finally equipped to prove the main results of this section.

3.5.4 $(n/q)^{d/4-1/2}$-Approximation for Non-negative Coefficient Polynomials

Theorem 3.5.7**.**

Consider any $f\in{\mathbb{R}}^{+}_{d}[x]$ for $d\geq 2$, and any $q$ divisible by $2d$. Let $C$ be the system of polynomial constraints given by $\{\|x\|_{2}^{2}=1;\forall\beta\in{\mathbb{N}}_{2q/d}^{n},x^{\beta}\geq 0\}$. Then we have,

[TABLE]

• **Proof:  ** Let $h$ be any $(d-2,2)$-folded polynomial whose unfolding yields $f$ and whose folds have non-negative coefficients and let $s$ be the $(\overline{q},2q/d)$-folded polynomial given by $h^{q/d}$ where $\overline{q}:=(d-2)q/d$. Finally, consider any $\alpha\in{\mathbb{N}}_{\!\leq\overline{q}/2}^{n}$ and let $S_{2\alpha}$ be the multilinear component of $s$ as defined in Lemma˜3.5.1. We will establish that for any $\gamma\in\{0,1\}_{k}^{n}$ (where $k:=\overline{q}/2-|\alpha|$),

[TABLE]

which on combining with the application of Lemma˜3.5.2 to $s$ and its composition with Lemma˜3.5.6, yields the claim. To elaborate, we apply Lemma˜3.5.2 to $s$ with $d_{1}=\overline{q},d_{2}=2q/d$ and then for every $\alpha\in{\mathbb{N}}_{\!\leq\overline{q}/2}^{n}$ we apply Lemma˜3.5.6 with $d_{1}=\overline{q}-2|\alpha|,d_{2}=2q/d$, to get

[TABLE]

which on combining with Eq.˜3.8 yields the claim.

It remains to establish Eq.˜3.8. So fix any $\alpha,\gamma$ satisfying the above conditions. Let $t:=|\alpha|$ and let $k:=\overline{q}/2-|\alpha|$. Clearly $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}\geq f(y/\|y\|_{2})$ where $y:=a+z$, and

[TABLE]

and $a$ is the unit vector that maximizes the quadratic polynomial

[TABLE]

Since $\|y\|_{2}=O(1)$, $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}\geq f(y)/2^{O(d)}$. Now clearly by non-negativity we have,

[TABLE]

Thus we have,

[TABLE]

which completes the proof since we’ve established Eq.˜3.8.

3.5.5 $(n/q)^{d/2-1}$-Approximation for General Polynomials

Theorem 3.5.8**.**

Consider any $f\in{\mathbb{R}}^{+}_{d}[x]$ for $d\geq 2$, and any $q$ divisible by $2d$. Then we have,

[TABLE]

• **Proof:  ** Let $h$ be the unique $(d-2,2)$-folded polynomial whose unfolding yields $f$ and such that for any $\beta\in{\mathbb{N}}_{d-2}^{n}$, the fold $\mkern 1.2mu\overline{\mkern-1.2muh\mkern-1.2mu}\mkern 1.2mu_{\beta}$ of $h$ is equal up to scaling, to the quadratic form of the corresponding $(n\times n)$ block of the SOS-symmetric matrix representation $\mathsf{M}_{f}$ of $f$. That is, for any $I,J\in[n]^{d/2-1}$, s.t. $\alpha({I})+\alpha({J})=\beta$,

[TABLE]

Let $s$ be the $(\overline{q},2q/d)$-folded polynomial given by $h^{q/d}$ where $\overline{q}:=(d-2)q/d$. Consider any $\alpha\in{\mathbb{N}}_{\!\leq\overline{q}/2}^{n}$ and $\gamma\in\{0,1\}_{\overline{q}-2|\alpha|}^{n}$, and let $S_{2\alpha}$ be the multilinear component of $s$ as defined in Lemma˜3.5.1. Below, we will show

[TABLE]

which would complete the proof after applying Lemma˜3.5.2 to $s$ and composing the result with Lemma˜3.5.5. To elaborate, we apply Lemma˜3.5.2 to $s$ with $d_{1}=\overline{q},d_{2}=2q/d$ and then for every $\alpha\in{\mathbb{N}}_{\!\leq\overline{q}/2}^{n}$ we apply Lemma˜3.5.5 with $d_{1}=\overline{q}-2|\alpha|,d_{2}=2q/d$, to get

[TABLE]

which on combining with Eq.˜3.9 yields the claim.

Fix any $\alpha,\gamma$ satisfying the above conditions. Let $k:=\overline{q}-2\alpha$. Let $t:=|\alpha|$, and let

[TABLE]

$\Xi$ is an independent and uniformly randomly chosen $(k+1)$-th root of unity, and for any $i\in[n]$, $\zeta_{i}$ is an independent and uniformly randomly chosen $(2\alpha_{i}+1)$-th root of unity, and for any $i\in[n]$, $b_{i}$ is an independent $\mathrm{Bernoulli}(p)$ random variable ($p$ is a parameter that will be set later). By Lemma 3.4.7 and definition of $h$, we see that for any $y$, ${\lVert f\rVert^{c}_{\infty}}\geq\|(h(y))(x)\|^{c}_{2}$. Thus, we have,

[TABLE]

where the last inequality follows by the following argument: one would like to show that there always exists $p\in[0,1]$ such that $\|p^{k}\cdot h_{k}(x)+\dots p^{0}\cdot h_{0}(x))\|_{sp}\geq\|h_{k}(x)\|_{sp}\cdot 2^{-O(k)}$. So let $p$ be such that $|p^{k}\cdot u^{T}M_{k}v+\dots p^{0}\cdot u^{T}M_{0}v|\geq|u^{T}M_{k}v|\cdot 2^{-O(k)}$ (such a $p$ exists by Chebyshev inequality) where $M_{k}$ is the matrix representation of $h_{k}(x)$ realizing $\|h_{k}\|_{sp}$ and $u,v$ are the maximum singular vectors of $M_{k}$. $M_{k-1},\dots,M_{0}$ are arbitrary matrix representations of $h_{k-1},\dots h_{0}$ respectively. But $p^{k}\cdot M_{k}+\dots p^{0}\cdot M_{0}$ is a matrix representation of $p^{k}\cdot h_{k}+\dots p^{0}\cdot h_{0}$. Thus $\mathchoice{\lVert p^{k}\cdot h_{k}+\dots p^{0}\cdot h_{0}\rVert_{\mathrm{sp}}}{\lVert p^{k}\cdot h_{k}+\dots p^{0}\cdot h_{0}\rVert_{\mathrm{sp}}}{\lVert p^{k}\cdot h_{k}+\dots p^{0}\cdot h_{0}\rVert_{\mathrm{sp}}}{\lVert p^{k}\cdot h_{k}+\dots p^{0}\cdot h_{0}\rVert_{\mathrm{sp}}}\geq|u^{T}M_{k}v|/2^{-O(k)}=\mathchoice{\lVert h_{k}\rVert_{\mathrm{sp}}}{\lVert h_{k}\rVert_{\mathrm{sp}}}{\lVert h_{k}\rVert_{\mathrm{sp}}}{\lVert h_{k}\rVert_{\mathrm{sp}}}\cdot 2^{-O(q)}$.

This completes the proof as we’ve established Eq.˜3.9.

3.5.6 Algorithms

It is straightforward to extract algorithms from the proofs of Theorem˜3.5.7 and Theorem˜3.5.8.

Non-negative coefficient polynomials

Let $f$ be a degree-$d$ polynomial with non-negative coefficients and let $h$ be a $(d-2,2)$-folded polynomial whose unfolding yields $f$. Consider any $q$ divisible by $2d$ and let $\overline{q}:=(d-2)q/d$. Pick and return the best vector from the set

[TABLE]

General Polynomials

Let $f$ be a degree-$d$ polynomial and let $h$ be the unique $(d-2,2)$-folded polynomial whose unfolding yields $f$ and such that for any $\beta\in{\mathbb{N}}_{d-2}^{n}$, the fold $\mkern 1.2mu\overline{\mkern-1.2muh\mkern-1.2mu}\mkern 1.2mu_{\beta}$ of $h$ is equal up to scaling, to the quadratic form of the corresponding $(n\times n)$ block of the SOS-symmetric matrix representation $\mathsf{M}_{f}$ of $f$. That is, for any $I,J\in[n]^{d/2-1}$, s.t. $\alpha({I})+\alpha({J})=\beta$,

[TABLE]

Consider any $q$ divisible by $2d$ and let $\overline{q}:=(d-2)q/d$. Let the set $S$ be defined by,

[TABLE]

where $\Omega_{p}$ denotes the set of $p$-th roots of unity. Pick and return the best vector from the set

[TABLE]

Note that one need only search through all roots of unity vectors $\zeta$ supported on $\mathsf{S}(\gamma)$ and all $\{0,1\}$-vectors $b$ supported on $\mathsf{S}(\alpha)$. The Lemma˜3.4.7 can trivially be made constructive in time $2^{O(q)}$. Lastly, to go from complexes to reals, the Lemma˜3.4.8 can trivially be made constructive using $2^{O(d)}$ time. Thus, the algorithm runs in time $n^{O(q)}$.

3.6 Oracle Lower Bound

Khot and Naor [KN08] observed that the problem of maximizing a polynomial over unit sphere can be reduced to computing diameter of centrally symmetric convex body. This observation was also used by So [So11a] later. We recall the reduction here: For a convex set $K$, let $K^{\circ}$ denote the polar of $K$, i.e., $K^{\circ}=\{y{\;\;:\;\;}\forall x\in K\left\langle x,y,\leq\right\rangle 1\}$. For a degree-$3$ polynomial $P(x,y,z)$ on $3n$ variables, let $\mathchoice{\lVert x\rVert_{P}}{\lVert x\rVert_{P}}{\lVert x\rVert_{P}}{\lVert x\rVert_{P}}=\mathchoice{\lVert P(x,\cdot,\cdot)\rVert_{sp}}{\lVert P(x,\cdot,\cdot)\rVert_{sp}}{\lVert P(x,\cdot,\cdot)\rVert_{sp}}{\lVert P(x,\cdot,\cdot)\rVert_{sp}}$ where $P(x,\cdot,\cdot)$ is a degree-2 restriction of $P$ with $x$ variables set. Let $\mathbb{B}_{P}=\{x{\;\;:\;\;}\mathchoice{\lVert x\rVert_{P}}{\lVert x\rVert_{P}}{\lVert x\rVert_{P}}{\lVert x\rVert_{P}}\leq 1\}$. From the definition of polar and $\mathchoice{\lVert\cdot\rVert_{sp}}{\lVert\cdot\rVert_{sp}}{\lVert\cdot\rVert_{sp}}{\lVert\cdot\rVert_{sp}}$, we have:

[TABLE]

For general convex bodies, a lower bound for number of queries with “weak separation oracle” for approximating the diameter of the convex body was proved by Brieden et al. [BGK*+*01] and later improved by Khot and Naor [KN08]. We recall the definition:

Definition 3.6.1**.**

For a given a convex body $P$, a weak separation oracle $A$ is an algorithm which on input $(x,\varepsilon)$ behaves as following:

If $x\in A+\varepsilon\mathbb{B}_{2}$, $A$ accepts it.

• -

Else $A$ outputs a vector $c\in\mathbb{Q}^{n}$ with $\mathchoice{\lVert c\rVert_{\infty}}{\lVert c\rVert_{\infty}}{\lVert c\rVert_{\infty}}{\lVert c\rVert_{\infty}}=1$ such that for all $y$ such that $y+\varepsilon\mathbb{B}_{2}\subset P$ we have $c^{T}x+\varepsilon\geq c^{T}y$.

Let $K_{s,v}$ be the convex set $K^{(n)}_{s,v}=\operatorname{conv}\left(\mathbb{B}_{2}^{n}\cup\{sv,-sv\}\right)$, for unit vector $v$. Brieden et al. [BGK*+*01] proved the following theorem:

Theorem 3.6.2**.**

Let $A$ be a randomized algorithm, for every convex set $P$, with access to a weak separation oracle for $P$. Let $\mathcal{K}(n,s)=\{K^{(n)}_{s,u}\}_{u\in\mathbb{S}^{n-1}_{2}}\cup\{\mathbb{B}^{n}_{2}\}$. If for every $K\in\mathcal{K}(n,s)$ and $s=\frac{\sqrt{n}}{\lambda}$, we have:

[TABLE]

where $\operatorname{\operatorname{diam}}(K)$ is the diameter of $K$, then $A$ must use at least $\operatorname{\operatorname{O}}(\lambda 2^{\lambda^{2}/2})$ oracle queries for $\lambda\in[\sqrt{2},\sqrt{n/2}]$.

Using $\lambda=\log n$, we get that to get $s=\frac{\sqrt{n}}{\log n}$ approximation to diameter, $A$ must use super-polynomial number of queries to the weak separation oracle. We note that this was later improved to give analogous lower bound on the number of queries for an approximation factor $\sqrt{\frac{n}{\log n}}$ by Khot and Naor [KN08].

Below, we show that the family of hard convex bodies considered by Brieden et al. [BGK*+*01] can be realized as $\{\mathbb{B}_{P}^{\circ}\}_{P\in\mathcal{P}}$ by a family of polynomials $\mathcal{P}$ – which, in turn, establishes a lower bound of $\operatorname{\operatorname{\Omega}}{\frac{\sqrt{n}}{\log n}}$ on the approximation for polynomial optimization, achievable using this approach, for the case of $d=3$. For a unit vector $u\in\mathbb{S}_{2}^{n-1}$, let $P_{u}$ be the polynomial defined as:

[TABLE]

A matrix representation of $P_{u}(x,\cdot,\cdot)$, with rows indexed by $y$ and columns indexed by $z$ variables is as follows:

[TABLE]

This proves: $\mathchoice{\lVert x\rVert_{P_{u}}}{\lVert x\rVert_{P_{u}}}{\lVert x\rVert_{P_{u}}}{\lVert x\rVert_{P_{u}}}=\mathchoice{\lVert P_{u}(x,\cdot,\cdot)\rVert_{sp}}{\lVert P_{u}(x,\cdot,\cdot)\rVert_{sp}}{\lVert P_{u}(x,\cdot,\cdot)\rVert_{sp}}{\lVert P_{u}(x,\cdot,\cdot)\rVert_{sp}}=\mathchoice{\lVert A_{u}\rVert_{sp}}{\lVert A_{u}\rVert_{sp}}{\lVert A_{u}\rVert_{sp}}{\lVert A_{u}\rVert_{sp}}=\max\{\mathchoice{\lVert x\rVert_{2}}{\lVert x\rVert_{2}}{\lVert x\rVert_{2}}{\lVert x\rVert_{2}},s\left\lvert u^{T}x\right\rvert\}$.

Let $B=\{x{\;\;:\;\;}\mathchoice{\lVert x\rVert_{2}}{\lVert x\rVert_{2}}{\lVert x\rVert_{2}}{\lVert x\rVert_{2}}\leq 1\}$ and $C_{u}=\{x{\;\;:\;\;}s\cdot\left\lvert u^{T}x\right\rvert\leq 1\}$. We note that, $B^{\circ}=\{y\in\mathbb{R}^{n}{\;\;:\;\;}\mathchoice{\lVert y\rVert_{2}}{\lVert y\rVert_{2}}{\lVert y\rVert_{2}}{\lVert y\rVert_{2}}\leq 1\}$ and, $C_{u}^{\circ}=\{\lambda\cdot u{\;\;:\;\;}\lambda\in[-s,s]\}=\operatorname{conv}(\left\{-s\cdot u,s\cdot u\right\})$.

Next, we observe: $\mathbb{B}_{P_{u}}=B\cap C_{u}$. It follows from De Morgan’s law of polars that: $\mathbb{B}_{P_{u}}^{\circ}=(B\cap C_{u})^{\circ}=\operatorname{conv}(B^{\circ}\cup C_{u}^{\circ})=\operatorname{conv}(\mathbb{B}_{2}^{n}\cup\{-s\cdot u,s\cdot u\})=K_{s,u}^{(n)}$. Finally, we observe that for the polynomial $P_{0}=\sum_{i=1}^{n}x_{i}y_{i}z_{1}$, we have: $\mathbb{B}_{P_{0}}=\mathbb{B}_{2}^{n}$.

Hence, for polynomial $Q\in\mathcal{P}=\{P_{u}\}_{u\in\mathbb{S}_{2}^{n-1}}\cup\{P_{0}\}$, no randomized polynomial can approximate $\operatorname{\operatorname{diam}}{\mathbb{B}_{Q}}$ within factor $\frac{\sqrt{n}}{q}$ without using more than $2^{\Omega(q)}$ number of queries. Since the algorithm of Khot and Naor [KN08] reduces the problem of optimizing polynomial $Q$ to computing $\operatorname{\operatorname{diam}}(\mathbb{B}_{Q})$, $\mathcal{P}$ shows that their analysis is almost tight.

3.7 Constant Level Lower Bounds for Polynomials with Non-negative Coefficients

Let $G=(V,E)$ be a random graph drawn from the distribution $G_{n,p}$ for $p\geq n^{-1/3}$. Let $\mathcal{C}\subseteq\binom{V}{4}$ be the set of $4$-cliques in $G$. The polynomial $f$ is defined as

[TABLE]

Clearly, $f$ is multilinear and every coefficient of $f$ is nonnegative. In this section, we prove the following two lemmas that establish a polynomial gap between $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$ and $\Lambda\left(f\right)$.

Lemma 3.7.1** (Soundness).**

With probability at least $1-\frac{1}{n}$ over the choice of the graph $G$, we have $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}\leq n^{2}p^{6}\cdot\left(\log n\right)^{O(1)}$.

Lemma 3.7.2** (Completeness).**

With probability at least $1-\frac{1}{n}$ over the choice of the graph $G$, we have

[TABLE]

when $p\in[n^{-1/3},n^{-1/4}]$.

Note that the gap between the two quantities if $\tilde{\Omega}(n^{1/6})$ when $p=n^{-1/3}$, which is the choice we make.

3.7.1 Upper Bound on $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$

Reduction to counting shattered cliques

We say that an ordered $4$-clique $(i_{1},\ldots,i_{4})$ is shattered by $4$ disjoint sets $Z_{1},\ldots,Z_{4}$ if for each $k\in[4]$, $i_{k}\in Z_{k}$. Let $Y_{j_{1}},\ldots,Y_{j_{4}}$ be the sets containing the coordinates $i_{1},\ldots,i_{4}$. Let $\mathcal{C}_{G}$ denote the set of (ordered) 4-cliques in $G$, and let $\mathcal{C}_{G}(Z_{1},Z_{2},Z_{3},Z_{4})$ denote the set of cliques shattered by $Z_{1},\ldots,Z_{4}$.

We reduce the problem of bounding $\mathchoice{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}{\lVert f\rVert_{\infty}}$, to counting shattered 4-cliques.

Claim 3.7.3**.**

There exist disjoint sets $Z_{1},\ldots,Z_{4}\subseteq[n]$ such that

[TABLE]

• **Proof:  ** Let $x^{*}\in\mathbb{S}^{n-1}$ be the vector that maximizes $f$. Without loss of generality, assume that every coordinate of $x^{*}$ is nonnegative. Let $y^{*}$ be another unit vector defined as

[TABLE]

Since both $x^{*}$ and $\frac{\mathds{1}}{\sqrt{n}}$ are unit vectors, the denominator is at most $2$. This implies that $f(y^{*})\geq\frac{f(x^{*})}{2^{4}}$, and each coordinate of $y^{*}$ is at least $\frac{1}{2\sqrt{n}}$. For $1\leq j\leq\log_{2}n$, let $Y_{j}$ be the set

[TABLE]

The sets $Y_{1},\dots,Y_{\log_{2}n}$ partition $[n]$. Since $1=\sum_{i\in[n]}y_{i}^{2}>|Y_{j}|\cdot 2^{-2j}$, we have for each $j$, $|Y_{j}|\leq 2^{2j}$. Let $Z_{1},Z_{2},Z_{3}$, and $Z_{4}$ be pairwise disjoint random subsets of $[n]$ chosen as follows:

• –

Randomly partition each $Y_{j}$ to $Y_{j,1},\dots,Y_{j,4}$ where each element of $Y_{j}$ is put into exactly one of $Y_{j,1},\dots,Y_{j,4}$ uniformly and independently.

• –

Sample $r_{1},\ldots,r_{4}$ independently and randomly from $\{1,\dots,\log_{2}n\}$.

• –

For $k=1,\dots,4$, take $Z_{k}:=Y_{r_{k},k}$

We use $\mathcal{P}$ to denote random partitions $\left\{\left(Y_{j,1},\dots,Y_{j,4}\right)\right\}_{j\in[\log n]}$ and $r$ to denote the random choices $r_{1},\ldots,r_{4}$. Note that the events $i_{k}\in Z_{k}$ are independent for different $k$, and that $Z_{1},\ldots,Z_{4}$ are independent given $\mathcal{P}$. Thus, we have

[TABLE]

Then, by linearity of expectation,

[TABLE]

which proves the claim.

We will show that with high probability, $G$ satisfies the property that every four disjoint sets $Z_{1},\dots,Z_{4}\subseteq V$ shatter at most $O\left(\sqrt{|Z_{1}||Z_{2}||Z_{3}||Z_{4}|}\cdot n^{2}p^{6}\cdot(\log n)^{O(1)}\right)$ cliques, proving Lemma˜3.7.1.

Counting edges and triangles

For a vertex $i\in[n]$, we use $\mathsf{N}(i)$ to denote the set of vertices in the graph $G$. For ease of notation, we use $a\lesssim b$ to denote $a\leq b\cdot(\log n)^{O(1)}$. We first collect some simple consequences of Chernoff bounds.

Claim 3.7.4**.**

Let $G\sim G_{n,p}$ with $p\geq n^{-1/3}$. Then, with probability $1-\frac{1}{n}$, we have

For all distinct $i_{1},i_{2}\in[n]$, $\left\lvert\mathsf{N}(i_{1})\cap\mathsf{N}(i_{2})\right\rvert~\lesssim~np^{2}$.

• -

For all distinct $i_{1},i_{2},i_{3}\in[n]$, $\left\lvert\mathsf{N}(i_{1})\cap\mathsf{N}(i_{2})\cap\mathsf{N}(i_{3})\right\rvert~\lesssim~np^{3}$.

• -

For all sets $S_{1},S_{2}\subseteq[n]$, $\left\lvert E(S_{1},S_{2})\right\rvert~\lesssim~\max\left\{\left\lvert S_{1}\right\rvert\left\lvert S_{2}\right\rvert p,\left\lvert S_{1}\right\rvert+\left\lvert S_{2}\right\rvert\right\}$.

We also need the following bound on the number of triangles shattered by three disjoint sets $S_{1},S_{2}$ and $S_{3}$, denoted by $\Delta_{G}(S_{1},S_{2},S_{3})$. As for $4$-cliques, a triangle is said to be shattered if it has exactly one vertex in each the sets.

Claim 3.7.5**.**

Let $G\sim G_{n,p}$ with $p\geq n^{-1/3}$. Then, with probability $1-\frac{1}{n}$, for all disjoint sets $S_{1},S_{2},S_{3}\subseteq[n]$

[TABLE]

• **Proof:  ** With probability at least $1-\frac{1}{n}$, $G$ satisfies the conclusion of ˜3.7.4. Fix such a $G$, and consider arbitrary subsets $S_{1},S_{2},S_{3}\subseteq V$. Consider the bipartite graph $H$ where the left side vertices correspond to edges in $E(S_{1},S_{2})$, the right side vertices correspond to vertices in $S_{3}$, and there is an edge from $(i_{1},i_{2})\in E(S_{1},S_{2})$ to $i_{3}\in S_{3}$ when both $(i_{1},i_{3}),(i_{2},i_{3})\in E$. Clearly, $\left\lvert\Delta_{G}(S_{1},S_{2},S_{3})\right\rvert$ is equal to the number of edges in $H$.

Consider two different edges $(i_{1},i_{2}),(i_{1}^{\prime},i_{2}^{\prime})\in E(S_{1},S_{2})$. These two edges are incident on at least $3$ distinct vertices, say $\{i_{1},i_{2},i_{1}^{\prime}\}$. Hence, the number of vertices $i_{3}\in[n]$ that are adjacent to all $\{i_{1},i_{2},i_{1}^{\prime},i_{2}^{\prime}\}$ in $G$ is at most $\left\lvert\mathsf{N}(i_{1})\cap\mathsf{N}(i_{2})\cap\mathsf{N}(i_{1}^{\prime})\right\rvert\lesssim np^{3}$. This gives that the number of pairs triangles sharing a common vertex in $S_{3}$ is at most $\left\lvert E(S_{1},S_{2})\right\rvert^{2}\cdot np^{3}(\log n)^{O(1)}$.

Let $d_{H}(i_{3})$ denote the degree of a vertex $i_{3}$ in $H$, and let $\Delta$ denote the number of shattered triangles. Counting the above pairs of triangles using the degrees gives

[TABLE]

An application of Cauchy-Schwarz gives

[TABLE]

which proves the claim.

Bounding $4$-clique Density

Let $G\sim G_{n,p}$ be a graph satisfying the conclusions of ˜3.7.4 and ˜3.7.5. Let $S_{1},\ldots,S_{4}\subseteq[n]$ be disjoint sets with sizes $n_{1}\leq n_{2}\leq n_{3}\leq n_{4}$. We consider two cases:

Case 1: $\left\lvert E(S_{1},S_{2})\right\rvert~\lesssim~n_{1}n_{2}p$

Note that each edge $(i_{1},i_{2})$ can only participate in at most $\left\lvert\mathsf{N}(i_{1})\cap\mathsf{N}(i_{2})\right\rvert$ triangles, and each triangle $(i_{1},i_{2},i_{3})$ can only be extended to at most $\left\lvert\mathsf{N}(i_{1})\cap\mathsf{N}(i_{2})\cap\mathsf{N}(i_{3})\right\rvert$ 4-cliques. Thus, ˜3.7.4 gives

[TABLE]

• -

Case 2: $\left\lvert E(S_{1},S_{2})\right\rvert~\lesssim~n_{1}+n_{2}$

The ˜3.7.5 gives

[TABLE]

which together with ˜3.7.4 implies

[TABLE]

Considering the first term, we note that

[TABLE]

since $n_{3}\leq n_{4}$ and $np^{3}\geq 1$. Similarly, for the second term, we have

[TABLE]

Combined with ˜3.7.3, this completes the proof of Lemma˜3.7.1.

3.7.2 Lower Bound on $\Lambda\left(f\right)$

Recall that given a random graph $G=([n],E)$ drawn from the distribution $G_{n,p}$, the polynomial $f$ is defined as

[TABLE]

where $\mathcal{C}\subseteq\binom{[n]}{4}$ is the set of $4$-cliques in $G$. Let $\mathsf{A}\in\mathbb{R}^{[n]^{2}\times[n]^{2}}$ be the natural matrix representation of $24f$ (corresponding to ordered copies of cliques) with

[TABLE]

Let $E^{\prime}\subseteq{[n]}^{2}$ be the set of ordered edges i.e., $(i_{1},i_{2})\in E^{\prime}$ if and only if $\left\{i_{1},i_{2}\right\}\in E$. Note that $|E^{\prime}|=2m$ where $m$ is the number of edges in $G$. All nonzero entries of $\mathsf{A}$ are contained in the principal submatrix $\mathsf{A}_{E^{\prime}}$, formed by the rows and columns indexed by $E^{\prime}$.

A simple lower bound on $\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}$

We first give a simple proof that $\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}\geq\sqrt{n^{2}p^{5}}$ with high probability.

Lemma 3.7.6**.**

$\mathchoice{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}{\lVert f\rVert_{\mathrm{sp}}}\geq\Omega(\sqrt{n^{2}p^{5}})=\Omega(n^{1/6})$* with high probability.*

• **Proof:  ** Consider any matrix representation $M$ of $24f$ and its principal submatrix $\mathsf{M}_{E^{\prime}}$. It is easy to observe that the Frobenius norm of $M_{E^{\prime}}$ satisfies ${\lVert M_{E^{\prime}}\rVert^{2}_{F}}\geq 24\left\lvert\mathcal{C}\right\rvert$, minimized when $M=\mathsf{A}$. Since ${\lVert M_{E^{\prime}}\rVert^{2}_{F}}\leq\left\lvert E^{\prime}\right\rvert\cdot{\lVert A_{E^{\prime}}\rVert^{2}_{2}}$, we have that with high probability,

[TABLE]

Lower bound for the stronger relaxation computing $\Lambda\left(f\right)$

We now prove Lemma˜3.7.2, which says that $\Lambda\left(f\right)\geq\frac{n^{1/6}}{\log^{2}n}$ with high probability. In order to show a lower bound, we look at the dual SDP for computing $\Lambda\left(f\right)$, which is a maximization problem over positive semidefinite, SoS-symmetric matrices $\mathsf{M}$ with $\mathop{\mathsf{Tr}}\!\left({\mathsf{M}}\right)=1$. We exhibit such a matrix $\mathsf{M}\in\mathbb{R}^{[n]^{2}\times[n]^{2}}$ for which the value of the objective $\left\langle\mathsf{A},\mathsf{M}\right\rangle$ is large.

For large $\langle\mathsf{A},\mathsf{M}\rangle$, one natural attempt is to take $\mathsf{M}$ to be $\mathsf{A}$ and modify it to satisfy other conditions. Note that $\mathsf{A}$ is already SoS-symmetric. However, $\mathop{\mathsf{Tr}}\!\left({\mathsf{A}}\right)=0$, which implies that the minimum eigenvalue is negative.

Let $\lambda_{\min}$ be the minimum eigenvalue of $\mathsf{A}$, which is also the minimum eigenvalue of $\mathsf{A}_{E^{\prime}}$. Let $I_{E^{\prime}}\in\mathbb{R}^{[n]^{2}\times[n]^{2}}$ be such that $I[(i_{1},i_{2}),(i_{1},i_{2})]=1$ if $(i_{1},i_{2})\in E^{\prime}$ and all other entries are [math]. Note that $I_{E^{\prime}}$ is a diagonal matrix with $\mathop{\mathsf{Tr}}\!\left({I_{E^{\prime}}}\right)=2m$. Adding $-\lambda_{\min}\cdot I_{E^{\prime}}$ to $\mathsf{A}$ makes it positive semidefinite, so setting

[TABLE]

makes sure that $\mathsf{M}$ is positive semidefinite, $\mathop{\mathsf{Tr}}\!\left({\mathsf{M}}\right)=1$, and $\left\langle\mathsf{A},\mathsf{M}\right\rangle=\frac{12|\mathcal{C}|}{m\cdot\left\lvert\lambda_{\min}\right\rvert}$ (each $4$-clique in $\mathcal{C}$ contributes $24$). Since $\left\lvert\mathcal{C}\right\rvert=\Theta(n^{4}p^{6})$ and $m=\Theta(n^{2}p)$ with high probability, if $\left\lvert\lambda_{\min}\right\rvert=O(np^{5/2})$, $\left\langle\mathsf{A},\mathsf{M}\right\rangle=\Theta\left(n^{2}p^{5/2}\right)$, which is $\Omega(n^{1/6})$ when $p=\Omega(n^{-1/3})$.

The $\mathsf{M}$ defined in Eq.˜3.10 does not directly work since it is not SoS-symmetric. However, the following claim proves that this issue can be fixed by losing a factor $2$ in $\langle\mathsf{A},\mathsf{M}\rangle$.

Claim 3.7.7**.**

There exists $\mathsf{M}$ such that it is SoS-symmetric, positive semidefinite with $\mathop{\mathsf{Tr}}\!\left({\mathsf{M}}\right)=1$, and $\langle\mathsf{A},\mathsf{M}\rangle\geq\frac{6|\mathcal{C}|}{m\cdot\left\lvert\lambda_{\min}\right\rvert}$.

• **Proof:  ** Let $Q_{E^{\prime}}\in\mathbb{R}^{[n]^{2}\times[n]^{2}}$ be the matrix such that

• –

For $(i_{1},i_{2})\in E^{\prime}$, $Q_{E^{\prime}}[(i_{1},i_{1}),(i_{2},i_{2})]=Q_{E^{\prime}}[(i_{2},i_{2}),(i_{1},i_{1})]=1$.

• –

For $i\in[n]$, $Q_{E^{\prime}}[(i,i),(i,i)]=\deg_{G}(i)$, where $\deg_{G}(i)$ denotes the degree of $i$ in $G$.

• –

All other entries are [math].

We claim that $I_{E^{\prime}}+Q_{E^{\prime}}$ is SoS-symmetric: $(I_{E^{\prime}}+Q_{E^{\prime}})[(i_{1},i_{2}),(i_{3},i_{4})]$ has a nonzero entry if and only if $i_{1}=i_{2}=i_{3}=i_{4}$ or two different numbers $j_{1},j_{2}$ appear exactly twice and $(j_{1},j_{2})\in E^{\prime}$ (in this case $(I_{E^{\prime}}+Q_{E^{\prime}})[(i_{1},i_{2}),(i_{3},i_{4})]=1$). Since $\mathsf{A}$ is SoS-symmetric, so $\mathsf{A}+\left\lvert\lambda_{\min}\right\rvert\cdot(I_{E^{\prime}}+Q_{E^{\prime}})$ is also SoS-symmetric.

It is easy to see that $Q_{E^{\prime}}$ is diagonally dominant, and hence positive semidefinite. Since we already argued that $\mathsf{A}+\left\lvert\lambda_{\min}\right\rvert\cdot I_{E^{\prime}}$ is positive semidefinite, $\mathsf{A}+\left\lvert\lambda_{\min}\right\rvert\cdot(I_{E^{\prime}}+Q_{E^{\prime}})$ is also positive semidefinite. Also, $\mathop{\mathsf{Tr}}\!\left({Q_{E^{\prime}}}\right)=\sum_{i\in[n]}\deg_{G}(i)=2m$. Thus, we take

[TABLE]

By the above arguments, we have that $\mathsf{M}$ that is PSD, SoS-symmetric with $\mathop{\mathsf{Tr}}\!\left({\mathsf{M}}\right)=1$, and

[TABLE]

as desired.

It only remains to bound $\lambda_{\min}$, which is the minimum eigenvalue of $\mathsf{A}$ and $\mathsf{A}_{E^{\prime}}$. For $p$ in the range $[n^{-1/3},n^{-1/4}]$, we will show a bound of $\tilde{O}(n^{3/2}p^{4})$ below, which when combined with the above claim, completes the proof of Lemma˜3.7.2.

Bounding the smallest eigenvalue via the trace method

Our estimate $\left\lvert\lambda_{\min}\right\rvert=O(np^{5/2})$ is based on the following observation: $\mathsf{A}_{E^{\prime}}$ is a $2m\times 2m$ random matrix where each row and column is expected to have $\Theta(n^{2}p^{5})$ ones (the expected number of $4$-cliques an edge participates in). An adjacency matrix of a random graph with average degree $d$ has a minimum eigenvalue $-\Theta(\sqrt{d})$, hence the estimate $\left\lvert\lambda_{\min}\right\rvert=O(np^{5/2})$. Even though $\mathsf{A}_{E^{\prime}}$ is not sampled from a typical random graph model (and even $E^{\prime}$ is a random variable), we will be able to prove the following weaker estimate, which suffices for our purposes.

Lemma 3.7.8**.**

With high probability over the choice of the graph $G$, we have

[TABLE]

• **Proof:  ** Instead of $\mathsf{A}_{E^{\prime}}$, we directly study $\mathsf{A}$ to bound $\lambda_{\min}$. For simplicity, we consider the following matrix $\widehat{\mathsf{A}}$, where each row and column is indexed by an unordered pair $\{i,j\}\in\binom{[n]}{2}$, and $\widehat{\mathsf{A}}[\{i_{1},i_{2}\},\{i_{3},i_{4}\}]=1$ if and only if $i_{1},i_{2},i_{3},i_{4}$ form a $4$-clique. $\mathsf{A}$ has only zero entries in the rows or columns indexed by $(i,i)$ for all $i\in[n]$, and for two pairs $i_{1}\neq i_{2}$ and $i_{3}\neq i_{4}$, we have

[TABLE]

Therefore, $\left\lvert\lambda_{\min}\left(\mathsf{A}\right)\right\rvert\leq 4\cdot\left\lvert\lambda_{\min}\left(\widehat{\mathsf{A}}\right)\right\rvert$ and it suffices to bound the minimum eigenvalue of $\widehat{\mathsf{A}}$. We consider the matrix $\widehat{\mathsf{N}}_{E}:=\widehat{\mathsf{A}}-p^{4}\cdot\widehat{\mathsf{J}}_{E}$, where $\widehat{\mathsf{J}}_{E}\in\mathbb{R}^{\binom{[n]}{2}\times\binom{[n]}{2}}$ is such that

[TABLE]

Since $\widehat{\mathsf{J}}_{E}$ is a rank-$1$ matrix with a positive eigenvalue, the minimum eigenvalues of $\widehat{\mathsf{A}}$ and $\widehat{\mathsf{N}}_{E}$ are the same. In summary, $\widehat{\mathsf{N}}_{E}$ is the following matrix.

[TABLE]

We use the trace method to bound $\mathchoice{\lVert\widehat{\mathsf{N}}_{E}\rVert_{2}}{\lVert\widehat{\mathsf{N}}_{E}\rVert_{2}}{\lVert\widehat{\mathsf{N}}_{E}\rVert_{2}}{\lVert\widehat{\mathsf{N}}_{E}\rVert_{2}}$, based on the observation that for every even $r\in\mathbb{N}$, $\mathchoice{\lVert\widehat{\mathsf{N}}_{E}\rVert_{2}}{\lVert\widehat{\mathsf{N}}_{E}\rVert_{2}}{\lVert\widehat{\mathsf{N}}_{E}\rVert_{2}}{\lVert\widehat{\mathsf{N}}_{E}\rVert_{2}}\leq\left(\mathop{\mathsf{Tr}}\!\left({(\widehat{\mathsf{N}}_{E})^{r}}\right)\right)^{1/r}$. Fix an even $r\in\mathbb{N}$. The expected value of the trace can be represented as

[TABLE]

where each $I^{j}=\left\{i^{j}_{1},i^{j}_{2}\right\}\in\binom{[n]}{2}$ is an edge of the complete graph on $n$ vertices (call it a potential edge) and $I^{r+1}:=I^{1}$.

Fix $r$ potential edges $I^{1},\dots,I^{r}$, let $t:=\prod_{k=1}^{r}\widehat{\mathsf{N}}_{E}[I^{k},I^{k+1}]$, and consider $\operatorname*{\mathbb{E}}[t]$. Let $E_{0}:=\{I^{1},\dots,I^{r}\}$ be the set of distinct edges represented by $I^{1},\dots,I^{r}$. Note that the expected value is [math] if one of $I^{j}$ does not become an edge. Therefore, $\operatorname*{\mathbb{E}}[t]=p^{|E_{0}|}\cdot\mathchoice{{\mathbb{E}}\left[t~|~E_{0}\subseteq E\right]}{{\mathbb{E}}[t~|~E_{0}\subseteq E]}{{\mathbb{E}}[t~|~E_{0}\subseteq E]}{{\mathbb{E}}[t~|~E_{0}\subseteq E]}$.

Let $D\subseteq[r]$ be the set of $j\in[r]$ such that all four vertices in $I^{j}$ and $I^{j+1}$ are distinct i.e.,

[TABLE]

For $j\in[r]\setminus D$, $\{i^{j}_{1},i^{j}_{2},i^{j+1}_{1},i^{j+1}_{2}\}$ cannot form a $4$-clique, so given that $I^{j},I^{j+1}\in E$, we have $\widehat{\mathsf{N}}_{E}[I^{j},I^{j+1}]=-p^{4}$. For $j\in D$, let $E_{j}:=\left\{\left\{i^{j}_{1},i^{j+1}_{1}\right\},\left\{i^{j}_{1},i^{j+1}_{2}\right\},\left\{i^{j}_{2},i^{j+1}_{1}\right\},\left\{i^{j}_{2},i^{j+1}_{2}\right\}\right\}\setminus E_{0}$ be the set of edges in the $4$-clique created by $\left\{i^{j}_{1},i^{j}_{2},i^{j+1}_{1},i^{j+1}_{2}\right\}$ except ones in $E_{0}$. Then

[TABLE]

Suppose there exists $j\in D$ such that $\left\lvert E_{j}\right\rvert=4$ and $E_{j}\cap(\cup_{j^{\prime}\in D\setminus\{j\}}E_{j^{\prime}})=\emptyset$. Then, given that $E_{0}\subseteq E$, $\widehat{\mathsf{N}}_{E}[I^{j},I^{j+1}]$ is independent of all $\left\{\widehat{\mathsf{N}}_{E}[I^{k},I^{k+1}]\right\}_{k\in D\setminus\{j\}}$, and

[TABLE]

Therefore, $\operatorname*{\mathbb{E}}[t]=0$ unless for all $j\in D$, either $|E_{j}|\leq 3$ or there exists $j^{\prime}\in D\setminus\{j\}$ with $E_{j}\cap E_{j^{\prime}}\neq\emptyset$.

Let $E_{D}:=\bigcup_{j\in D}E_{j}$. Note that $E_{0}$ and $E_{D}$ completely determines $t$. $\operatorname*{\mathbb{E}}[t]$ can be written as

[TABLE]

where $a(F)$ denotes the number of $j\in D$ with $E_{j}\not\subseteq F$. Since $E_{D}\subseteq F\cup\left(\bigcup_{j:E_{j}\not\subseteq F}E_{j})\right)$ and $4a(F)+|F|\geq|E_{D}|$, we have

[TABLE]

We now count the number of terms which contribute to the sum. Fix a graph $H$ with $r$ labelled edges $I^{1},\dots,I^{r}$ (possibly repeated) and $q:=q(H)$ vertices, without any isolated vertex (so $q\leq 2r$). There are at most $\binom{q}{2}^{r}\leq(2r)^{2r}$ such graphs. Then $I^{1},\dots,I^{r}$, as edges in $\binom{[n]}{2}$, are determined by a map $V_{H}\to[n]$. There are at most $n^{q}$ such mappings.

Let $E_{0}:=E_{0}(H),D:=D(H),E_{j}:=E_{j}(H),E^{\prime}:=E^{\prime}(H)$ be defined as before. Note that $E_{0}$ is set the edges of $H$. As observed before, the contribution from $H$ is [math] if there exists $j\in D$ such that $|E_{j}|=4$ and $E_{j}$ is disjoint from $\{E_{j^{\prime}}\}_{j^{\prime}\in D\setminus\{j\}}$. Let $\mathcal{H}$ be the set of $H$ that has nonzero contribution. Then,

[TABLE]

We will prove the following bound on the maximum contribution of any $H\in\mathcal{H}$.

Claim 3.7.9**.**

Let $\mathcal{H}$ be defined as above. Then, for all $H\in\mathcal{H}$, we have

[TABLE]

where

[TABLE]

Using the above claim, we can bound $\mathchoice{{\mathbb{E}}\left[\mathop{\mathsf{Tr}}\!\left({\left(\widehat{\mathsf{N}}_{E}\right)^{r}}\right)\right]}{{\mathbb{E}}[\mathop{\mathsf{Tr}}\!\left({\left(\widehat{\mathsf{N}}_{E}\right)^{r}}\right)]}{{\mathbb{E}}[\mathop{\mathsf{Tr}}\!\left({\left(\widehat{\mathsf{N}}_{E}\right)^{r}}\right)]}{{\mathbb{E}}[\mathop{\mathsf{Tr}}\!\left({\left(\widehat{\mathsf{N}}_{E}\right)^{r}}\right)]}$ as

[TABLE]

where $B_{p}$ is given by ˜3.7.9 for different ranges of $p$. By Markov’s inequality, we get that with probability $1-\frac{1}{n}$, we have $\mathop{\mathsf{Tr}}\!\left({\left(\widehat{\mathsf{N}}_{E}\right)^{r}}\right)\leq(8r)^{2r}\cdot n^{3}\cdot B_{p}^{r}$, which gives

[TABLE]

Choosing $r=\Theta(\log n)$ then proves the lemma.

It remains to prove ˜3.7.9.

Analyzing Contributing Subgraphs

Recall that graphs $H\in\mathcal{H}$ were constructed from edges $\{I^{1},\ldots,I^{r}\}$, with edge $I^{j}$ consisting of vertices $\{i_{1}^{j},i_{2}^{j}\}$. Also, we define $q(H)=\left\lvert V(H)\right\rvert$. Moreover, we defined the following sets for graph $H$

[TABLE]

Moreover, the graph $H$ is in $\mathcal{H}$ only if for every $j\in D$, either $\left\lvert E_{j}(H)\right\rvert\leq 3$ or there exists $j^{\prime}\in D\setminus\{j\}$ such that $E_{j}(H)\cap E_{j^{\prime}}(H)\neq\emptyset$. The ˜3.7.9 then follows from the following combinatorial claim (taking $b=\log(1/p)/\log n$).

Claim 3.7.10**.**

Any graph $H\in\mathcal{H}$ satisfies, for all $b\in[0,1/3]$

[TABLE]

where $c=5/3-14b/3$  for $b\in[0,1/4]$ and $c=3/2-4b$  for $b\in[1/4,1/3]$.

• **Proof:  ** Fix a graph $H\in\mathcal{H}$. Let $j=1,\dots,r$, let $V_{j}:=\left\{i^{j}_{1},i^{j}_{2}\right\}_{1\leq j\leq r}$ (i.e., the set of vertices covered by $I^{1},\dots,I^{j}$). For each $j=2,\dots,r$, let $v_{j}:=|V_{j}|-|V_{j-1}|$ and classify the index $j$ to one of the following types.

• –

Type $-1$: $I^{j}\cap I^{j-1}\neq\emptyset$ (equivalently, $j-1\notin D$).

• –

Type $k$ ($0\leq k\leq 2$): $I^{j}$ and $I^{j-1}$ are disjoint, and $v_{j}=k$ (i.e., adding $I^{j}$ introduces $k$ new vertices).

Let $T_{k}$ ($-1\leq k\leq 2$) be the set of indices of Type $k$, and let $t_{k}:=|T_{k}|$. The number of vertices $q$ is bounded by

[TABLE]

Let $H_{j}$ be the graph with $V_{j}$ as vertices and edges

[TABLE]

For $j=2,\dots,r$, let $e_{j}=\left\lvert E(H_{j})\right\rvert-\left\lvert E(H_{j-1})\right\rvert$. For an index $j\in T_{2}$, adding two vertices ${i^{j}_{1},i^{j}_{2}}$ introduces at least $5$ edges in $H_{j}$ compared to $H_{j-1}$ (i.e., six edges in the $4$-clique on $\left\{i^{j-1}_{1},i^{j-1}_{2},i^{j}_{1},i^{j}_{2}\right\}$ except $I^{j-1}$), so $e_{j}\geq 5$. Similarly, we get $e_{j}\geq 3$ for each $j\in T_{1}$.

The lemma is proved via the following charging argument. For each index $j=2,\dots,r$, we get value $b$ for each edge in $H_{j}\setminus H_{j-1}$ and get value $c$ for the new index. If $j\in T_{-1}$, we get an additional value of $4b$. We give this value to vertices in $V_{j}\setminus V_{j-1}$. If we do not give more value than we get and each vertex in $V(H)\setminus V_{1}$ gets more than $1$, this means

[TABLE]

proving the claim. For example, if $j$ is an index of Type $1$, it gets a value at least $3b+c$ and needs to give value $1$, such a charging can be done if $3b+c\geq 1$. Similarly, a type 0 vertex does not need to give any value and has a surplus. We will choose parameters so that each $j$ of types $-1$, $1$ or [math] can distribute the value to vertices added in $V_{j}\setminus V_{j-1}$. However, if $j$ is an index of Type $2$, it needs to distribute the value it gets ($5b+c$) to two vertices, and we will allow it to be “compensated” by vertices of other types, which may have a surplus.

Consider an index $j\in T_{2}$. The fact that $j\in T_{2}$ guarantees that earlier edges $I^{1},\dots,I^{j-1}$ are all vertex disjoint from $I^{j}$. If later edges $I^{j+1},\dots,I^{r}$ are all vertex disjoint from $I^{j}$, then $|E_{j-1}|=4$ and $E_{j-1}$ is disjoint from $\{E_{j^{\prime}}\}_{j^{\prime}\in D\setminus\{j-1\}}$, and this means that $H\not\in\mathcal{H}$. Thus, there exists $j^{\prime}>j$ such that $I^{j^{\prime}}$ and $I^{j}$ share a vertex. Take the smallest $j^{\prime}>j$, and say that $j^{\prime}$ compensates $j$. Note that $j^{\prime}\notin T_{2}$.

We will allow a type 1 index to compensate at most one of type 2 index, and a type -1 or 0 index to compensate at most two of type 2 indices. We consider below the constraints implied by each kind of compensation.

One Type $1$ index $j^{\prime}$ compensates one Type $2$ index $j$

$v_{j^{\prime}}+v_{j}=3$ and $e_{j^{\prime}}+e_{j}\geq 8$ ($5$ from $e_{j}$ and $3$ from $e_{j^{\prime}}$). This is possible if $8b+2c\geq 3$. 2. 2.

One Type [math] index $j^{\prime}$ compensates one Type $2$ index $j$

$v_{j^{\prime}}+v_{j}=2$ and $e_{j^{\prime}}+e_{j}\geq 5$ ($5$ from $e_{j}$). This is possible if $5b+2c\geq 2$. 3. 3.

One Type [math] index $j^{\prime}$ compensates two Type $2$ indices $j_{1}$ and $j_{2}$ (say $j_{1}<j_{2}$).

There are two cases.

1. (a)

$e_{j^{\prime}}+e_{j_{1}}+e_{j_{2}}\geq 11$: $v_{j^{\prime}}+v_{j_{1}}+v_{j_{2}}=4$. This is possible if $11b+3c\geq 4$. 2. (b)

$e_{j^{\prime}}+e_{j_{1}}+e_{j_{2}}=10$: since $e_{j_{1}},e_{j_{2}}\geq 5$, this means that $e_{j^{\prime}}=0$.

First, we note that since $j_{1}$ is a type 2 index and $j^{\prime}$ is the smallest index $j$ such that $I^{j_{1}}\cap I_{j}\neq\emptyset$, in the graph $H_{j^{\prime}-1}$, vertices in $I^{j_{1}}$ only have edges to vertices in $I^{j_{1}-1}$ and $I^{j_{1}+1}$. Similarly, vertices in $I^{j_{2}}$ only have edges to vertices in $I^{j_{2}-1}$ and $I^{j_{2}+1}$.

Since $I^{j^{\prime}}$ shares one vertex each with $I^{j_{1}}$ and $I^{j_{2}}$, let $I^{j^{\prime}}=\{i^{j^{\prime}}_{1},i^{j^{\prime}}_{2}\}$ with $i^{j^{\prime}}_{1}\in I^{j_{1}}$ and $i^{j^{\prime}}_{2}\in I^{j_{2}}$. Since $e_{j^{\prime}}=0$ means that $I^{j^{\prime}}=\{i^{j^{\prime}}_{1},i^{j^{\prime}}_{2}\}$ was in $H_{j^{\prime}-1}$. However, this is an edge between vertices in $I^{j_{1}}$ and $I^{j_{2}}$. By the above argument, this is only possible if $j_{2}=j_{1}+1$. Also, since $j^{\prime}$ is type 0 and $I^{j^{\prime}}$ shares a vertex with $I^{j_{2}}$, we must have $j^{\prime}>j_{2}+1$ (otherwise $j^{\prime}$ would be type -1).

Consider $I^{j^{\prime}-1}$, which are vertex disjoint from both $I^{j_{1}}$ and $I^{j_{2}}$. If $I^{j^{\prime}-1}\neq I^{j_{1}-1}$, at least one edge between $I^{j^{\prime}-1}$ and $I^{j}$ was not in $H_{j^{\prime}-1}$, contradicting the assumption $e_{j^{\prime}}=0$. Therefore, $I^{j^{\prime}-1}=I^{j_{1}-1}$. For the same reason, $I^{j^{\prime}-1}=I^{j_{2}+1}$. Thus, in particular, we have $I^{j_{2}+1}=I^{j_{1}-1}$. Thus, $j_{2}+1$ is also a type 0 index. Moreover, it cannot compensate any previous index, since any such index would already be compensated by $j_{1}-1$.

In this case we consider that $I^{j_{2}+1}$ and $I^{j^{\prime}}$ jointly compensate $j_{1}$ and $j_{2}$. $v_{j^{\prime}}+v_{j_{2}+1}+v_{j_{1}}+v_{j_{2}}=4$ and $e_{j_{2}+1}+e_{j^{\prime}}+e_{j_{1}}+e_{j_{2}}\geq 10$. Compensation is possible if $10b+4c\geq 4$. 4. 4.

One Type $-1$ index $j^{\prime}$ compensates one Type $2$ index $j$.

$v_{j^{\prime}}+v_{j}\leq 3$ and $e_{j^{\prime}}+e_{j}\geq 5$ ($5$ from $e_{j}$). Compensation is possible if $5b+4b+2c\geq 2$. 5. 5.

One Type $-1$ index $j^{\prime}$ compensates two Type $2$ indices $j_{1}$ and $j_{2}$

We have $v_{j^{\prime}}+v_{j_{1}}+v_{j_{2}}\leq 5$ and $e_{j^{\prime}}+e_{j_{1}}+e_{j_{2}}\geq 10$. Compensation is possible if $10b+4b+3c\geq 5$.

Each index $j$ of Type $2$ is compensated by exactly one other index $j^{\prime}$. We also require indices of types $1$ and $-1$ which do not compensate any other index, to have value at least 1 (to account for the one vertex added). This is true if $3b+c\geq 1$ and $4b+c\geq 1$.

Aggregating the above conditions (and discarding the redundant ones), we take

[TABLE]

It is easy to check that the maximum is attained by $c=5/3-14b/3$ when $b\in[0,1/4]$ and $c=3/2-4b$ when $b\in[1/4,1/3]$.

3.8 Lifting $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ Lower Bounds to Higher

Levels

For a matrix $B\in\Re^{[n]^{q/2}\times[n]^{q/2}}$, let $B^{S}$ denote the matrix obtained by symmetrizing $B$, i.e. for any $I,J\in[n]^{q/2}$,

[TABLE]

Equivalently, $B^{S}$ can be defined as follows:

[TABLE]

where for any $K\in[n]^{q}$, $B^{\pi}[K]:=B[\pi(K)]$.

For a matrix $M\in\Re^{[n]^{2}\times[n]^{2}}$ let $T\in\mathbb{R}^{[n]^{4}}$ denote the tensor given by, $T[i_{1},i_{2},i_{3},i_{4}]=M[(i_{1},i_{2}),(i_{3},i_{4})]$. Also for any non-negative integers $x,y$ satisfying $x+y=4$, let $M_{x,y}\in\Re^{[n]^{x}\times[n]^{y}}$ denote the matrix given by, $M[(i_{1},\dots,i_{x}),(j_{1},\dots j_{y})]=T[i_{1},\dots,i_{x},j_{1},\dots j_{y}]$. We will use the following result that we prove in Section˜3.8.3.

Theorem 3.8.1** (Lifting “Stable” Spectral Lower Bounds).**

Let $M\in\Re^{[n]^{2}\times[n]^{2}}$ be a degree-$4$-SOS-symmetric matrix satisfying

[TABLE]

Then for any $q$ divisible by $4$,

[TABLE]

3.8.1 Gap Between $\mathchoice{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}{\lVert\cdot\rVert_{\mathrm{sp}}}$ and $\mathchoice{\lVert\cdot\rVert_{\infty}}{\lVert\cdot\rVert_{\infty}}{\lVert\cdot\rVert_{\infty}}{\lVert\cdot\rVert_{\infty}}$

for Non-Neg. Coefficient Polynomials

Lemma 3.8.2**.**

Consider any homogeneous polynomial $g$ of even degree-$t$ and let $M_{g}\in\Re^{[n]^{t/2}\times[n]^{t/2}}$ be its SoS-symmetric matrix representation. Then $\mathchoice{\lVert g\rVert_{\mathrm{sp}}}{\lVert g\rVert_{\mathrm{sp}}}{\lVert g\rVert_{\mathrm{sp}}}{\lVert g\rVert_{\mathrm{sp}}}\geq{\lVert M_{g}\rVert^{2}_{F}}/\mathchoice{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}$.

• **Proof:  ** We know by strong duality, that

[TABLE]

The claim follows by substituting $X:=M_{g}/\mathchoice{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}$.

Theorem 3.8.3**.**

For any $q$ divisible by $4$ and $f$ as defined in Section˜3.7, we have that w.h.p.

[TABLE]

• **Proof:  ** Let $f$ be the degree-$4$ homogeneous polynomial as defined in Section˜3.7 and let $M=M_{f}$ be its SoS-symmetric matrix representation. Let $g:=f^{q/4}$ and let $M_{g}$ be its SoS-symmetric matrix representation. Thus $M_{g}=(M^{\otimes q/4})^{S}$ and it is easily verified that w.h.p., ${\lVert M\rVert^{2}_{F}}\geq\widetilde{\Omega}(n^{4}p^{6})=\widetilde{O}(n^{2})$ and also ${\lVert M_{g}\rVert^{2}_{F}}\geq\widetilde{\Omega}((n^{4}p^{6})^{q/4}/q^{O(q)})=\widetilde{\Omega}(n^{q/2}/q^{O(q)})$.

It remains to estimate $\mathchoice{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}$ so that we may apply Lemma˜3.8.2. Implicit, in the proof of Lemma˜3.7.8, is that w.h.p. $M$ has one eigenvalue of magnitude $O(n^{2}p)=O(n^{5/3})$ and at most $O(n^{2}p)=O(n^{5/3})$ eigenvalues of magnitude $\widetilde{O}(n^{3/2}p^{4})=\widetilde{O}(n^{1/6})$. Thus, $\mathchoice{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}=\widetilde{O}(n^{11/6})$ w.h.p. Now we have that $\mathchoice{\lVert M_{1,3}\rVert_{S_{1}}}{\lVert M_{1,3}\rVert_{S_{1}}}{\lVert M_{1,3}\rVert_{S_{1}}}{\lVert M_{1,3}\rVert_{S_{1}}}\leq\sqrt{n}\cdot\mathchoice{\lVert M_{1,3}\rVert_{F}}{\lVert M_{1,3}\rVert_{F}}{\lVert M_{1,3}\rVert_{F}}{\lVert M_{1,3}\rVert_{F}}=\sqrt{n}\cdot\mathchoice{\lVert M\rVert_{F}}{\lVert M\rVert_{F}}{\lVert M\rVert_{F}}{\lVert M\rVert_{F}}=\widetilde{O}(n^{3/2})$ w.h.p. Thus, on applying Theorem˜3.8.1 to $M/\widetilde{O}(n^{11/6})$, we get that $\mathchoice{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}{\lVert M_{g}\rVert_{S_{1}}}/\widetilde{O}(n^{11q/24})\leq 2^{O(q)}$ w.h.p.

Thus, applying Lemma˜3.8.2 yields the claim.

3.8.2 Tetris Theorem

Let $M\in\Re^{[n]^{2}\times[n]^{2}}$ be a degree-$4$ SoS-Symmetric matrix, let $M_{A}:=M_{3,1}\otimes M_{0,4}\otimes M_{3,1}$, let $M_{B}:=M_{3,1}\otimes M_{1,3}$, let $M_{C}:=M$ and let $M_{D}:=\mathop{\mathsf{Vec}}\!\left({M}\right)\mathop{\mathsf{Vec}}\!\left({M}\right)^{T}=M_{0,4}\otimes M_{4,0}$. For any permutation $\pi\in\mathbb{S}_{q/2}$ let $\overline{\pi}\in\mathbb{S}_{n^{q/2}}$ denote the permutation that maps any $i\in[n]^{q/2}$ to $\pi(i)$. Also let $P_{\pi}\in\Re^{[n]^{q/2}\times[n]^{q/2}}$ denote the row-permutation matrix induced by the permutation $\overline{\pi}$. Let $P:=\sum_{\pi\in\mathbb{S}_{q/2}}P_{\pi}$. Let $\mathcal{R}(a,b,c,d):={(c!^{\,2}\,2!^{\,2c})~(b!\,(2a+b)!\,3!^{\,2a+2b})~(d!\,(a+d)!\,4!^{\,a+2d})}$. Define

[TABLE]

Theorem 3.8.4**.**

Let $M\in\Re^{[n]^{2}\times[n]^{2}}$ be a degree-$4$-SOS-symmetric matrix. Then

[TABLE]

We shall prove this claim in Section˜3.8.4 after first exploring its consequences.

3.8.3 Lifting Stable Degree-$4$ Lower Bounds

Theorem 3.8.5** **(Lifting “Stable” Spectral Lower Bounds: Restatement

of Theorem˜3.8.1).

Let $M\in\Re^{{[n]}^{2}\times{[n]}^{2}}$ be a degree-$4$-SOS-symmetric matrix satisfying

[TABLE]

Then for any $q$ divisible by $4$,

[TABLE]

• **Proof:  ** Implicit in the proof of Theorem˜3.8.4 is the following:

[TABLE]

First note that $(q/4)!\cdot 4!^{\,q/4}/\mathcal{R}(a,b,c,d)\leq 2^{O(q)}$ since for any integers $i,j,k,l$

[TABLE]

Next note that $\mathchoice{\lVert M_{0,4}\rVert_{S_{1}}}{\lVert M_{0,4}\rVert_{S_{1}}}{\lVert M_{0,4}\rVert_{S_{1}}}{\lVert M_{0,4}\rVert_{S_{1}}}=\mathchoice{\lVert M\rVert_{F}}{\lVert M\rVert_{F}}{\lVert M\rVert_{F}}{\lVert M\rVert_{F}}\leq\mathchoice{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}\leq 1$. Combining this with the fact that $\mathchoice{\lVert M_{1,3}\rVert_{S_{1}}}{\lVert M_{1,3}\rVert_{S_{1}}}{\lVert M_{1,3}\rVert_{S_{1}}}{\lVert M_{1,3}\rVert_{S_{1}}},\mathchoice{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}{\lVert M\rVert_{S_{1}}}\leq 1$, we get that $\mathchoice{\lVert M_{A}\rVert_{S_{1}}}{\lVert M_{A}\rVert_{S_{1}}}{\lVert M_{A}\rVert_{S_{1}}}{\lVert M_{A}\rVert_{S_{1}}},\mathchoice{\lVert M_{B}\rVert_{S_{1}}}{\lVert M_{B}\rVert_{S_{1}}}{\lVert M_{B}\rVert_{S_{1}}}{\lVert M_{B}\rVert_{S_{1}}},\mathchoice{\lVert M_{C}\rVert_{S_{1}}}{\lVert M_{C}\rVert_{S_{1}}}{\lVert M_{C}\rVert_{S_{1}}}{\lVert M_{C}\rVert_{S_{1}}},\mathchoice{\lVert M_{D}\rVert_{S_{1}}}{\lVert M_{D}\rVert_{S_{1}}}{\lVert M_{D}\rVert_{S_{1}}}{\lVert M_{D}\rVert_{S_{1}}}\leq 1$, since $\mathchoice{\lVert X\otimes Y\rVert_{S_{1}}}{\lVert X\otimes Y\rVert_{S_{1}}}{\lVert X\otimes Y\rVert_{S_{1}}}{\lVert X\otimes Y\rVert_{S_{1}}}=\mathchoice{\lVert X\rVert_{S_{1}}}{\lVert X\rVert_{S_{1}}}{\lVert X\rVert_{S_{1}}}{\lVert X\rVert_{S_{1}}}\cdot\mathchoice{\lVert Y\rVert_{S_{1}}}{\lVert Y\rVert_{S_{1}}}{\lVert Y\rVert_{S_{1}}}{\lVert Y\rVert_{S_{1}}}$ for any (possibly rectangular) matrices $X$ and $Y$. Further note that Schatten $1$-norm is invariant to multiplication by a permutation matrix. Thus, the claim follows by applying triangle inequality to the $O_{q}(q^{q})$ terms in Section˜3.8.3.

Corollary 3.8.6** (Lifting “Stable” SoS Lower Bounds).**

Let $M\in\Re^{[n]^{2}\times[n]^{2}}$ be a degree-$4$-SOS-symmetric matrix satisfying

[TABLE]

Then for any $q$ divisible by $4$,

[TABLE]

(i.e. $\!\left({M^{\otimes q/4}}\right)^{S}$ is a degree-$q$ SOS moment matrix).

• **Proof:  ** Observe that Theorem˜3.8.4 implies the claim since $M_{A},M_{B},M_{C},$ and $M_{D}$ are PSD and the set of PSD matrices is closed under transpose, Kronecker product, scaling, conjugation, and addition.

3.8.4 Proof of Tetris Theorem

We start with defining a  hypergraphical matrix which will allow a more intuitive paraphrasing of Theorem˜3.8.4. By now, this is an important formalism in the context of SoS, and closely-related objects have been defined in several works, including [DM15], [RRS16], [BHK*+*16].

Hypergraphical Matrix

Definition 3.8.7**.**

For symbolic sets $L=\{\ell_{1},\dots\ell_{q_{1}}\},R=\{r_{1},\dots r_{q_{2}}\}$, a $d$-uniform template-hypergraph represented by $(L,R,E)$, is a $d$-uniform hypergraph on vertex set $L\uplus R$ with $E$ being the set of hyperedges.

For $I=(i_{1},\dots i_{q_{1}})[n]^{q_{1}},J=(j_{1},\dots j_{q_{2}})\in[n]^{q_{2}}$, we also define a related object called edge-set instantiation (and denoted by $E(I,J)$) as the set of size-$d$ multisets induced by $E$ on substituting $\ell_{t}=i_{t}$ and $r_{t}=j_{t}$.

Remark.

There is a subtle distinction between $E$ and $E(I,J)$ above, in that $E$ is a set of $d$-sets and $E(I,J)$ is a set of size-$d$ multisets (i.e. $e\in E(I,J)$ can have repeated elements).

Definition 3.8.8**.**

Given an SoS-symmetric order-$d$ tensor $\mathsf{T}$ and a $d$-uniform template-hypergraph $H=(L,R,E)$ with $|L|=q_{1},|R|=q_{2}$, we define the $d$-uniform degree-$(q_{1},q_{2})$ hypergraphical matrix $\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H)$ as

[TABLE]

for any $I\in[n]^{q_{1}},J\in[n]^{q_{2}}$.

In order to represent Theorem˜3.8.4 in the language of hypergraphical matrices, we first show how to represent $M^{\otimes q/4}$ and $M_{A}^{\otimes a}\otimes M_{B}^{\otimes b}\otimes M_{C}^{\otimes c}\otimes M_{D}^{\otimes d}$ in this language.

Kronecker Products of Hypergraphical Matrices

We begin with the observation that the kronecker product of hypergraphical matrices yields another hypergraphical matrix (corresponding to the "disjoint-union" of the template-hypergraphs).

Definition 3.8.9**.**

let $H=(L,R,E)$, $H^{\prime}=(L^{\prime},R^{\prime},E^{\prime})$ be template-hypergraphs with $|L|=q_{1},|R|=q_{2},|L^{\prime}|=q_{3},|R^{\prime}|=q_{4}$. Let $\mkern 1.2mu\overline{\mkern-1.2muH\mkern-1.2mu}\mkern 1.2mu=(\mkern 1.2mu\overline{\mkern-1.2muL\mkern-1.2mu}\mkern 1.2mu,\mkern 1.2mu\overline{\mkern-1.2muR\mkern-1.2mu}\mkern 1.2mu,\mkern 1.2mu\overline{\mkern-1.2muE\mkern-1.2mu}\mkern 1.2mu)$ be a template-hypergraph with $|\mkern 1.2mu\overline{\mkern-1.2muL\mkern-1.2mu}\mkern 1.2mu|=q_{1}+q_{3},|\mkern 1.2mu\overline{\mkern-1.2muR\mkern-1.2mu}\mkern 1.2mu|=q_{2}+q_{4}$, where $\mkern 1.2mu\overline{\mkern-1.2mu\ell\mkern-1.2mu}\mkern 1.2mu_{t}=\ell_{t}$  if $t\in[q_{1}]$, $\mkern 1.2mu\overline{\mkern-1.2mu\ell\mkern-1.2mu}\mkern 1.2mu_{t}=\ell^{\prime}_{t}$  if $t\in[q_{1}+1,q_{1}+q_{3}]$, $\mkern 1.2mu\overline{\mkern-1.2mur\mkern-1.2mu}\mkern 1.2mu_{t}=r_{t}$  if $t\in[q_{2}]$, $\mkern 1.2mu\overline{\mkern-1.2mur\mkern-1.2mu}\mkern 1.2mu_{t}=r^{\prime}_{t}$  if $t\in[q_{2}+1,q_{2}+q_{4}]$, and $\mkern 1.2mu\overline{\mkern-1.2muE\mkern-1.2mu}\mkern 1.2mu=E\uplus E^{\prime}$. We call $\mkern 1.2mu\overline{\mkern-1.2muH\mkern-1.2mu}\mkern 1.2mu$ the disjoint-union of $H$ and $H^{\prime}$, which we denote by $H\uplus H^{\prime}$.

Observation 3.8.10**.**

Let $\mathsf{T}$ be an SOS-symmetric order-$d$ tensor and let $H=(L,R,E)$, $H^{\prime}=(L^{\prime},R^{\prime},E^{\prime})$ be template-hypergraphs. Then,

[TABLE]

Remark.

Note that the disjoint-union operation on template-hypergraphs does not commute, i.e. $\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H\uplus H^{\prime})\neq\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H^{\prime}\uplus H)$ (since kronecker-product does not commute).

Now consider a degree-$4$ SoS-symmetric matrix $M$ (as in the statment of Theorem˜3.8.4) and let $\mathsf{T}$ be the SoS-symmetric tensor corresponding to $M$. Then for any $x+y=4$ we have that $M_{x,y}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H_{x,y})$, where $H_{x,y}=(L,R,E)$ is the template-hypergraph satisfying $L=\{\ell_{1},\dots\ell_{x}\},R=\{r_{1},\dots r_{y}\}$ and $E=\{\{\ell_{1},\dots\ell_{x},r_{1},\dots r_{y}\}\}$. Combining this observation with ˜3.8.10 yields that $M_{A}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H_{A}),~M_{B}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H_{B}),~M_{C}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H_{C}),~M_{D}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H_{D})$, where $H_{A}:=H_{3,1}\uplus H_{0,4}\uplus H_{3,1}$, $H_{B}:=H_{3,1}\uplus H_{1,3}$,  $H_{C}:=H_{2,2}$, and $H_{D}:=H_{4,0}\uplus H_{0,4}$. Lastly, another application of ˜3.8.10 to the above, yields

Observation 3.8.11**.**

For a template-hypergraph $H$, let $H^{\uplus t}$ denote $\biguplus_{g\in[t]}H$. Then,

1. (1)

$M^{\otimes q/4}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H_{2,2}^{\uplus q/4})$. 2. (2)

$M_{A}^{\otimes a}\otimes M_{B}^{\otimes b}\otimes M_{C}^{\otimes c}\otimes M_{D}^{\otimes d}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H(a,b,c,d))$* where*

[TABLE]

For technical reasons we also define the following related template-hypergraph:

[TABLE]

where $\overline{H}_{A}$ is the template-hypergraph whose corresponding hypergraphical matrix is $M_{A}^{T}$.

To finish paraphrasing Theorem˜3.8.4, we are left with studying the effect of permutations on hypergraphical matrices - which is the content of the following section.

Hypergraphical Matrices under Permutation

Recall that for any matrix $B\in\Re^{[n]^{q/2}\times[n]^{q/2}}$ and $\pi\in\mathbb{S}_{q}$, $B^{\pi}$ is the matrix satisfying $B^{\pi}[K]:=B[\pi(K)]$ where $K\in[n]^{q/2}\times[n]^{q/2}$.

Also recall that for any permutation $\sigma\in\mathbb{S}_{q/2}$, $\overline{\sigma}\in\mathbb{S}_{n^{q/2}}$ denotes the permutation that maps any $i\in[n]^{q/2}$ to $\sigma(i)$ and also that $P_{\sigma}\in\Re^{[n]^{q/2}\times[n]^{q/2}}$ denotes the $[n]^{q/2}\times[n]^{q/2}$ row-permutation matrix induced by the permutation $\overline{\sigma}$.

We next define a permuted template hypergraph in order to capture how permuting a hypergraphical matrix (in the senses above) can be seen as permutations of the vertex set of the hypergraph.

Definition 3.8.12** (Permuted Template-Hypergraph).**

For any $\pi\in\mathbb{S}_{q}$ (even $q$), and a $d$-uniform template-hypergraph $H=(L,R,E)$ with $|L|=|R|=q/2$, let $H^{\pi}=(L^{\prime},R^{\prime},E)$ denote the template-hypergraph obtained by setting $\ell^{\prime}_{t}:=k_{t}$  and $r_{t}=k_{t+q/2}$  for $t\in[q/2]$, where $K=(k_{1},\dots k_{q})=\pi(L\oplus R)$.

Similarly, for any $\sigma_{1},\sigma_{2}\in\mathbb{S}_{q/2}$, let $H^{\sigma_{1},\sigma_{2}}=(L^{\prime},R^{\prime},E)$ denote the template-hypergraph obtained by setting $\ell^{\prime}_{t}:=\sigma_{1}(\ell_{t})$  and $r^{\prime}_{t}=\sigma_{2}(r_{t})$.

We then straightforwardly obtain

Observation 3.8.13**.**

For any $\pi\in\mathbb{S}_{q}$, $\sigma_{1},\sigma_{2}\in\mathbb{S}_{q/2}$, SoS-symmetric order-$d$ tensor $\mathsf{T}$ and any $d$-uniform template-hypergraph $H$,

1. (1)

$\!\left({\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H)}\right)^{\pi}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H^{\pi})$** 2. (2)

$P_{\sigma_{1}}\cdot\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H)\cdot P^{T}_{\sigma_{2}}=\mathsf{M}_{\textrm{hyp}}^{\mathsf{T}}(H^{\sigma_{1},\sigma_{2}})$**

Thus, to prove Theorem˜3.8.4, it remains to establish

[TABLE]

We will establish this in the next section by comparing the template-hypergraphs generated (with multiplicities) in the LHS with those generated in the RHS.

Proof of Section˜3.8.4

We start with some definitions to track the template-hypergraphs generated in the LHS and RHS of Section˜3.8.4. For any $12a+8b+4c+8d=q$, let

[TABLE]

Firstly, it is easily verified that whenever $(a,b,c,d)\neq(a^{\prime},b^{\prime},c^{\prime},d^{\prime})$, $\mathcal{F}(a,b,c,d)\cap\mathcal{F}(a^{\prime},b^{\prime},c^{\prime},d^{\prime})=\phi$, and that $\mathcal{F}(a,b,c,d)\cap\mkern 1.2mu\overline{\mkern-1.2mu\mathcal{F}\mkern-1.2mu}\mkern 1.2mu(a,b,c,d)=\phi$. It is also easily verified that for any $12a+8b+4c+8d=q$, and any $H\in\mathcal{F}(a,b,c,d)$,

[TABLE]

and for any $H\in\mathcal{F}$,

[TABLE]

Thus in order to prove Section˜3.8.4, it is sufficient to establish that

[TABLE]

It is sufficient to establish that

[TABLE]

since the other direction is straightforward. To this end, consider any $H=(L,R,E)\in\mathcal{F}$, and for any $x+y=4$, define

[TABLE]

Now clearly $H\in\mathcal{F}(a,b,c,d)$ iff

[TABLE]

and $H\in\mkern 1.2mu\overline{\mkern-1.2mu\mathcal{F}\mkern-1.2mu}\mkern 1.2mu(a,b,c,d)$ iff

[TABLE]

Thus we need only find $12a^{\prime}+8b^{\prime}+4c^{\prime}+8d^{\prime}=q$, such that Eq.˜3.16 or Eq.˜3.17 is satisfied.

We will assume w.l.o.g. that $s_{0,4}\geq s_{4,0}$ and show that one can satisfy Eq.˜3.16, since if $s_{(0,4)}<s_{(4,0)}$, an identical argument allows one to show that Eq.˜3.17 is satisfiable. So let $d^{\prime}=s_{4,0}$, $c^{\prime}=s_{2,2}$, $b^{\prime}=s_{1,3}$ and $a^{\prime}=(s_{3,1}-s_{1,3})/2$. Since $H\in\mathcal{F}$, it must be true that $4s_{4,0}+3s_{3,1}+2s_{2,2}+s_{(1,3)}=q/2$. Thus, $12a^{\prime}+8b^{\prime}+4c^{\prime}+8d^{\prime}=8s_{4,0}+6s_{3,1}+4s_{2,2}+2s_{1,3}=q$ as desired. We will next see that $(a^{\prime},b^{\prime},c^{\prime},d^{\prime})$ and $s_{x,y}$ satisfy Eq.˜3.16. We have by construction that $s_{4,0}=d^{\prime}$, $s_{2,2}=c^{\prime}$, $s_{1,3}=b^{\prime}$ and $s_{3,1}=2a^{\prime}+b^{\prime}$. It remains to show that $s_{0,4}=a^{\prime}+d^{\prime}$. Now we know that $4s_{4,0}+3s_{3,1}+2s_{2,2}+s_{1,3}=q/2$ and $4s_{0,4}+3s_{1,3}+2s_{2,2}+s_{3,1}=q/2$. Subtracting the two equations yields $s_{0,4}-s_{4,0}=(s_{3,1}-s_{1,3})/2$. This implies $a^{\prime}+d^{\prime}=s_{4,0}+(s_{3,1}-s_{1,3})/2=s_{0,4}$, and furthermore it implies that $a^{\prime}$ is non-negative since we assumed $s_{0,4}\geq s_{4,0}$. So Eq.˜3.16 is satisfied. Thus we have established Eq.˜3.15, which completes the proof of Theorem˜3.8.4.

Chapter 4 Matrix Norm: Hardness and Approximation

4.1 Introduction

We consider the problem of finding the $p\to q$ of a given matrix $A\in{\mathbb{R}}^{m\times n}$, which is defined as

[TABLE]

The quantity $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}$ is a natural generalization of the well-studied spectral norm, which corresponds to the case $p=q=2$. For general $p$ and $q$, this quantity computes the maximum distortion (stretch) of the operator $A$ from the normed space $\ell_{p}^{n}$ to $\ell_{q}^{m}$.

The case when $p=\infty$ and $q=1$ is the well known Grothendieck problem [KN12, Pis12], where the goal is to maximize $\left\langle y,Ax\right\rangle$ subject to $\mathchoice{\lVert x\rVert_{\infty}}{\lVert x\rVert_{\infty}}{\lVert x\rVert_{\infty}}{\lVert x\rVert_{\infty}},\mathchoice{\lVert y\rVert_{\infty}}{\lVert y\rVert_{\infty}}{\lVert y\rVert_{\infty}}{\lVert y\rVert_{\infty}}\leq 1$. In fact, via simple duality arguments (see Section˜4.2), the general problem computing $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}$ can be seen to be equivalent to the following variant of the Grothendieck problem (and to $\mathchoice{\lVert A^{T}\rVert_{q^{*}\to p^{*}}}{\lVert A^{T}\rVert_{q^{*}\to p^{*}}}{\lVert A^{T}\rVert_{q^{*}\to p^{*}}}{\lVert A^{T}\rVert_{q^{*}\to p^{*}}}$)

[TABLE]

where $p^{*},q^{*}$ denote the dual norms of $p$ and $q$, satisfying $1/p+1/p^{*}=1/q+1/q^{*}=1$.

Hypercontractive norms.

The case when $p<q$, known as the case of hypercontractive norms, also has a special significance to the analysis of random walks, expansion and related problems in hardness of approximation [Bis11, BBH*+*12]. The problem of computing $\mathchoice{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}$ is also known to be equivalent to determining the maximum acceptance probability of a quantum protocol with multiple unentangled provers, and is related to several problems in quantum information theory [HM13, BH15].

Bounds on hypercontractive norms of operators are also used to prove expansion of small sets in graphs. Indeed, if $f$ is the indicator function of set $S$ of measure $\delta$ in a graph with adjacency matrix $A$, then we have that for any $p\leq q$,

[TABLE]

It was proved by Barak et al. [BBH*+*12] that the above connection to small-set expansion can in fact be made two-sided for a special case of the $2\to q$. They proved by that to resolve the promise version of the small-set expansion (SSE) problem, it suffices to distinguish the cases $\mathchoice{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}\leq c\cdot\sigma_{\min}$ and $\mathchoice{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}\geq C\cdot\sigma_{\min}$, where $\sigma_{\min}$ is the least non-zero singular value of $A$ and $C>c>1$ are appropriately chosen constants based on the parameters of the SSE problem. Thus, the approximability of $2\to q$ is closely related to the small-set expansion problem. In particular, proving the NP-hardness of approximating the $2\to q$ is (necessarily) an intermediate goal towards proving the Small-Set Expansion Hypothesis of Raghavendra and Steurer [RS10]. However, relatively few results algorithmic and hardness results are known for approximating hypercontractive norms. Interpolation algorithms developed in [Ste05, GMU23] give an upper bound of $O(\max\left\{m,n\right\}^{3-2\sqrt{2}})$ on the best possible approximation factor. For the case of $2\to q$ (for any $q>2$), Barak et al. [BBH*+*12] give an approximation algorithm for the promise version of the problem described above, running in time $\exp\left(\tilde{O}(n^{2/q})\right)$. They also provide an additive approximation algorithm for the $2\to 4$ (where the error depends on $2\to 2$ and $2\to\infty$ of $A$), which was extended to the $2\to q$ by Harrow and Montanaro [HM13]. Some average case algorithms based on SoS hierarchy was recently developed in [DHPT25]. The result concerns

SoS certifiability of sub-gaussian matrices, which can be viewed as finding SoS upper-bound for $2\to q$ norms, for even $q$s, of i.i.d sub-gaussian matrices. Barak et al. also prove NP-hardness of approximating $\mathchoice{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}$ within a factor of $1+\tilde{O}(1/n^{o(1)})$, and hardness of approximating better than $\exp{O((\log n)^{1/2-\varepsilon})}$ in polynomial time, assuming the Exponential Time Hypothesis (ETH). This reduction was also used by Harrow, Natarajan and Wu [HNW16] to prove that $\tilde{O}(\log n)$ levels of the Sum-of-Squares SDP hierarchy cannot approximate $\mathchoice{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}{\lVert A\rVert_{2\to 4}}$ within any constant factor.

It is natural to ask if the bottleneck in proving (constant factor) hardness of approximation for $2\to q$ arises from the fact from the nature of the domain (the $\ell_{2}$ ball) or from hypercontractive nature of the objective. As discussed in Section˜4.1.5, all hypercontractive norms present a barrier for gadget reductions, since if a “true” solution $x$ is meant to encode the assignment to a (say) label cover problem with consistency checked via local gadgets, then (for $q>p$), a “cheating solution” may make the value of $\mathchoice{\lVert Ax\rVert_{q}}{\lVert Ax\rVert_{q}}{\lVert Ax\rVert_{q}}{\lVert Ax\rVert_{q}}$ very large by using a sparse $x$ which does not carry any meaningful information about the underlying label cover problem.

We show that (somewhat surprisingly, at least for the authors) it is indeed possible to overcome the barrier for gadget reductions for hypercontractive norms, for any $2<p<q$ (and by duality, for any $p<q<2$). This gives the first NP-hardness result for hypercontractive norms (under randomized reductions). Assuming ETH, this also rules out a constant factor approximation algorithm that runs in $2^{n^{\delta}}$ for some $\delta:=\delta(p,q)$.

Theorem 4.1.1**.**

For any $p,q$ such that $1<p\leq q<2$ or $2<p\leq q<\infty$ and a constant $c>1$, it is NP-hard to approximate $p\to q$ within a factor of $c$. The reduction runs in time $n^{B_{p}\cdot q}$ for $2<p<q$, where $B_{p}=\operatorname{poly}(1/(1-\gamma_{p^{*}}))$.

We show that the above hardness can be strengthened to any constant factor via a simple tensoring argument. In fact, this also shows that it is hard to approximate $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}$ within almost polynomial factors unless NP is in randomized quasi-polynomial time. This is the content of the following theorem.

Theorem 4.1.2**.**

For any $p,q$ such that $1<p\leq q<2$ or $2<p\leq q<\infty$ and $\epsilon>0$, there is no polynomial time algorithm that approximates the $p\to q$ of an $n\times n$ matrix within a factor $2^{\log^{1-\epsilon}n}$ unless $\textsf{NP}\subseteq\textsf{BPTIME}\left(2^{(\log n)^{O(1)}}\right)$. When $q$ is an even integer, the same inapproximability result holds unless $\textsf{NP}\subseteq\textsf{DTIME}\left(2^{(\log n)^{O(1)}}\right)$

We also note that the operator $A$ arising in our reduction in Theorem˜4.1.1 satisfies $\sigma_{\min}(A)\approx 1$ (and is in fact a product of a carefully chosen projection and a scaled random Gaussian matrix). For such an $A$, we prove the hardness of distinguishing $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}\leq c$ and $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}\geq C$, for constants $C>c>1$. For the corresponding problem in the case of $2\to q$, Barak et al. [BBH*+*12] gave a subexponential algorithm running in time $\exp\left(O(n^{2/q})\right)$ (which works for every $C>c>1$). On the other hand, since the running time of our reduction is $n^{O(q)}$, we get that assuming ETH, we show that no algorithm can distinguish the above cases for $p\to q$ in time $\exp\left(n^{o(1/q)}\right)$, for any $p\leq q$ when $2\notin[p,q]$.

While the above results give some possible reductions for working with hypercontractive norms, it remains an interesting problem to understand the role of the domain as a barrier to proving hardness results for the $2\to q$ problems. In fact, no hardness results are available even for the more general problem of polynomial optimization over the $\ell_{2}$ ball. We view the above theorem as providing some evidence that while hypercontractive norms have been studied as a single class so far, the case when $2\in[p,q]$ may be qualitatively different (with respect to techniques) from the case when $2\notin[p,q]$. This is indeed known to be true in the non-hypercontractive case with $p\geq q$. In fact, our results are obtained via new hardness results for the case $p\geq q$, as described below.

The non-hypercontractive case.

In this case, $q\geq p\geq 1$. For non-negative matrix, this can be efficiently approximated. The work [BV11] showed a polynomial time algorithm and recently [OV25] showed a near linear time algorithm. So we are mainly interested in the general matrix case. Several results are known in the case when $p\geq q$, and we summarize known results for matrix norms in Fig.˜4.1, for the both the hypercontractive and non-hypercontractive cases.

While the case of $p=q=2$ corresponds to the spectral norm, the problem is also easy when $q=\infty$ (or equivalently $p=1$) since this corresponds to selecting the row of $A$ with the maximum $\ell_{p^{*}}$ norm. Note that in general, Fig.˜4.1 is symmetric about the principal diagonal. Also note that if $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}$ is a hypercontractive norm ($p<q$) then so is the equivalent $\mathchoice{\lVert A^{T}\rVert_{q^{*}\to p^{*}}}{\lVert A^{T}\rVert_{q^{*}\to p^{*}}}{\lVert A^{T}\rVert_{q^{*}\to p^{*}}}{\lVert A^{T}\rVert_{q^{*}\to p^{*}}}$ (the hypercontractive and non-hypercontractive case are separated by the non-principal diagonal).

As is apparent from the figure, the problem of approximating $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}$ for $p\geq q$ admits good approximations when $2\in[q,p]$, and is hard otherwise. For the case when $2\notin[q,p]$, an upper bound of $O(\max\{m,n\}^{3-2\sqrt{2}})$ on the approximation ratio was proved in [Ste05, GMU23]. Bhaskara and Vijayaraghavan [BV11] showed NP-hardness of approximation within any constant factor, and hardness of approximation within an $O\left(2^{(\log n)^{1-\varepsilon}}\right)$ factor for arbitrary $\varepsilon>0$ assuming $\textsf{NP}\not\subseteq\textsf{DTIME}\left(2^{(\log n)^{O(1)}}\right)$.

Determining the right constants in these approximations when $2\in[q,p]$ has been of considerable interest in the analysis and optimization community. For the case of $\infty\to 1$, Grothendieck’s theorem [Gro56] shows that the integrality gap of a semidefinite programming (SDP) relaxation is bounded by a constant, and the (unknown) optimal value is now called the Grothendieck constant $K_{G}$. Krivine [Kri77] proved an upper bound of $\pi/(2\ln(1+\sqrt{2}))=1.782\ldots$ on $K_{G}$, and it was later shown by Braverman et al. that $K_{G}$ is strictly smaller than this bound. The best known lower bound on $K_{G}$ is about $1.676$, due to (an unpublished manuscript of) Reeds [Ree91] (see also [KO09] for a proof).

An upper bound of $K_{G}$ on the approximation factor also follows from the work of Nesterov [Nes98] for any $p\geq 2\geq q$. A later work of Steinberg [Ste05] also gave an upper bound of $\min\left\{\gamma_{p}/\gamma_{q},\gamma_{q^{*}}/\gamma_{p^{*}}\right\}$, where $\gamma_{p}$ denotes $p^{th}$ norm of a standard normal random variable (i.e., the $p$-th root of the $p$-th Gaussian moment). Note that Steinberg’s bound is less than $K_{G}$ for some values of $(p,q)$, in particular for all values of the form $(2,q)$ with $q\leq 2$ (and equivalently $(p,2)$ for $p\geq 2$), where it equals $1/\gamma_{q}$ (and $1/\gamma_{p^{*}}$ for $(p,2)$).

On the hardness side, Briët, Regev and Saket [BRS15] showed NP-hardness of $\pi/2$ for the $\infty\to 1$, strengthening a hardness result of Khot and Naor based on the Unique Games Conjecture (UGC) [KN08] (for a special case of the Grothendieck problem when the matrix $A$ is positive semidefinite). Assuming UGC, a hardness result matching Reeds’ lower bound was proved by Khot and O’Donnell [KO09], and hardness of approximating within $K_{G}$ was proved by Raghavendra and Steurer [RS09b].

For a related problem known as the $L_{p}$-Grothendieck problem, where the goal is to maximize $\left\langle x,Ax\right\rangle$ for $\mathchoice{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}\leq 1$, results by Steinberg [Ste05] and Kindler, Schechtman and Naor [KNS10] give an upper bound of $\gamma_{p}^{2}$, and a matching lower bound was proved assuming UGC by [KNS10], which was strengthened to NP-hardness by Guruswami et al. [GRSW16]. However, note that this problem is quadratic and not necessarily bilinear, and is in general much harder than the Grothendieck problems considered here. In particular, the case of $p=\infty$ only admits an $\Theta(\log n)$ approximation instead of $K_{G}$ for the bilinear version [AMMN06, ABH*+*05].

We extend the hardness results of [BRS15] for the $\infty\to 1$ and $2\to 1$ norms of a matrix to any $p\geq 2\geq q$. The hardness factors obtained match the performance of known algorithms (due to Steinberg [Ste05]) for the cases of $2\to q$ and $p\to 2$.

Theorem 4.1.3**.**

For any $p,q$ such that $\infty\geq p\geq 2\geq q\geq 1$ and $\epsilon>0$, it is NP-hard to approximate the $p\to q$ within a factor $1/(\gamma_{p^{*}}\gamma_{q})-\epsilon$.

Both Theorem˜4.1.1 and Theorem˜4.1.3 are consequences of a more technical theorem, which proves hardness of approximating $\mathchoice{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}$ for $r<2$ (and hence $\mathchoice{\lVert A\rVert_{r^{*}\to 2}}{\lVert A\rVert_{r^{*}\to 2}}{\lVert A\rVert_{r^{*}\to 2}}{\lVert A\rVert_{r^{*}\to 2}}$ for $r^{*}>2$) while providing additional structure in the matrix $A$ produced by the reduction. This is proved in Section˜4.3. We also show our methods can be used to provide a simple proof (albeit via randomized reductions) of the $2^{\Omega((\log n)^{1-\varepsilon})}$ hardness for the non-hypercontractive case when $2\notin[q,p]$, which was proved by [BV11]. This is presented in Section˜4.4.5.

Some of these hardness results imply approximation lower bounds on some natural versions of sparse Principal Component Analysis (PCA) as discussed in [ACCV21]. Technique similar to the one used in this hardness result was also issued to prove a hardness result for a minimization version of $p\to q$ norm problem in [BBB*+*19].

The candidate hard instance arising from our approach (as well as [BRS15]) leads to a PSD matrix. For the norm $\infty\to 1$i.e., the little Grothendieck problem, our hardness factor is $\nicefrac{{\pi}}{{2}}$. It was shown in a subsequent work [BLT22] that the hardness factor for the general matrix is at least $\nicefrac{{\pi}}{{2}}+\varepsilon_{0}$ for some $\varepsilon_{0}$. This required them to go beyond our technique here.

We also give an improved approximation for $p\to q$ when $2\in[q,p]$ (inspired by the above hardness result) which achieves an approximation factor of $C_{0}\cdot(1/(\gamma_{p^{*}}\gamma_{q}))$, where $C_{0}\approx 1/(\ln(1+\sqrt{2}))$ is a constant comparable to that arising in Krivine’s upper bound on the Grothendieck constant [Kri77].

The work [BRW21], demonstrated a use of this algorithm for approximately solving a variant of low rank approximation problem. In their iterative algorithm, each rank one update turns out to be the outer product of the vector achieving $p\to 2$-norm of a matrix ($p\geq 2$).

4.1.1 Hardness Results: Collected

For the sake accessibility, we collect the hardness results proved in this chapter:

Theorem 4.1.4**.**

We group the results using various range of $p$ and $q$ and hypercontractivity:

1. (1)

For any $p,q$ such that $1<p\leq q<2$ or $2<p\leq q<\infty$ and $\epsilon>0$: there is no polynomial time algorithm that approximates the $p\to q$ of an $n\times n$ matrix within a factor $2^{\log^{1-\epsilon}n}$ unless $\textsf{NP}\subseteq\textsf{BPTIME}\left(2^{(\log n)^{O(1)}}\right)$. When $q$ is an even integer, the same inapproximability result holds unless $\textsf{NP}\subseteq\textsf{DTIME}\left(2^{(\log n)^{O(1)}}\right)$ 2. (2)

For any $p,q$ such that $\infty\geq p\geq q>2$ or $2>p\geq q\geq 1$ and $\epsilon>0$: there is no polynomial time algorithm that approximates the $p\to q$ of an $n\times n$ matrix within a factor $2^{\log^{1-\epsilon}n}$ unless $\textsf{NP}\subseteq\textsf{BPTIME}\left(2^{(\log n)^{O(1)}}\right)$.

We note here that the hardness result proved in **[BV11]** is slightly stronger due to the weaker assumption of $\textsf{NP}\not\subseteq\textsf{DTIME}\left(2^{(\log n)^{O(1)}}\right)$ (approximation factor remains the same). 3. (3)

For any $p,q$ such that $\infty\geq p\geq 2\geq q\geq 1$ and $\epsilon>0$: it is NP-hard to approximate the $p\to q$ within a factor $1/(\gamma_{p^{*}}\gamma_{q})-\epsilon$.

4.1.2 The Search For Optimal Constants and Optimal Algorithms

The goal of determining the right approximation ratio for these problems is closely related to the question of finding the optimal (rounding) algorithms. For the Grothendieck problem, the goal is to find $y\in{\mathbb{R}}^{m}$ and $x\in{\mathbb{R}}^{n}$ with $\mathchoice{\lVert y\rVert_{\infty}}{\lVert y\rVert_{\infty}}{\lVert y\rVert_{\infty}}{\lVert y\rVert_{\infty}},\mathchoice{\lVert x\rVert_{\infty}}{\lVert x\rVert_{\infty}}{\lVert x\rVert_{\infty}}{\lVert x\rVert_{\infty}}\leq 1$, and one considers the following semidefinite relaxation:

[TABLE]

By the bilinear nature of the problem above, it is clear that the optimal $x,y$ can be taken to have entries in $\left\{-1,1\right\}$. A bound on the approximation ratio111Since we will be dealing with problems where the optimal solution may not be integral, we will use the term “approximation ratio” instead of “integrality gap”. of the above program is then obtained by designing a good “rounding” algorithm which maps the vectors $u^{i},v^{j}$ to values in $\left\{-1,1\right\}$. Krivine’s analysis [Kri77] corresponds to a rounding algorithm which considers a random vector ${\bf g}\sim{\mathcal{N}\left(0,I_{m+n}\right)}$ and rounds to $x,y$ defined as

[TABLE]

for some appropriately chosen transformations $\phi$ and $\psi$. This gives the following upper bound on the approximation ratio of the above relaxation, and hence on the value of the Grothendieck constant $K_{G}$:

[TABLE]

Braverman et al. [BMMN13] show that the above bound can be strictly improved (by a very small amount) using a two-dimensional analogue of the above algorithm, where the value $y_{i}$ is taken to be a function of the two-dimensional projection $(\langle\phi(u^{i}),{\bf g}_{1}\rangle,\langle\phi(u^{i}),{\bf g}_{2}\rangle)$ for independent Gaussian vectors ${\bf g}_{1},{\bf g}_{2}\in{\mathbb{R}}^{m+n}$ (and similarly for $x$). Naor and Regev [NR14] show that such schemes are optimal in the sense that it is possible to achieve an approximation ratio arbitrarily close to the true (but unknown) value of $K_{G}$ by using $k$-dimensional projections for a large (constant) $k$. A similar existential result was also proved by Raghavendra and Steurer [RS09b] who proved that the there exists a (slightly different) rounding algorithm which can achieve the (unknown) approximation ratio $K_{G}$.

For the case of arbitrary $p\geq 2\geq q$, Nesterov [Nes98] considered the convex program in Fig.˜4.3, denoted as $\mathsf{CP}(A)$, generalizing the one above. We reproduce it in Section˜4.2.3. Although the stated bound in [Nes98] is slightly weaker (as it is proved for a larger class of problems), the approximation ratio of the above relaxation can be shown to be bounded by $K_{G}$. By using the Krivine rounding scheme of considering the sign of a random Gaussian projection (aka random hyperplane rounding) one can show that Krivine’s upper bound on $K_{G}$ still applies to the above problem.

Motivated by applications to robust optimization, Steinberg [Ste05] considered the dual of (a variant of) the above relaxation, and obtained an upper bound of $\min\left\{\gamma_{p}/\gamma_{q},\gamma_{q^{*}}/\gamma_{p^{*}}\right\}$ on the approximation factor. Note that while Steinberg’s bound is better (approaches 1) as $p$ and $q$ approach 2, it is unbounded when $p,q^{*}\to\infty$ (as in the Grothendieck problem).

Based on the inapproximability result mentioned above, it is natural to ask if this is the “right form” of the approximation ratio. Indeed, this ratio is $\pi/2$ when $p^{*}=q=1$, which is the ratio obtained by Krivine’s rounding scheme, up to a factor of $\ln(1+\sqrt{2})$. We extend Krivine’s result to all $p\geq 2\geq q$ as below for the relaxation $\mathsf{CP}(A)$ (see Fig.˜4.3):

Theorem 4.1.5**.**

There exists a fixed constant $\varepsilon_{0}\leq 0.00863$ such that for all $p\geq 2\geq q$, the approximation ratio of the convex relaxation $\mathsf{CP}(A)$ is upper bounded by

[TABLE]

Perhaps more interestingly, the above theorem is proved via a generalization of hyperplane rounding, which we believe may be of independent interest. Indeed, for a given collection of vectors $w^{1},\ldots,w^{m}$ considered as rows of a matrix $W$, Gaussian hyperplane rounding corresponds to taking the “rounded” solution $y$ to be the

[TABLE]

We consider the natural generalization to (say) $\ell_{r}$ norms, given by

[TABLE]

We refer to $y$ as the “Hölder dual” of $W{\bf g}$, since the above rounding can be obtained by viewing $W{\bf g}$ as lying in the dual ($\ell_{r^{*}}$) ball, and finding the $y$ for which Hölder’s inequality is tight. Indeed, in the above language, Nesterov’s rounding corresponds to considering the $\ell_{\infty}$ ball (hyperplane rounding). While Steinberg used a somewhat different relaxation, the rounding there can be obtained by viewing $W{\bf g}$ as lying in the primal $(\ell_{r})$ ball instead of the dual one. In case of hyperplane rounding, the analysis is motivated by the identity that for two unit vectors $u$ and $v$, we have

[TABLE]

We prove the appropriate extension of this identity to $\ell_{r}$ balls (and analyze the functions arising there) which may also be of interest for other optimization problems over $\ell_{r}$ balls.

4.1.3 Relation to Factorization Theory

Let $X,Y$ be Banach spaces, and let $A:X\to Y$ be a continuous linear operator. As before, the norm $\mathchoice{\lVert A\rVert_{X\to Y}}{\lVert A\rVert_{X\to Y}}{\lVert A\rVert_{X\to Y}}{\lVert A\rVert_{X\to Y}}$ is defined as

[TABLE]

The operator $A$ is said to be factorized through Hilbert space if the factorization constant of $A$ defined as

[TABLE]

is bounded, where the infimum is taken over all Hilbert spaces $H$ and all operators $B:H\to Y$ and $C:X\to H$. The factorization gap for spaces $X$ and $Y$ is then defined as $\Phi(X,Y):=\sup_{A}\Phi(A)$ where the supremum runs over all continuous operators $A:X\to Y$.

The theory of factorization of linear operators is a cornerstone of modern functional analysis and has also found many applications outside the field (see [Pis86, AK06a] for more information). An application to theoretical computer science was found by Tropp [Tro09] who used the Grothendieck factorization [Gro56] to give an algorithmic version of a celebrated column subset selection result of Bourgain and Tzafriri [BT87].

As an almost immediate consequence of convex programming duality, our new algorithmic results also imply some improved factorization results for $\ell_{p}^{n},\ell_{q}^{m}$ (a similar observation was already made by Tropp [Tro09] in the special case of $\ell_{\infty}^{n},\ell_{1}^{m}$ and for a slightly different relaxation). We first state some classical factorization results, for which we will use $T_{2}(X)$ and $C_{2}(X)$ to respectively denote the Type-2 and Cotype-2 constants of $X$. We refer the interested reader to Section˜4.8 for a more detailed description of factorization theory as well as the relevant functional analysis preliminaries.

The Kwapień-Maurey [Kwa72a, Mau74] theorem states that for any pair of Banach spaces $X$ and $Y$

[TABLE]

However, Grothendieck’s result [Gro56] shows that a much better bound is possible in a case where $T_{2}(X)$ is unbounded. In particular,

[TABLE]

for all $m,n\in{\mathbb{N}}$. Pisier [Pis80] showed that if $X$ or $Y$ satisfies the approximation property (which is always satisfied by finite-dimensional spaces), then

[TABLE]

We show that the approximation ratio of Nesterov’s relaxation is in fact an upper bound on the factorization gap for the spaces $\ell_{p}^{n}$ and $\ell_{q}^{m}$. Combined with our upper bound on the integrality gap, we show an improved bound on the factorization constant, i.e., for any $p\geq 2\geq q$ and $m,n\in{\mathbb{N}}$, we have that for $X=\ell_{p}^{n}$, $Y=\ell_{q}^{m}$

[TABLE]

where $\varepsilon_{0}\leq 0.00863$ as before. This improves on Pisier’s bound for all $p\geq 2\geq q$, and for certain ranges of $(p,q)$ it also improves upon $K_{G}$ and the bound of Kwapień-Maurey.

4.1.4 Approximability and Factorizability

Let $(X_{n})$ and $(Y_{m})$ be sequences of Banach spaces such that $X_{n}$ is over the vector space ${\mathbb{R}}^{n}$ and $Y_{m}$ is over the vector space ${\mathbb{R}}^{m}$. We shall say a pair of sequences $((X_{n}),(Y_{m}))$ factorize if $\Phi(X_{n},Y_{m})$ is bounded by a constant independent of $m$ and $n$. Similarly, we shall say a pair of families $((X_{n}),(Y_{m}))$ are computationally approximable if there exists a polynomial $R(m,n)$, such that for every $m,n\in{\mathbb{N}}$, there is an algorithm with runtime $R(m,n)$ approximating $\mathchoice{\lVert A\rVert_{X_{n}\to Y_{m}}}{\lVert A\rVert_{X_{n}\to Y_{m}}}{\lVert A\rVert_{X_{n}\to Y_{m}}}{\lVert A\rVert_{X_{n}\to Y_{m}}}$ within a constant independent of $m$ and $n$ (given an oracle for computing the norms of vectors and a separation oracle for the unit balls of the norms). We consider the natural question of characterizing the families of norms that are approximable and their connection to factorizability and Cotype.

The pairs $(p,q)$ for which $(\ell_{p}^{n},\ell_{q}^{m})$ is known (resp. not known) to factorize, are precisely those pairs $(p,q)$ which are known to be computationally approximable (resp. inapproximable assuming hardness conjectures like $\textrm{P}\neq\textsf{NP}$ and ETH). Moreover, the Hilbertian case which trivially satisfies factorizability, is also known to be computationally approximable (with approximation factor 1).

It is tempting to ask whether the set of computationally approximable pairs coincides with the set of factorizable pairs or the pairs for which $X^{*}_{n},Y_{m}$ have bounded (independent of $m,n$) Cotype-2 constant. Further yet, is there a connection between the approximation factor and the factorization constant, or approximation factor and Cotype-2 constants (of $X_{n}^{*}$ and $Y_{m}$)? Our work gives some modest additional evidence towards such conjectures. Such a result would give credibility to the appealing intuitive idea of the approximation factor being dependent on the “distance” to a Hilbert space.

4.1.5 Proof Overview: Hardness

The hardness of proving hardness for hypercontractive norms.

Reductions for various geometric problems use a “smooth” version of the Label Cover problem, composed with long-code functions for the labels of the variables. In various reductions, including the ones by Guruswami et al. [GRSW16] and Briët et al. [BRS15] (which we closely follow) the solution vector $x$ to the geometric problem consists of the Fourier coefficients of the various long-code functions, with a “block” $x_{v}$ for each vertex of the label-cover instance. The relevant geometric operation (transformation by the matrix $A$ in our case) consists of projecting to a space which enforces the consistency constraints derived from the label-cover problem, on the Fourier coefficients of the encodings.

However, this strategy presents with two problems when designing reductions for hypercontractive norms. Firstly, while projections maintain the $\ell_{2}$ norm of encodings corresponding to consistent labelings and reduce that of inconsistent ones, their behaviour is harder to analyze for $\ell_{p}$ norms for $p\neq 2$. Secondly, the global objective of maximizing $\mathchoice{\lVert Ax\rVert_{q}}{\lVert Ax\rVert_{q}}{\lVert Ax\rVert_{q}}{\lVert Ax\rVert_{q}}$ is required to enforce different behavior within the blocks $x_{v}$, than in the full vector $x$. The block vectors $x_{v}$ in the solution corresponding to a satisfying assignment of label cover are intended to be highly sparse, since they correspond to “dictator functions” which have only one non-zero Fourier coefficient. This can be enforced in a test using the fact that for a vector $x_{v}\in{\mathbb{R}}^{t}$, $\mathchoice{\lVert x_{v}\rVert_{q}}{\lVert x_{v}\rVert_{q}}{\lVert x_{v}\rVert_{q}}{\lVert x_{v}\rVert_{q}}$ is a convex function of $\mathchoice{\lVert x_{v}\rVert_{p}}{\lVert x_{v}\rVert_{p}}{\lVert x_{v}\rVert_{p}}{\lVert x_{v}\rVert_{p}}$ when $p\leq q$, and is maximized for vectors with all the mass concentrated in a single coordinate. However, a global objective function which tries to maximize $\sum_{v}\mathchoice{\lVert x_{v}\rVert}{\lVert x_{v}\rVert}{\lVert x_{v}\rVert}{\lVert x_{v}\rVert}_{q}^{q}$, also achieves a high value from global vectors $x$ which concentrate all the mass on coordinates corresponding to few vertices of the label cover instance, and do not carry any meaningful information about assignments to the underlying label cover problem.

Since we can only check for a global objective which is the $\ell_{q}$ norm of some vector involving coordinates from blocks across the entire instance, it is not clear how to enforce local Fourier concentration (dictator functions for individual long codes) and global well-distribution (meaningful information regarding assignments of most vertices) using the same objective function. While the projector $A$ also enforces a linear relation between the block vectors $x_{u}$ and $x_{v}$ for all edges $(u,v)$ in the label cover instance, using this to ensure well-distribution across blocks seems to require a very high density of constraints in the label cover instance, and no hardness results are available in this regime.

Our reduction.

We show that when $2\notin[p,q]$, it is possible to bypass the above issues using hardness of $\mathchoice{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}$ as an intermediate (for $r<2$). Note that since $\mathchoice{\lVert z\rVert_{r}}{\lVert z\rVert_{r}}{\lVert z\rVert_{r}}{\lVert z\rVert_{r}}$ is a concave function of $\mathchoice{\lVert z\rVert_{2}}{\lVert z\rVert_{2}}{\lVert z\rVert_{2}}{\lVert z\rVert_{2}}$ in this case, the test favors vectors in which the mass is well-distributed and thus solves the second issue. For this, we use local tests based on the Berry-Esséen theorem (as in [GRSW16] and [BRS15]). Also, since the starting point now is the $\ell_{2}$ norm, the effect of projections is easier to analyze. This reduction is discussed in Section˜4.3.

By duality, we can interpret the above as a hardness result for $\mathchoice{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}$ when $p>2$ (using $r=p^{*}$). We then convert this to a hardness result for $p\to q$ in the hypercontractive case by composing $A$ with an “approximate isometry” $B$ from $\ell_{2}\to\ell_{q}$ (i.e., $\forall y~\mathchoice{\lVert By\rVert_{q}}{\lVert By\rVert_{q}}{\lVert By\rVert_{q}}{\lVert By\rVert_{q}}\approx\mathchoice{\lVert y\rVert_{2}}{\lVert y\rVert_{2}}{\lVert y\rVert_{2}}{\lVert y\rVert_{2}}$) since we can replace $\mathchoice{\lVert Ax\rVert_{2}}{\lVert Ax\rVert_{2}}{\lVert Ax\rVert_{2}}{\lVert Ax\rVert_{2}}$ with $\mathchoice{\lVert BAx\rVert_{q}}{\lVert BAx\rVert_{q}}{\lVert BAx\rVert_{q}}{\lVert BAx\rVert_{q}}$. Milman’s version of the Dvoretzky theorem [Ver17] implies random operators to a sufficiently high dimensional ($n^{O(q)}$) space satisfy this property, which then yields constant factor hardness results for the $p\to q$. A similar application of Dvoretzky’s theorem also appears in an independent work of Krishnan et al. [KMW18] on sketching matrix norms.

We also show that the hardness for hypercontractive norms can be amplified via tensoring. This was known previously for the $2\to 4$ using an argument based on parallel repetition for QMA [HM13], and for the case of $p=q$ [BV11]. We give a simple argument based on convexity, which proves this for all $p\leq q$, but appears to have gone unnoticed previously. The amplification is then used to prove hardness of approximation within almost polynomial factors.

Non-hypercontractive norms.

We also use the hardness of $\mathchoice{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}{\lVert A\rVert_{2\to r}}$ to obtain hardness for the non-hypercontractive case of $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}$ with $q<2<p$, by using an operator that “factorizes” through $\ell_{2}$. In particular, we obtain hardness results for $\mathchoice{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}$ and $\mathchoice{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}{\lVert A\rVert_{2\to q}}$ (of factors $1/\gamma_{p^{*}}$and $1/\gamma_{q}$ respectively) using the reduction in Section˜4.3. We then combine these hardness results using additional properties of the operator $A$ obtained in the reduction, to obtain a hardness of factor $(1/\gamma_{p^{*}})\cdot(1/\gamma_{q})$ for the $p\to q$ for $p>2>q$. The composition, as well as the hardness results for hypercontractive norms, are presented in Section˜4.4.

We also obtain a simple proof of the $2^{\Omega((\log n)^{1-\varepsilon})}$ hardness for the non-hypercontractive case when $2\notin[q,p]$ (already proved by Bhaskara and Vijayaraghavan [BV11]) via an approximate isometry argument as used in the hypercontractive case. In the hypercontractive case, we started from a constant factor hardness of the $p\to 2$ and the same factor for $p\to q$ using the fact that for a random Gaussian matrix $B$ of appropriate dimensions, we have $\mathchoice{\lVert Bx\rVert_{q}}{\lVert Bx\rVert_{q}}{\lVert Bx\rVert_{q}}{\lVert Bx\rVert_{q}}\approx\mathchoice{\lVert x\rVert_{2}}{\lVert x\rVert_{2}}{\lVert x\rVert_{2}}{\lVert x\rVert_{2}}$ for all $x$. We then amplify the hardness via tensoring. In the non-hypercontractive case, we start with a hardness for $p\to p$ (obtained via the above isometry), which we first amplify via tensoring. We then apply another approximate isometry result due to Schechtman [Sch87], which gives a samplable distribution $\mathcal{D}$ over random matrices $B$ such that with high probability over $B$, we have $\mathchoice{\lVert Bx\rVert_{q}}{\lVert Bx\rVert_{q}}{\lVert Bx\rVert_{q}}{\lVert Bx\rVert_{q}}\approx\mathchoice{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}$ for all $x$.

We thus view the above results as showing that combined with a basic hardness for $p\to 2$, the basic ideas of duality, tensoring, and embedding (which builds on powerful results from functional analysis) can be combined in powerful ways to prove strong results in both the hypercontractive and non-hypercontractive regimes.

4.1.6 Proof Overview: Algorithm

As discussed above, we consider Nesterov’s convex relaxation and generalize the hyperplane rounding scheme using “Hölder duals” of the Gaussian projections, instead of taking the sign. As in the Krivine rounding scheme, this rounding is applied to transformations of the SDP solutions. The nature of these transformations depends on how the rounding procedure changes the correlation between two vectors. Let $u,v\in{\mathbb{R}}^{N}$ be two unit vectors with $\left\langle u,v\right\rangle=\rho$. Then, for ${\bf g}\sim{\mathcal{N}\left(0,I_{N}\right)}$, $\left\langle{\bf g},u\right\rangle$ and $\left\langle{\bf g},v\right\rangle$ are $\rho$-correlated Gaussian random variables. Hyperplane rounding then gives $\pm 1$ valued random variables whose correlation is given by

[TABLE]

The transformations $\phi$ and $\psi$ (to be applied to the vectors $u$ and $v$) in Krivine’s scheme are then chosen depending on the Taylor series for the $\sin$ function, which is the inverse of function computed on the correlation. For the case of Hölder-dual rounding, we prove the following generalization of the above identity

[TABLE]

where ${}_{2}F_{1}$ denotes a hypergeometric function with the specified parameters. The proof of the above identity combines simple tools from Hermite analysis with known integral representations from the theory of special functions, and may be useful in other applications of the rounding procedure.

Note that in the Grothendieck case, we have $\gamma_{p^{*}}^{p^{*}}=\gamma_{q}^{q}=\sqrt{2/\pi}$, and the remaining part is simply the $\sin^{-1}$ function. In the Krivine rounding scheme, the transformations $\phi$ and $\psi$ are chosen to satisfy $(2/\pi)\cdot\sin^{-1}\left(\left\langle\phi(u),\psi(v)\right\rangle\right)=c\cdot\left\langle u,v\right\rangle$, where the constant $c$ then governs the approximation ratio. The transformations $\phi(u)$ and $\psi(v)$ taken to be of the form $\phi(u)=\oplus_{i=1}^{\infty}a_{i}\cdot u^{\otimes i}$ such that

[TABLE]

If $f$ represents (a normalized version of) the function of $\rho$ occurring in the identity above (which is $\sin^{-1}$ for hyperplane rounding), then the approximation ratio is governed by the function $h$ obtained by replacing every Taylor coefficient of $f^{-1}$ by its absolute value. While $f^{-1}$ is simply the $\sin$ function (and thus $h$ is the $\sinh$ function) in the Grothendieck problem, no closed-form expressions are available for general $p$ and $q$.

The task of understanding the approximation ratio thus reduces to the analytic task of understanding the family of the functions $h$ obtained for different values of $p$ and $q$. Concretely, the approximation ratio is given by the value $1/(h^{-1}(1)\cdot\gamma_{q}\,\gamma_{p^{*}})$. At a high level, we prove bounds on $h^{-1}(1)$ by establishing properties of the Taylor coefficients of the family of functions $f^{-1}$, i.e., the family given by

[TABLE]

While in the cases considered earlier, the functions $h$ are easy to determine in terms of $f^{-1}$ via succinct formulae [Kri77, Haa81, AN04] or can be truncated after the cubic term [NR14], neither of these are true for the family of functions we consider. Hypergeometric functions are a rich and expressive class of functions, capturing many of the special functions appearing in Mathematical Physics and various ensembles of orthogonal polynomials. Due to this expressive power, the set of inverses is not well understood. In particular, while the coefficients of $f$ are monotone in $p$ and $q$, this is not true for $f^{-1}$. Moreover, the rates of decay of the coefficients may range from inverse polynomial to super-exponential. We analyze the coefficients of $f^{-1}$ using complex-analytic methods inspired by (but quite different from) the work of Haagerup [Haa81] on bounding the complex Grothendieck constant. The key technical challenge in our work is in arguing systematically about a family of inverse hypergeometric functions which we address by developing methods to estimate the values of a family of contour integrals.

While our methods only gives a bound of the form $h^{-1}(1)\geq\sinh^{-1}(1)/(1+\varepsilon_{0})$, we believe this is an artifact of the analysis and the true bound should indeed be $h^{-1}(1)\geq\sinh^{-1}(1)$.

4.2 Preliminaries and Notation

4.2.1 Matrix Norms

For a vector $x\in{\mathbb{R}}^{n}$, throughout this chapter we will use $x(i)$ to denote its $i$-th coordinate. For $p\in[1,\infty)$, we define $\mathchoice{\lVert\cdot\rVert_{\ell_{p}}}{\lVert\cdot\rVert_{\ell_{p}}}{\lVert\cdot\rVert_{\ell_{p}}}{\lVert\cdot\rVert_{\ell_{p}}}$ to denote the counting $p$-norm and $\mathchoice{\lVert\cdot\rVert_{L_{p}}}{\lVert\cdot\rVert_{L_{p}}}{\lVert\cdot\rVert_{L_{p}}}{\lVert\cdot\rVert_{L_{p}}}$ to denote the expectation $p$-norm; i.e., for a vector $x\in{\mathbb{R}}^{n}$,

[TABLE]

Clearly $\mathchoice{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}=\mathchoice{\lVert x\rVert_{L_{p}}}{\lVert x\rVert_{L_{p}}}{\lVert x\rVert_{L_{p}}}{\lVert x\rVert_{L_{p}}}\cdot n^{1/p}$. For $p=\infty$, we define $\mathchoice{\lVert x\rVert_{\ell_{\infty}}}{\lVert x\rVert_{\ell_{\infty}}}{\lVert x\rVert_{\ell_{\infty}}}{\lVert x\rVert_{\ell_{\infty}}}=\mathchoice{\lVert x\rVert_{L_{\infty}}}{\lVert x\rVert_{L_{\infty}}}{\lVert x\rVert_{L_{\infty}}}{\lVert x\rVert_{L_{\infty}}}:=\max_{i\in[n]}|x(i)|$. We will use $p^{*}$ to denote the ‘dual’ of $p$, i.e. $p^{*}=p/(p-1)$. Unless stated otherwise, we usually work with $\mathchoice{\lVert\cdot\rVert_{\ell_{p}}}{\lVert\cdot\rVert_{\ell_{p}}}{\lVert\cdot\rVert_{\ell_{p}}}{\lVert\cdot\rVert_{\ell_{p}}}$. We also define inner product $\left\langle x,y\right\rangle$ to denote the inner product under the counting measure unless stated otherwise; i.e., for two vectors $x,y\in{\mathbb{R}}^{n}$, $\left\langle x,y\right\rangle:=\sum_{i\in[n]}x(i)y(i)$.

Next, we record a well-known fact about $p$-norms that is used in establishing many duality statements.

Observation 4.2.1**.**

For any $p\in[1,\infty]$, $\mathchoice{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}=\sup_{\mathchoice{\lVert y\rVert_{\ell_{p^{*}}}}{\lVert y\rVert_{\ell_{p^{*}}}}{\lVert y\rVert_{\ell_{p^{*}}}}{\lVert y\rVert_{\ell_{p^{*}}}}=1}\,\left\langle y,x\right\rangle$.

We now formally state two related quantities studied in this chapter:

Definition 4.2.2**.**

For $p,q\in[1,\infty]$, the $p\to q$ problem is defined as follows:

[TABLE]

for an $m\times n$ matrix $A$.

Combining ˜4.2.1 and ˜4.2.2, we see that the original Grothendieck problem is precisely $(\infty,\infty)$-Grothendieck. And a natural generalization then is as follows:

Definition 4.2.3**.**

For $p,q\in[1,\infty]$, we define a generalization of the Grothendieck problem, namely $(p,q)$-Grothendieck, as the problem of computing

[TABLE]

given an $m\times n$ matrix $A$.

4.2.2 Preliminaries for Hardness Results

The symmetry of the hardness results in ˜4.1.4 is explained by the following observation:

Observation 4.2.4**.**

For any $p,q\in[1,\infty]$ and any matrix $A$,

[TABLE]

• **Proof:  ** Using $\left\langle y,Ax\right\rangle=\left\langle x,A^{T}y\right\rangle$,

[TABLE]

The following observation will be useful for composing hardness maps for $p\to 2$ and $2\to q$ to get $p\to q$ hardness for when $p>q$ and $p\geq 2\geq q$.

Observation 4.2.5**.**

For any $p,q,r\in[1,\infty]$ and any matrices $B,C$,

[TABLE]

Fourier Analysis

We introduce some basic facts about Fourier analysis of Boolean functions used in our proof of hardness results. Let $R\in{\mathbb{N}}$ be a positive integer, and consider a function $f:\{\pm 1\}^{R}\to{\mathbb{R}}$. For any subset $S\subseteq[R]$ let $\chi_{S}:=\prod_{i\in S}x_{i}$. Then we can represent $f$ as

[TABLE]

where

[TABLE]

The Fourier transform refers to a linear operator $F$ that maps $f$ to $\widehat{f}$ as defined as (4.2). We interpret $\widehat{f}$ as a $2^{R}$-dimensional vector whose coordinates are indexed by $S\subseteq[R]$. Endow the expectation norm and the expectation norm to $f$ and $\widehat{f}$ respectively; i.e.,

[TABLE]

as well as the corresponding inner products $\left\langle f,g\right\rangle$ and $\left\langle\widehat{f},\widehat{g}\right\rangle$ consistent with their $2$-norms. We also define the inverse Fourier transform $F^{T}$ to be a linear operator that maps a given $\widehat{f}:2^{R}\to{\mathbb{R}}$ to $f:\{\pm 1\}^{R}\to{\mathbb{R}}$ defined as in (4.1). We state the following well-known facts from Fourier analysis.

Observation 4.2.6**.**

$F$* is an orthogonal transformation; i.e., for any $f,g:\{\pm 1\}^{R}\to{\mathbb{R}}$,*

[TABLE]

Observation 4.2.7** (Parseval’s Theorem).**

For any $f:\{\pm 1\}^{R}\to{\mathbb{R}}$, $\mathchoice{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}=\mathchoice{\lVert Ff\rVert_{\ell_{2}}}{\lVert Ff\rVert_{\ell_{2}}}{\lVert Ff\rVert_{\ell_{2}}}{\lVert Ff\rVert_{\ell_{2}}}$.

Observation 4.2.8**.**

$F$* and $F^{T}$ form an adjoint pair; i.e., for any $f:\{\pm 1\}^{R}\to{\mathbb{R}}$ and $\widehat{g}:2^{R}\to{\mathbb{R}}$,*

[TABLE]

Observation 4.2.9**.**

$F^{T}F$* is the identity operator.*

In Section˜4.3, we also consider a partial Fourier transform $F_{P}$ that maps a given function $f:\{\pm 1\}^{R}\to{\mathbb{R}}$ to a vector $\widehat{f}:[R]\to{\mathbb{R}}$ defined as $\widehat{f}(i)=\mathchoice{\underset{x\in\{\pm 1\}^{R}}{\mathbb{E}}\left[f(x)\cdot x_{i}\right]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[f(x)\cdot x_{i}]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[f(x)\cdot x_{i}]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[f(x)\cdot x_{i}]}$ for all $i\in[R]$. It is the original Fourier transform where $\widehat{f}$ is further projected to $R$ coordinates corresponding to linear coefficients. The partial inverse Fourier transform $F_{P}^{T}$ is a transformation that maps a vector $\widehat{f}:[R]\to{\mathbb{R}}$ to a function $f:\{\pm 1\}^{R}\to{\mathbb{R}}$ as in (4.1) restricted to $S=\{i\}$ for some $i\in[R]$. These partial transforms satisfy similar observations as above: (1) $\mathchoice{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}\geq\mathchoice{\lVert F_{P}f\rVert_{\ell_{2}}}{\lVert F_{P}f\rVert_{\ell_{2}}}{\lVert F_{P}f\rVert_{\ell_{2}}}{\lVert F_{P}f\rVert_{\ell_{2}}}$, (2) $\mathchoice{\lVert F_{P}^{T}\widehat{f}\rVert_{L_{2}}}{\lVert F_{P}^{T}\widehat{f}\rVert_{L_{2}}}{\lVert F_{P}^{T}\widehat{f}\rVert_{L_{2}}}{\lVert F_{P}^{T}\widehat{f}\rVert_{L_{2}}}=\mathchoice{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}$, (3) $F_{P}$ and $F_{P}^{T}$ form an adjoint pair, and (4) $(F_{P}^{T}F_{P})f=f$ if and only if $f$ is a linear function.

We now state and prove a useful lemma involving Fourier coefficients of linear functions that is very useful for our proof of hardness.

An Useful Lemma for Dictatorship Test

Lemma 4.2.10**.**

Let $f:\{\pm 1\}^{R}\to{\mathbb{R}}$ be a linear function for some positive integer $R\in{\mathbb{N}}$ and $\widehat{f}:[R]\to{\mathbb{R}}$ be its linear Fourier coefficients defined by

[TABLE]

For all $\varepsilon>0$, there exists $\delta>0$ such that if $\mathchoice{\lVert f\rVert_{L_{r}}}{\lVert f\rVert_{L_{r}}}{\lVert f\rVert_{L_{r}}}{\lVert f\rVert_{L_{r}}}>(\gamma_{r}+\varepsilon)\mathchoice{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}$ then $\mathchoice{\lVert\widehat{f}\rVert_{\ell_{4}}}{\lVert\widehat{f}\rVert_{\ell_{4}}}{\lVert\widehat{f}\rVert_{\ell_{4}}}{\lVert\widehat{f}\rVert_{\ell_{4}}}>\delta\mathchoice{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}$ for all $1\leq r<2$.

Before we can prove the above, we state and prove an implication of Berry-Esséen estimate for fractional moments (similar to Lemma 3.3 of [GRSW16], see also [KNS10]).

Lemma 4.2.11**.**

There exist universal constants $c>0$ and $\delta_{0}>0$ such that the following statement is true. If $X_{1},\ldots,X_{n}$ are bounded independent random variables with $\left\lvert X_{i}\right\rvert\leq 1$, $\mathchoice{{\mathbb{E}}\left[X_{i}\right]}{{\mathbb{E}}[X_{i}]}{{\mathbb{E}}[X_{i}]}{{\mathbb{E}}[X_{i}]}=0$ for $i\in[n]$, and $\sum_{i\in[n]}\mathchoice{{\mathbb{E}}\left[X_{i}^{2}\right]}{{\mathbb{E}}[X_{i}^{2}]}{{\mathbb{E}}[X_{i}^{2}]}{{\mathbb{E}}[X_{i}^{2}]}=1$, $\sum_{i\in[n]}\mathchoice{{\mathbb{E}}\left[\left\lvert X_{i}\right\rvert^{3}\right]}{{\mathbb{E}}[\left\lvert X_{i}\right\rvert^{3}]}{{\mathbb{E}}[\left\lvert X_{i}\right\rvert^{3}]}{{\mathbb{E}}[\left\lvert X_{i}\right\rvert^{3}]}\leq\delta$ for some $0<\delta<\delta_{0}$, then for every $p\geq 1$:

[TABLE]

• **Proof:  ** The proof is almost similar to that of Lemma 2.1 of [KNS10]. From Berry-Esséen theorem (see [vB72] for the constant), we get that:

[TABLE]

for every $u>0$ and where $g\sim{\mathcal{N}\left(0,1\right)}$. By Hoeffding’s lemma,

[TABLE]

for every $t>0$. Combining the above observations, we get:

[TABLE]

where $\Gamma(\cdot,\cdot)$ is the upper incomplete gamma function and $a$ is a large constant determined later depending on $\delta$ and $p$. The second term is bounded as

[TABLE]

Hence $\int_{a}^{\infty}u^{p}\mathrm{e}^{-\nicefrac{{u^{2}}}{{2}}}du\leq\frac{a^{p+1}e^{-\nicefrac{{a^{2}}}{{2}}}}{1+a^{2}-p}$.

We know, $\Gamma(\nicefrac{{p+1}}{{2}},x)\to x^{\frac{p-1}{2}}\mathrm{e}^{-x}$ as $x\to\infty$. We choose $a=\gamma_{p}\sqrt{\log\frac{1}{\delta}}$. Hence, there exists $\delta_{0}$ so that for all small enough $\delta<\delta_{0}$, we have $\Gamma(\nicefrac{{p+1}}{{2}},2a^{2})\sim 2^{\frac{p-1}{2}}a^{p-1}\delta^{2\gamma_{p}^{2}}\ll\delta a^{p}$ where the last inequality follows from the fact that $2\gamma_{p}^{2}>1$ (as $p>1$). Putting all this together, we get:

[TABLE]

where $c$ is an absolute constant independent of $a$ and $p$. This completes the proof of the lemma.

Finally, we prove Lemma˜4.2.10:

• **Proof Lemma˜4.2.10:  ** We will prove this lemma by the method of contradiction. Let us assume $\mathchoice{\lVert\widehat{f}\rVert_{\ell_{4}}}{\lVert\widehat{f}\rVert_{\ell_{4}}}{\lVert\widehat{f}\rVert_{\ell_{4}}}{\lVert\widehat{f}\rVert_{\ell_{4}}}\leq\delta\mathchoice{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}$, for $\delta$ to be fixed later.

Let us define $y_{i}\stackrel{{\scriptstyle\mathrm{def}}}{{=}}\frac{\widehat{f}(i)}{\mathchoice{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}}$. Then, for all $x\in\{-1,1\}^{R}$,

[TABLE]

Let $Z_{i}=x_{i}\cdot y_{i}$ be the random variable when $x_{i}$ is independently uniformly randomly chosen from $\{-1,1\}$. Now

[TABLE]

and

[TABLE]

where the penultimate inequality follows from Cauchy-Schwarz ineqality.

Hence, by applying Lemma˜4.2.11 on the random variables $Z_{1},\ldots,Z_{n}$, we get:

[TABLE]

We choose $\delta>0$ small enough (since $1\leq r<2$, setting $\delta<\frac{\sqrt{\varepsilon}}{\min(\delta_{0},\sqrt{\gamma_{2}}\log\frac{c\gamma_{2}}{\varepsilon})}=\frac{\sqrt{\varepsilon}}{\min(\delta_{0},\log\frac{c}{\varepsilon})}$ suffices) so that $\delta^{2}(\log\frac{1}{\delta})^{r}<\frac{\varepsilon}{c\gamma_{r}}$. For this choise of $\delta$, we get: $\mathchoice{\lVert f\rVert_{L_{r}}}{\lVert f\rVert_{L_{r}}}{\lVert f\rVert_{L_{r}}}{\lVert f\rVert_{L_{r}}}\leq(\gamma_{r}+\varepsilon)\mathchoice{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}$ – a contradiction. And hence the proof follows.

Starting Point of Hardness Reduction: Smooth Label Cover

An instance of Label Cover is given by a quadruple $\mathcal{L}=(G,[R],[L],\Sigma)$ that consists of a regular connected graph $G=(V,E)$, a label set $[R]$ for some positive integer $n$, and a collection $\Sigma=((\pi_{e,v},\pi_{e,w}):e=(v,w)\in E)$ of pairs of maps both from $[R]$ to $[L]$ associated with the endpoints of the edges in $E$. Given a labeling $\ell:V\to[R]$, we say that an edge $e=(v,w)\in E$ is *satisfied * if $\pi_{e,v}(\ell(v))=\pi_{e,w}(\ell(w))$. Let $\operatorname{\operatorname{OPT}}(\mathcal{L})$ be the maximum fraction of satisfied edges by any labeling.

The following hardness result for Label Cover, given in [GRSW16], is a slight variant of the original construction due to [Kho02a]. The theorem also describes the various structural properties, including smoothness, that are identified by the hard instances.

Theorem 4.2.12**.**

For any $\xi>0$ and $J\in{\mathbb{N}}$, there exist positive integers $R=R(\xi,J),L=L(\xi,J)$ and $D=D(\xi)$, and a Label Cover instance $(G,[R],[L],\Sigma)$ as above such that

(Hardness): It is NP-hard to distinguish between the following two cases:

• –

(Completeness): $\operatorname{\operatorname{OPT}}(\mathcal{L})=1$.

• –

(Soundness): $\operatorname{\operatorname{OPT}}(\mathcal{L})\leq\xi$.

• -

(Structural Properties):

• –

($J$-Smoothness): For every vertex $v\in V$ and distinct $i,j\in[R]$, we have

[TABLE]

• –

($D$-to-$1$): For every vertex $v\in V$, edge $e\in E$ incident on $v$, and $i\in[L]$, we have $|\pi^{-1}_{e,v}(i)|\leq D$; that is at most $D$ elements in $[R]$ are mapped to the same element in $[L]$.

• –

(Weak Expansion): For any $\delta>0$ and vertex set $V^{\prime}\subseteq V$ such that $|V^{\prime}|=\delta\cdot|V|$, the number of edges among the vertices in $|V^{\prime}|$ is at least $(\delta^{2}/2)|E|$.

4.2.3 Preliminaries for Algorithm

For $p\geq 2\geq q\geq 1$, we will use the following notation: $a\stackrel{{\scriptstyle\mathrm{def}}}{{=}}p^{*}-1$ and $b\stackrel{{\scriptstyle\mathrm{def}}}{{=}}q-1$. We note that $a,b\in[0,1]$.

For an $m\times n$ matrix $M$ (or vector, when $n=1$). For a unitary function $f$, we define $f[M]$ to be the matrix $M$ with entries defined as $(f[M])_{i,j}=f(M_{i,j})$ for $i\in[m],j\in[n]$. For vectors $u,v\in{\mathbb{R}}^{\ell}$, we denote by $u\circ v\in{\mathbb{R}}^{\ell}$ the entry-wise/Hadamard product of $u$ and $v$. We denote the concatenation of two vectors $u$ and $v$ by $u\oplus v$. For a vector $u$, we use $D_{u}$ to denote the diagonal matrix with the entries of $u$ forming the diagonal, and for a matrix $M$ we use $\operatorname{\operatorname{diag}}(M)$ to denote the vector of diagonal entries.

For a function $f(\tau)=\sum_{k\geq 0}f_{k}\cdot\tau^{k}$ defined as a power series, we denote the function $\mathrm{abs}\left(f\right)(\tau)\stackrel{{\scriptstyle\mathrm{def}}}{{=}}\sum_{k\geq 0}\left\lvert f_{k}\right\rvert\cdot\tau^{k}$.

Next, we recall the relaxation due to Nesterov [Nes98]:

Note that since $q^{*}\geq 2$ and $p\geq 2$, the above program is convex in the entries of the Gram matrix of the vectors $\left\{u^{i}\right\}_{i\in[m]}\cup\left\{v^{j}\right\}_{j\in[n]}$.

Next, we collect some basic ideas of Hermite Analysis that are relevant to analysis of our algorithm.

Hermite Analysis Preliminaries

Let $\gamma$ denote the standard Gaussian probability distribution. For this section (and only for this section), the (Gaussian) inner product for functions $f,h\in({\mathbb{R}},\gamma)\rightarrow{\mathbb{R}}$ is defined as

[TABLE]

Under this inner product there is a complete set of orthonormal polynomials $(H_{k})_{k\in{\mathbb{N}}}$ defined below.

Definition 4.2.13**.**

For a natural number $k$, then the $k$-th Hermite polynomial $H_{k}:{\mathbb{R}}\to{\mathbb{R}}$

[TABLE]

Any function $f$ satisfying $\int_{{\mathbb{R}}}|f(\tau)|^{2}\,d\gamma(\tau)<\infty$ has a Hermite expansion given by $f=\sum_{k\geq 0}\widehat{f}_{k}\cdot H_{k}$ where $\widehat{f}_{k}=\left\langle f,H_{k}\right\rangle\,.$

We have

Fact 4.2.14**.**

$H_{k}(\tau)$* is an even (resp. odd) function when $k$ is even (resp. odd).*

We also have the Plancherel Identity (as Hermite polynomials form an orthonormal basis):

Fact 4.2.15**.**

For two real valued functions $f$ and $h$ with Hermite coefficients $\widehat{f}_{k}$ and $\widehat{h}_{k}$, respectively, we have:

[TABLE]

The generating function of appropriately normalized Hermite polynomials satisfies the following identity:

[TABLE]

Similar to the noise operator in Fourier analysis, we define the corresponding noise operator $T_{\rho}\,{}$ for Hermite analysis:

Definition 4.2.16**.**

For $\rho\in[-1,1]$ and a real valued function $f$, we define the function $T_{\rho}\,{f}$ as:

[TABLE]

Again similar to the case of Fourier analysis, the Hermite coefficients admit the following identity:

Fact 4.2.17**.**

$\widehat{(T_{\rho}\,{f})}_{k}=\rho^{k}\cdot\widehat{f}_{k}\,.$**

Next, we collect some definitions and facts from the theory of special functions useful to our analysis of the algorithm:

Gamma, Hypergeometric and Parabolic Cylinder Function Preliminaries

For a natural number $k$ and a real number $\tau$, we denote the rising factorial as $(\tau)_{k}\stackrel{{\scriptstyle\mathrm{def}}}{{=}}\tau\cdot(\tau+1)\cdot\cdots(\tau+k-1)$. We now define the following fairly general classes of functions, and we later use them we obtain a Taylor series representation of $\widetilde{f}_{a,\,b}(\tau)$.

Definition 4.2.18**.**

The confluent hypergeometric function with parameters $\alpha,\beta$, and $\lambda$ as:

[TABLE]

The (Gaussian) hypergeometric function is defined as follows:

Definition 4.2.19**.**

The hypergeometric function with parameters $w,\alpha,\beta$ and $\lambda$ as:

[TABLE]

Next we define the $\Gamma$ function:

Definition 4.2.20**.**

For a real number $\tau$, we define:

[TABLE]

The $\Gamma$ function has the following property:

Fact 4.2.21** (Duplication Formula).**

[TABLE]

We also note the relationship between $\Gamma$ and $\gamma_{r}$:

Fact 4.2.22**.**

For $r\in[0,\infty)$,

[TABLE]

• **Proof:  **

[TABLE]

Next, we record some facts about parabolic cylinder functions:

Fact 4.2.23** (12.5.1 of [Loz03]).**

Let $U$ be the function defined as

[TABLE]

for all $\alpha$ such that $\Re(\alpha)>-\frac{1}{2}\,.$ The function $U(\alpha,\pm\lambda)$ is a parabolic cylinder function and is a standard solution to the differential equation: $\frac{d^{2}w}{d\lambda^{2}}-\left(\frac{\lambda^{2}}{4}+\alpha\right)w=0$.

Next we quote the confluent hypergeometric representation of the parabolic cylinder function $U$ defined above:

Fact 4.2.24** (12.4.1, 12.2.6, 12.2.7, 12.7.12, and 12.7.13 of [Loz03]).**

[TABLE]

Combining the previous two facts, we get the following:

Corollary 4.2.25**.**

For all real $\alpha>-\frac{1}{2}$, we have:

[TABLE]

4.3 Hardness of $2\to r$ with $r<2$

This section proves the following theorem that serves as a starting point of our hardness results. The theorem is stated for the expectation norm for consistency with the current literature, but the same statement holds for the counting norm, since if $A$ is an $n\times n$ matrix, $\mathchoice{\lVert A\rVert_{\ell_{2}\to\ell_{r}}}{\lVert A\rVert_{\ell_{2}\to\ell_{r}}}{\lVert A\rVert_{\ell_{2}\to\ell_{r}}}{\lVert A\rVert_{\ell_{2}\to\ell_{r}}}=n^{1/r-1/2}\cdot\mathchoice{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}$. Note that the matrix $A$ used in the reduction below does not depend on $r$.

Theorem 4.3.1**.**

For any $\varepsilon>0$, there is a polynomial time reduction that takes a 3-CNF formula $\phi$ and produces a symmetric matrix $A\in{\mathbb{R}}^{n\times n}$ with $n=|\phi|^{\operatorname{\operatorname{poly}}(1/\varepsilon)}$ such that

(Completeness) If $\phi$ is satisfiable, there exists $x\in{\mathbb{R}}^{n}$ with $|x(i)|=1$ for all $i\in[n]$ and $Ax=x$. In particular, $\mathchoice{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}\geq 1$ for all $1\leq r\leq\infty$.

• -

(Soundness) $\mathchoice{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}{\lVert A\rVert_{L_{2}\to L_{r}}}\leq\gamma_{r}+\varepsilon^{2-r}$ for all $1\leq r<2$.

We adapt the proof by Briët, Regev and Saket for the hardness of $2\to 1$ and $\infty\to 1$ norms to prove the above theorem. A small difference is that, unlike their construction which starts with a Fourier encoding of the long-code functions, we start with an evaluation table (to ensure that the resulting matrices are symmetric). We also analyze their dictatorship tests for the case of fractional $r$.

4.3.1 Reduction and Completeness

Let $\mathcal{L}=(G,[R],[L],\Sigma)$ be an instance of Label Cover with $G=(V,E)$. In the rest of this section, $n=|V|$ and our reduction will construct a self-adjoint linear operator ${\bf A}:{\mathbb{R}}^{N}\to{\mathbb{R}}^{N}$ with $N=|V|\cdot 2^{R}$, which yields a symmetric $N\times N$ matrix representing ${\bf A}$ in the standard basis. This section concerns the following four Hilbert spaces based on the standard Fourier analysis composed with $\mathcal{L}$.

Evaluation space ${\mathbb{R}}^{2^{R}}$. Each function in this space is denoted by $f:\{\pm 1\}^{R}\to{\mathbb{R}}$. The inner product is defined as $\left\langle f,g\right\rangle:=\mathchoice{\underset{x\in\{\pm 1\}^{R}}{\mathbb{E}}\left[f(x)g(x)\right]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[f(x)g(x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[f(x)g(x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[f(x)g(x)]}$, which induces $\mathchoice{\lVert f\rVert_{2}}{\lVert f\rVert_{2}}{\lVert f\rVert_{2}}{\lVert f\rVert_{2}}:=\mathchoice{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}{\lVert f\rVert_{L_{2}}}$. We also define $\mathchoice{\lVert f\rVert_{L_{p}}}{\lVert f\rVert_{L_{p}}}{\lVert f\rVert_{L_{p}}}{\lVert f\rVert_{L_{p}}}:=\mathchoice{\underset{x}{\mathbb{E}}\left[|f(x)|^{p}\right]}{{\mathbb{E}}_{x}[|f(x)|^{p}]}{{\mathbb{E}}_{x}[|f(x)|^{p}]}{{\mathbb{E}}_{x}[|f(x)|^{p}]}^{1/p}$ in this space. 2. 2.

Fourier space ${\mathbb{R}}^{R}$. Each function in this space is denoted by $\widehat{f}:[R]\to{\mathbb{R}}$. The inner product is defined as $\left\langle\widehat{f},\widehat{g}\right\rangle:=\sum_{i\in[R]}\widehat{f}(i)\widehat{g}(i)$, which induces $\mathchoice{\lVert\widehat{f}\rVert_{2}}{\lVert\widehat{f}\rVert_{2}}{\lVert\widehat{f}\rVert_{2}}{\lVert\widehat{f}\rVert_{2}}:=\mathchoice{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}{\lVert\widehat{f}\rVert_{\ell_{2}}}$. 3. 3.

Combined evaluation space ${\mathbb{R}}^{V\times 2^{R}}$. Each function in this space is denoted by ${\mathbf{f}}:V\times\{\pm 1\}^{R}\to{\mathbb{R}}$. The inner product is defined as $\left\langle{\mathbf{f}},{\mathbf{g}}\right\rangle:=\mathchoice{\underset{v\in V}{\mathbb{E}}\left[\mathchoice{\underset{x\in\{\pm 1\}^{R}}{\mathbb{E}}\left[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)\right]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}\right]}{{\mathbb{E}}_{v\in V}[\mathchoice{\underset{x\in\{\pm 1\}^{R}}{\mathbb{E}}\left[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)\right]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}]}{{\mathbb{E}}_{v\in V}[\mathchoice{\underset{x\in\{\pm 1\}^{R}}{\mathbb{E}}\left[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)\right]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}]}{{\mathbb{E}}_{v\in V}[\mathchoice{\underset{x\in\{\pm 1\}^{R}}{\mathbb{E}}\left[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)\right]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}{{\mathbb{E}}_{x\in\{\pm 1\}^{R}}[{\mathbf{f}}(v,x){\mathbf{g}}(v,x)]}]}$, which induces $\mathchoice{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}:=\mathchoice{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}$. We also define $\mathchoice{\lVert{\mathbf{f}}\rVert_{p}}{\lVert{\mathbf{f}}\rVert_{p}}{\lVert{\mathbf{f}}\rVert_{p}}{\lVert{\mathbf{f}}\rVert_{p}}:=\mathchoice{\underset{v,x}{\mathbb{E}}\left[|{\mathbf{f}}(v,x)|^{p}\right]}{{\mathbb{E}}_{v,x}[|{\mathbf{f}}(v,x)|^{p}]}{{\mathbb{E}}_{v,x}[|{\mathbf{f}}(v,x)|^{p}]}{{\mathbb{E}}_{v,x}[|{\mathbf{f}}(v,x)|^{p}]}^{1/p}$ in this space. 4. 4.

Combined Fourier space ${\mathbb{R}}^{V\times R}$. Each function in this space is denoted by ${\bf\widehat{f}}:V\times[R]\to{\mathbb{R}}$. The inner product is defined as $\left\langle{\bf\widehat{f}},{\bf\widehat{g}}\right\rangle:=\mathchoice{\underset{v\in V}{\mathbb{E}}\left[\sum_{i\in[R]}{\bf\widehat{f}}(v,i){\bf\widehat{g}}(v,i)\right]}{{\mathbb{E}}_{v\in V}[\sum_{i\in[R]}{\bf\widehat{f}}(v,i){\bf\widehat{g}}(v,i)]}{{\mathbb{E}}_{v\in V}[\sum_{i\in[R]}{\bf\widehat{f}}(v,i){\bf\widehat{g}}(v,i)]}{{\mathbb{E}}_{v\in V}[\sum_{i\in[R]}{\bf\widehat{f}}(v,i){\bf\widehat{g}}(v,i)]}$, which induces $\mathchoice{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}$, which is neither a counting nor an expectation norm.

Note that ${\mathbf{f}}\in{\mathbb{R}}^{V\times 2^{R}}$ and a vertex $v\in V$ induces $f_{v}\in{\mathbb{R}}^{2^{R}}$ defined by $f_{v}(x):={\mathbf{f}}(v,x)$, and similarly ${\bf\widehat{f}}\in{\mathbb{R}}^{V\times R}$ and a vertex $v\in V$ induces $\widehat{f}_{v}\in{\mathbb{R}}^{R}$ defined by $\widehat{f}_{v}(x):={\bf\widehat{f}}(v,x)$. As defined in Section˜4.2.2, we use the standard following (partial) Fourier transform $F$ that maps $f\in{\mathbb{R}}^{2^{R}}$ to $\widehat{f}\in{\mathbb{R}}^{R}$ as follows. 222We use only linear Fourier coefficients in this work. $F$ was defined as $F_{P}$ in Section 4.2.2.

[TABLE]

The (partial) inverse Fourier transform $F^{T}$ that maps $\widehat{f}\in{\mathbb{R}}^{R}$ to $f\in{\mathbb{R}}^{2^{R}}$ is defined by

[TABLE]

This Fourier transform can be naturally extended to combined spaces by defining ${\bf F}:{\mathbf{f}}\mapsto{\bf\widehat{f}}$ as $f_{v}\mapsto\widehat{f}_{v}$ for all $v\in V$. Then ${\bf F}^{T}$ maps ${\bf\widehat{f}}$ to ${\mathbf{f}}$ as $\widehat{f}_{v}\mapsto f_{v}$ for all $v\in V$.

Finally, let ${\bf\widehat{P}}:{\mathbb{R}}^{V\times R}\to{\mathbb{R}}^{V\times R}$ be the orthogonal projector to the following subspace of the combined Fourier space:

[TABLE]

Our transformation ${\bf A}:{\mathbb{R}}^{V\times 2^{R}}\to{\mathbb{R}}^{V\times 2^{R}}$ is defined by

[TABLE]

In other words, given ${\mathbf{f}}$, we apply the Fourier transform for each $v\in V$, project the combined Fourier coefficients to ${\bf\widehat{L}}$ that checks the Label Cover consistency, and apply the inverse Fourier transform. Since ${\bf\widehat{P}}$ is a projector, ${\bf A}$ is self-adjoint by design.

We also note that a similar reduction that produces $({\bf F}^{T}){\bf\widehat{P}}$ was used in Guruswami et al. [GRSW16] and Briët et al. [BRS15] for subspace approximation and Grothendieck-type problems, and indeed this reduction suffices for Theorem˜4.3.1 except the self-adjointness and additional properties in the completeness case.

Completeness.

We prove the following lemma for the completeness case. A simple intuition is that if $\mathcal{L}$ admits a good labeling, we can construct a ${\mathbf{f}}$ such that each $f_{v}$ is a linear function and ${\bf\widehat{f}}$ is already in the subspace ${\bf\widehat{L}}$. Therefore, each of Fourier transform, projection to ${\bf\widehat{L}}$, and inverse Fourier transform does not really change ${\mathbf{f}}$.

Lemma 4.3.2** (Completeness).**

Let $\ell:V\to[R]$ be a labeling that satisfies every edge of $\mathcal{L}$. There exists a function ${\mathbf{f}}\in{\mathbb{R}}^{V\times 2^{R}}$ such that ${\mathbf{f}}(v,x)$ is either $+1$ or $-1$ for all $v\in V,x\in\{\pm 1\}^{R}$ and ${\bf A}{\mathbf{f}}={\mathbf{f}}$.

• **Proof:  ** Let ${\mathbf{f}}(v,x):=x_{\ell(v)}$ for every $v\in V,x\in\{\pm 1\}^{R}$. Consider ${\bf\widehat{f}}={\bf F}{\mathbf{f}}$. For each vertex $v\in V$, ${\bf\widehat{f}}(v,i)=\widehat{f}_{v}(i)=1$ if $i=\ell(v)$ and [math] otherwise. Since $\ell$ satisfies every edge of $\mathcal{L}$, ${\bf\widehat{f}}\in{\bf\widehat{L}}$ and ${\bf\widehat{P}}{\bf\widehat{f}}={\bf\widehat{f}}$. Finally, since each $f_{v}$ is a linear function, the partial inverse Fourier transform $F^{T}$ satisfies $(F^{T})\widehat{f}_{v}=f_{v}$, which implies that $({\bf F}^{T}){\bf\widehat{f}}={\mathbf{f}}$. Therefore, ${\bf A}{\mathbf{f}}=({\bf F}^{T}{\bf\widehat{P}}{\bf F}){\mathbf{f}}={\mathbf{f}}$.

4.3.2 Soundness

We prove the following soundness lemma. This finishes the proof of Theorem˜4.3.1 since Theorem˜4.2.12 guarantees NP-hardness of Label Cover for arbitrarily small $\xi>0$ and arbitrarily large $J\in{\mathbb{N}}$.

Lemma 4.3.3** (Soundness).**

For every $\varepsilon>0$, there exist $\xi>0$ (that determines $D=D(\xi)$ as in Theorem˜4.2.12) and $J\in{\mathbb{N}}$ such that if $\operatorname{\operatorname{OPT}}(\mathcal{L})\leq\xi$, $\mathcal{L}$ is $D$-to-$1$, and $\mathcal{L}$ is $J$-smooth, $\mathchoice{\lVert{\bf A}\rVert_{L_{2}\to L_{r}}}{\lVert{\bf A}\rVert_{L_{2}\to L_{r}}}{\lVert{\bf A}\rVert_{L_{2}\to L_{r}}}{\lVert{\bf A}\rVert_{L_{2}\to L_{r}}}\leq\gamma_{r}+4\varepsilon^{2-r}$ for every $1\leq r<2$.

The proof closely follows that of Lemma 3.3 of [BRS15]. We (partially) generalize their proof for $1$-norm (of general norms) to $r$-norms when $1\leq r<2$.

• **Proof:  ** Let ${\mathbf{f}}\in{\mathbb{R}}^{V\times 2^{R}}$ be an arbitrary vector such that $\mathchoice{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}=1$. Let ${\bf\widehat{f}}={\bf F}{\mathbf{f}}$, ${\bf\widehat{g}}={\bf\widehat{L}}{\bf\widehat{f}}$, and ${\mathbf{g}}={\bf F}^{T}{\bf\widehat{g}}$ so that ${\mathbf{g}}=({\bf F}^{T}{\bf\widehat{L}}{\bf F}){\mathbf{f}}={\bf A}{\mathbf{f}}$. By Parseval’s theorem, $\mathchoice{\lVert\widehat{f}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{f}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{f}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{f}_{v}\rVert_{\ell_{2}}}\leq\mathchoice{\lVert f_{v}\rVert_{L_{2}}}{\lVert f_{v}\rVert_{L_{2}}}{\lVert f_{v}\rVert_{L_{2}}}{\lVert f_{v}\rVert_{L_{2}}}$ for all $v\in V$ and $\mathchoice{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}\leq\mathchoice{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}{\lVert{\mathbf{f}}\rVert_{L_{2}}}\leq 1$. Since ${\bf\widehat{L}}$ is an orthogonal projection, $\mathchoice{\lVert{\bf\widehat{g}}\rVert_{2}}{\lVert{\bf\widehat{g}}\rVert_{2}}{\lVert{\bf\widehat{g}}\rVert_{2}}{\lVert{\bf\widehat{g}}\rVert_{2}}\leq\mathchoice{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}{\lVert{\bf\widehat{f}}\rVert_{2}}\leq 1$. Fix $1\leq r<2$ and suppose

[TABLE]

Use Lemma˜4.2.10 to obtain $\delta=\delta(\epsilon)$ such that ${\lVert g_{v}\rVert^{p}_{L_{p}}}>(\gamma_{p}^{p}+\epsilon){\lVert\widehat{g}_{v}\rVert^{p}_{\ell_{2}}}$ implies $\mathchoice{\lVert\widehat{g}\rVert_{\ell_{4}}}{\lVert\widehat{g}\rVert_{\ell_{4}}}{\lVert\widehat{g}\rVert_{\ell_{4}}}{\lVert\widehat{g}\rVert_{\ell_{4}}}>\delta\mathchoice{\lVert\widehat{g}\rVert_{\ell_{2}}}{\lVert\widehat{g}\rVert_{\ell_{2}}}{\lVert\widehat{g}\rVert_{\ell_{2}}}{\lVert\widehat{g}\rVert_{\ell_{2}}}$ for all $1\leq p<2$ (so that $\delta$ does not depend on $r$), and consider

[TABLE]

We prove the following lemma that lower bounds the size of $V_{0}$.

Lemma 4.3.4**.**

For $V_{0}\subseteq V$ defined as in (4.9), we have $|V_{0}|\geq\epsilon^{2}|V|$.

• **Proof:  ** The proof closely follows the proof of Lemma 3.4 of [BRS15]. Define the sets

[TABLE]

From (4.8), we have

[TABLE]

We bound the four sums on the left side of (4.10) individually. Parseval’s theorem and the fact that $r<2$ implies $\mathchoice{\lVert g_{v}\rVert_{L_{r}}}{\lVert g_{v}\rVert_{L_{r}}}{\lVert g_{v}\rVert_{L_{r}}}{\lVert g_{v}\rVert_{L_{r}}}\leq\mathchoice{\lVert g_{v}\rVert_{L_{2}}}{\lVert g_{v}\rVert_{L_{2}}}{\lVert g_{v}\rVert_{L_{2}}}{\lVert g_{v}\rVert_{L_{2}}}=\mathchoice{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}$, and since $\mathchoice{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}\leq 1/\epsilon$ for every $v\in V_{0}$, the first sum in (4.10) can be bounded by

[TABLE]

Similarly, using the definition of $V_{1}$ the second sum in (4.10) is at most $\epsilon^{r}|V|$. By Lemma˜4.2.10, for each $v\in V_{2}$, we have ${\lVert g_{v}\rVert^{r}_{L_{r}}}\leq(\gamma_{r}^{r}+\epsilon){\lVert\widehat{g}_{v}\rVert^{r}_{\ell_{2}}}$. Therefore, the third sum in (4.10) is bounded as

[TABLE]

Finally, the fourth sum in (4.10) is bounded by

[TABLE]

Combining the above with (4.10) yields

[TABLE]

where the last inequality uses the fact that $\epsilon^{2-r}\geq\epsilon\geq\epsilon^{r}$.

Therefore, $|V_{0}|\geq\epsilon^{2}|V|$ and every vertex of $v$ satisfies $\mathchoice{\lVert\widehat{g}_{v}\rVert_{\ell_{4}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{4}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{4}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{4}}}>\delta\epsilon$ and $\mathchoice{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}{\lVert\widehat{g}_{v}\rVert_{\ell_{2}}}\leq 1/\epsilon$. Using only these two facts together with ${\bf\widehat{g}}\in{\bf\widehat{L}}$, Briët et al. [BRS15] proved that if the smoothness parameter $J$ is large enough given other parameters, $\mathcal{L}$ admits a labeling that satisfies a significant fraction of edges.

Lemma 4.3.5** **(Lemma

3.6 of [BRS15]).

Let $\beta:=\delta^{2}\epsilon^{3}$. There exists an absolute constant $c^{\prime}>0$ such that if $\mathcal{L}$ is $T$-to-$1$ and $T/(c^{\prime}\epsilon^{8}\beta^{4})$-smooth for some $T\in{\mathbb{N}}$, there is a labeling that satisfies at least $\epsilon^{8}\beta^{4}/1024$ fraction of $E$.

This finishes the proof of Lemma˜4.3.3 by setting $\xi:=\epsilon^{8}\beta^{4}/1024$ and $J:=D(\xi)/(c^{\prime}\epsilon^{8}\beta^{4})$ with $D(\xi)$ defined in Theorem˜4.2.12. Given a $3$-SAT formula, $\phi$, by the standard property of Smooth Label Cover, the size of the reduction is $|\phi|^{O(J\log(1/\xi))}=|\phi|^{\operatorname{\operatorname{poly}}(1/\varepsilon)}$.

4.4 Hardness of $p\to q$

In this section, we prove our main results. We prove Item˜(3) on hardness of approximating $p\to q$ when $p\geq 2\geq q$, and Item˜(1) on hardness of approximating $p\to q$ when $2<p<q$. By duality, the same hardness is implied for the case of $p<q<2$.

Our result for $p\geq 2\geq q$ in Section˜4.4.1 follows from Theorem˜4.3.1 using additional properties in the completeness case. For hypercontractive norms, we start by showing constant factor hardness via reduction from $p\to 2$ (see Section˜4.4.2), and then amplify the hardness factor by using the fact that all hypercontractive norms productivize under Kronecker product, which we prove in Section˜4.4.4.

4.4.1 Hardness for $p\geq 2\geq q$

We use Theorem˜4.3.1 to prove hardness of $p\to q$ for $p\geq 2\geq q$, which proves Item˜(3).

• **Proof Item˜(3):  ** Fix $p,q$, and $\delta>0$ such that $\infty\geq p\geq 2\geq q$ and $p>q$. Our goal is to prove that $p\to q$ is NP-hard to approximate within a factor $1/(\gamma_{p^{*}}\gamma_{q}+\delta)$. For $2\to q$ for $1\leq q<2$, Theorem 4.3.1 (with $\varepsilon\leftarrow\delta^{1/(2-q)}$) directly proves a hardness ratio of $1/(\gamma_{q}+\varepsilon^{2-q})=1/(\gamma_{q}+\delta)$. By duality, it also gives an $1/(\gamma_{p^{*}}+\delta)$ hardness for $p\to 2$ for $p>2$.

For $p\to q$ for $p>2>q$, apply Theorem˜4.3.1 with $\varepsilon=(\delta/3)^{\max(1/(2-p^{*}),1/(2-q))}$. It gives a polynomial time reduction that produces a symmetric matrix $A\in{\mathbb{R}}^{n\times n}$ given a 3-SAT formula $\phi$. Our instance for $p\to q$ is $AA^{T}=A^{2}$.

• –

(Completeness) If $\phi$ is satisfiable, there exists $x\in{\mathbb{R}}^{n}$ such that $|x(i)|=1$ for all $i\in[N]$ and $Ax=x$. Therefore, $A^{2}x=x$ and $\mathchoice{\lVert A^{2}\rVert_{L_{p}\to L_{q}}}{\lVert A^{2}\rVert_{L_{p}\to L_{q}}}{\lVert A^{2}\rVert_{L_{p}\to L_{q}}}{\lVert A^{2}\rVert_{L_{p}\to L_{q}}}\geq 1$.

• –

(Soundness) If $\phi$ is not satisfiable,

[TABLE]

This implies that

[TABLE]

This creates a gap of $1/(\gamma_{p^{*}}\gamma_{q}+\delta)$ between the completeness and the soundness case. The same gap holds for the counting norm since $\mathchoice{\lVert A^{2}\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A^{2}\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A^{2}\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A^{2}\rVert_{\ell_{p}\to\ell_{q}}}=n^{1/q-1/p}\cdot\mathchoice{\lVert A^{2}\rVert_{L_{p}\to L_{q}}}{\lVert A^{2}\rVert_{L_{p}\to L_{q}}}{\lVert A^{2}\rVert_{L_{p}\to L_{q}}}{\lVert A^{2}\rVert_{L_{p}\to L_{q}}}$.

4.4.2 Reduction from $p\to 2$ via Approximate Isometries

Let $A\in{\mathbb{R}}^{n\times n}$ be a hard instance of $p\to 2$. For any $q\geq 1$, if a matrix $B\in{\mathbb{R}}^{m\times n}$ satisfies $\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}=(1\pm o(1))\mathchoice{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}$ for all $x\in{\mathbb{R}}^{n}$, then $\mathchoice{\lVert BA\rVert_{p\to q}}{\lVert BA\rVert_{p\to q}}{\lVert BA\rVert_{p\to q}}{\lVert BA\rVert_{p\to q}}=(1\pm o(1))\mathchoice{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}{\lVert A\rVert_{p\to 2}}$. Thus, $BA$ will serve as a hard instance for $p\to q$ if one can compute such a matrix $B$ efficiently. In fact, a consequence of the Dvoretzky-Milman theorem is that a sufficiently tall random matrix $B$ satisfies the aforementioned property with high probability. In other words, for $m=m(q,n)$ sufficiently large, a random linear operator from $\ell_{2}^{n}$ to $\ell_{q}^{m}$ is an approximate isometry.

To restate this from a geometric perspective, for $m(q,n)$ sufficiently larger than $n$, a random section of the unit ball in $\ell_{q}^{m}$ is approximately isometric to the unit ball in $\ell_{2}^{n}$. In the interest of simplicity, we will instead state and use a corollary of the following matrix deviation inequality due to Schechtman (see [Sch06], Chapter 11 in [Ver17]).

Theorem 4.4.1** (Schechtman [Sch06]).**

Let $B$ be an $m\times n$ matrix with i.i.d. ${\mathcal{N}\left(0,1\right)}$ entries. Let $f:{\mathbb{R}}^{m}\to{\mathbb{R}}$ be a positive-homogeneous and subadditive function, and let $b$ be such that $f(y)\leq b\mathchoice{\lVert y\rVert_{\ell_{2}}}{\lVert y\rVert_{\ell_{2}}}{\lVert y\rVert_{\ell_{2}}}{\lVert y\rVert_{\ell_{2}}}$ for all $y\in{\mathbb{R}}^{m}$. Then for any $T\subset{\mathbb{R}}^{n}$,

[TABLE]

with probability at least $1-e^{-t^{2}}$, where $\mathrm{rad}(T)$ is the radius of $T$, and $\gamma(T)$ is the Gaussian complexity of $T$ defined as

[TABLE]

The above theorem is established by proving that the random process given by $X_{x}:=f(Bx)-\mathchoice{{\mathbb{E}}\left[f(Bx)\right]}{{\mathbb{E}}[f(Bx)]}{{\mathbb{E}}[f(Bx)]}{{\mathbb{E}}[f(Bx)]}$ has sub-gaussian increments with respect to $L_{2}$ and subsequently appealing to Talagrand’s Comparison tail bound.

We will apply this theorem with $f(\cdot)=\mathchoice{\lVert\cdot\rVert_{\ell_{q}}}{\lVert\cdot\rVert_{\ell_{q}}}{\lVert\cdot\rVert_{\ell_{q}}}{\lVert\cdot\rVert_{\ell_{q}}}$, $b=1$ and $T$ being the unit ball under $\mathchoice{\lVert\cdot\rVert_{\ell_{2}}}{\lVert\cdot\rVert_{\ell_{2}}}{\lVert\cdot\rVert_{\ell_{2}}}{\lVert\cdot\rVert_{\ell_{2}}}$. We first state a known estimate of $\mathchoice{{\mathbb{E}}\left[f(Bx)\right]}{{\mathbb{E}}[f(Bx)]}{{\mathbb{E}}[f(Bx)]}{{\mathbb{E}}[f(Bx)]}=\mathchoice{{\mathbb{E}}\left[\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}\right]}{{\mathbb{E}}[\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}]}{{\mathbb{E}}[\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}]}{{\mathbb{E}}[\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}]}$ for any fixed $x$ satisfying $\mathchoice{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}=1$. Note that when $\mathchoice{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}=1$, $Bx$ has the same distribution as an $m$-dimensional random vector with i.i.d. ${\mathcal{N}\left(0,1\right)}$ coordinates.

Theorem 4.4.2** (Biau and Mason [BM15]).**

Let $X\in{\mathbb{R}}^{m}$ be a random vector with i.i.d. ${\mathcal{N}\left(0,1\right)}$ coordinates. Then for any $q\geq 2$,

[TABLE]

We are now equipped to see that a tall random Gaussian matrix is an approximate isometry (as a linear map from $\ell_{2}^{n}$ to $\ell_{q}^{m}$) with high probability.

Corollary 4.4.3**.**

Let $B$ be an $m\times n$ matrix with i.i.d. ${\mathcal{N}\left(0,1\right)}$ entries where $m=\omega(n^{q/2})$. Then with probability at least $1-e^{-n}$, every vector $x\in{\mathbb{R}}^{n}$ satisfies,

[TABLE]

• **Proof:  ** We apply Theorem˜4.4.1 with function $f$ being the $\ell_{q}$ norm, $b=1$, and $t=\sqrt{n}$. Further we set $T$ to be the $\ell_{2}$ unit sphere, which yields $\gamma(T)=\Theta(\sqrt{n})$ and $\mathrm{rad}(T)=1$. Applying Theorem˜4.4.2 yields that with probability at least $1-e^{t^{2}}=1-e^{-n}$, for all $x$ with $\mathchoice{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}=1$, we have

[TABLE]

We thus obtain the desired constant factor hardness:

Proposition 4.4.4**.**

For any $p>2,~2\leq q<\infty$ and any $\varepsilon>0$, there is no polynomial time algorithm that approximates $p\to q$ (and consequently $q^{*}\to p^{*}$) within a factor of $1/\gamma_{p^{*}}-\varepsilon$  unless $\textsf{NP}\not\subseteq\textsf{BPP}$.

• **Proof:  ** By Corollary˜4.4.3, for every $n\times n$ matrix $A$ and a random $m\times n$ matrix $B$ with i.i.d. ${\mathcal{N}\left(0,1\right)}$ entries ($m=\omega(n^{q/2})$), with probability at least $1-e^{-n}$, we have

[TABLE]

Thus the reduction $A\rightarrow BA$ combined with $p\to 2$ hardness implied by Theorem˜4.3.1, yields the claim.

The generality of the concentration of measure phenomenon underlying the proof of the Dvoretzky-Milman theorem allows us to generalize Proposition˜4.4.4, to obtain constant factor hardness of maximizing various norms over the $\ell_{p}$ ball ($p>2$). In this more general version, the strength of our hardness assumption is dependent on the Gaussian width of the dual of the norm being maximized. Its proof is identical to that of Proposition˜4.4.4.

Theorem 4.4.5**.**

Consider any $p>2,\varepsilon>0$, and any family $(f_{m})_{m\in{\mathbb{N}}}$ of positive-homogeneous and subadditive functions where $f_{m}:{\mathbb{R}}^{m}\to{\mathbb{R}}$. Let $(b_{m})_{m\in{\mathbb{N}}}$ be such that $f_{m}(y)\leq b_{m}\cdot\mathchoice{\lVert y\rVert_{\ell_{2}}}{\lVert y\rVert_{\ell_{2}}}{\lVert y\rVert_{\ell_{2}}}{\lVert y\rVert_{\ell_{2}}}$ for all $y$ and let $N=N(n)$ be such that $\gamma_{*}(f_{N})=\omega(b_{N}\cdot\sqrt{n})$, where

[TABLE]

Then unless $\textsf{NP}\not\subseteq\textsf{BPTIME}\left(N(n)\right)$, there is no polynomial time $(1/\gamma_{p^{*}}-\varepsilon)$-approximation algorithm for the problem of computing $\sup_{\mathchoice{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}{\lVert x\rVert_{p}}=1}f_{m}(Ax)$, given an $m\times n$ matrix $A$.

4.4.3 Derandomized Reduction

In this section, we show how to derandomize the reduction in Proposition˜4.4.4 to obtain NP-hardness when $q\geq 2$ is an even integer and $p>2$. Similarly to Section˜4.4.2, given $A\in{\mathbb{R}}^{n\times n}$ as a hard instance of $p\to 2$, our strategy is to construct a matrix $B\in{\mathbb{R}}^{m\times n}$ and output $BA$ as a hard instance of $p\to q$.

Instead of requiring $B$ to satisfy $\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}=(1\pm o(1))\mathchoice{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}$ for all $x\in{\mathbb{R}}^{n}$, we show that $\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}\leq(1+o(1))\mathchoice{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}$ for all $x\in{\mathbb{R}}^{n}$ and $\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}\geq(1-o(1))\mathchoice{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}{\lVert x\rVert_{\ell_{2}}}$ when every coordinate of $x$ has the same absolute value. Since Theorem˜4.3.1 ensures that $\mathchoice{\lVert A\rVert_{\ell_{p}\to\ell_{2}}}{\lVert A\rVert_{\ell_{p}\to\ell_{2}}}{\lVert A\rVert_{\ell_{p}\to\ell_{2}}}{\lVert A\rVert_{\ell_{p}\to\ell_{2}}}$ is achieved by $x=Ax$ for such a well-spread $x$ in the completeness case, $BA$ serves as a hard instance for $p\to q$.

We use the following construction of $q$-wise independent sets to construct such a $B$ deterministically.

Theorem 4.4.6** (Alon, Babai, and Itai [ABI86]).**

For any $k\in{\mathbb{N}}$, one can compute a set $S$ of vectors in $\{\pm 1\}^{n}$ of size $O(n^{k/2})$, in time $n^{O(k)}$, such that the vector random variable $Y$ obtained by sampling uniformly from $S$ satisfies that for any $I\in\binom{[n]}{k}$, the marginal distribution Y\raise-2.15277pt\hbox{|}_{I} is the uniform distribution over $\{\pm 1\}^{k}$.

For a matrix $B$ as above, a randomly chosen row behaves similarly to an $n$-dimensional Rademacher random vector with respect to $\mathchoice{\lVert\cdot\rVert_{\ell_{q}}}{\lVert\cdot\rVert_{\ell_{q}}}{\lVert\cdot\rVert_{\ell_{q}}}{\lVert\cdot\rVert_{\ell_{q}}}$.

Corollary 4.4.7**.**

Let $R\in{\mathbb{R}}^{n}$ be a vector random variable with i.i.d. Rademacher ($\pm 1$) coordinates. For any even integer $q\geq 2$, there is an $m\times n$ matrix $B$ with $m=O(n^{q/2})$, computable in $n^{O(q)}$ time, such that for all $x\in{\mathbb{R}}^{n}$, we have

[TABLE]

• **Proof:  **Let $B$ be a matrix, the set of whose rows is precisely $S$. By Theorem˜4.4.6,

[TABLE]

We use the following two results that will bound $\mathchoice{\lVert BA\rVert_{\ell_{p}\to\ell_{q}}}{\lVert BA\rVert_{\ell_{p}\to\ell_{q}}}{\lVert BA\rVert_{\ell_{p}\to\ell_{q}}}{\lVert BA\rVert_{\ell_{p}\to\ell_{q}}}$ for the completeness case and the soundness case respectively.

Theorem 4.4.8** (Stechkin [Ste61]).**

Let $R\in{\mathbb{R}}^{n}$ be a vector random variable with i.i.d. Rademacher coordinates. Then for any $q\geq 2$ and any $x\in{\mathbb{R}}^{n}$ whose coordinates have the same absolute value,

[TABLE]

Theorem 4.4.9** (Khintchine inequality [Haa81]).**

Let $R\in{\mathbb{R}}^{n}$ be a vector random variable with i.i.d. Rademacher coordinates. Then for any $q\geq 2$ and any $x\in{\mathbb{R}}^{n}$,

[TABLE]

We finally prove the derandomimzed version of Proposition˜4.4.4 for even $q\geq 2$.

Proposition 4.4.10**.**

For any $p>2,\varepsilon>0$, and any even integer $q\geq 2$, it is NP-hard to approximate $p\to q$ within a factor of  $1/\gamma_{p^{*}}-\varepsilon$.

• **Proof:  ** Apply Theorem˜4.3.1 with $r_{1}\leftarrow p^{*}$ and $\varepsilon\leftarrow\varepsilon$. Given an instance $\phi$ of 3-SAT, Theorem˜4.3.1 produces a symmetric matrix $A\in{\mathbb{R}}^{n\times n}$ in polynomial time as a hard instance of $p\to 2$. Our instance for $p\to q$ is $BA$ where $B$ is the $m\times n$ matrix given by Corollary˜4.4.7 with $m=O(n^{q/2})$.

• –

(Completeness) If $\phi$ is satisfiable, there exists a vector $x\in\{\pm\frac{1}{\sqrt{n}}\}^{n}$ such that $Ax=x$. So we have $\mathchoice{\lVert BAx\rVert_{\ell_{q}}}{\lVert BAx\rVert_{\ell_{q}}}{\lVert BAx\rVert_{\ell_{q}}}{\lVert BAx\rVert_{\ell_{q}}}=\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}=(1-o(1))\cdot m^{1/q}\cdot\gamma_{q}$, where the last equality uses Corollary˜4.4.7 and Theorem˜4.4.8. Thus, $\mathchoice{\lVert BA\rVert_{\ell_{p}\to\ell_{q}}}{\lVert BA\rVert_{\ell_{p}\to\ell_{q}}}{\lVert BA\rVert_{\ell_{p}\to\ell_{q}}}{\lVert BA\rVert_{\ell_{p}\to\ell_{q}}}\geq(1-o(1))\cdot m^{1/q}\cdot\gamma_{q}$.

• –

(Soundness) If $\phi$ is not satisfiable, then for any $x$ with $\mathchoice{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}=1$,

[TABLE]

where the first inequality is a direct application of Theorem˜4.4.9.

4.4.4 Hypercontractive Norms Productivize

We will next amplify our hardness results using the fact that hypercontractive norms productivize under the natural operation of Kronecker or tensor product. Bhaskara and Vijayraghavan [BV11] showed this for the special case of $p=q$ and the Harrow and Montanaro [HM13] showed this for $2\to 4$ (via parallel repetition for $\mathrm{QMA(2)}$). In this section we prove this claim whenever $p\leq q$.

Theorem 4.4.11**.**

Let $A$ and $B$ be $m_{1}\times n_{1}$ and $m_{2}\times n_{2}$ matrices respectively. Then for any $1\leq p\leq q<\infty$, $\mathchoice{\lVert A\otimes B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\otimes B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\otimes B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\otimes B\rVert_{\ell_{p}\to\ell_{q}}}\leq\mathchoice{\lVert A\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\rVert_{\ell_{p}\to\ell_{q}}}\cdot\mathchoice{\lVert B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert B\rVert_{\ell_{p}\to\ell_{q}}}$.

• **Proof:  ** We will begin with some notation. Let $a_{i},b_{j}$ respectively denote the $i$-th and $j$-th rows of $A$ and $B$. Consider any $z\in{\mathbb{R}}^{[n_{1}]\times[n_{2}]}$ satisfying $\mathchoice{\lVert z\rVert_{\ell_{p}}}{\lVert z\rVert_{\ell_{p}}}{\lVert z\rVert_{\ell_{p}}}{\lVert z\rVert_{\ell_{p}}}=1$. For $k\in[n_{1}]$, let $z_{k}\in{\mathbb{R}}^{n_{2}}$ denote the vector given by $z_{k}(\ell):=z(k,\ell)$. For $j\in[m_{2}]$, let $\overline{z}_{j}\in{\mathbb{R}}^{n_{1}}$ denote the vector given by $\overline{z}_{j}(k):=\left\langle b_{j},z_{k}\right\rangle$. Finally, for $k\in[n_{1}]$, let $\lambda_{k}:={\lVert z_{k}\rVert^{p}_{\ell_{p}}}$ and let $v_{k}\in{\mathbb{R}}^{m_{2}}$ be the vector given by $v_{k}(j):=|\overline{z}_{j}(k)|^{p}/\lambda_{k}$.

We begin by ’peeling off’ $A$:

[TABLE]

In the special case of $p=q$, the proof ends here since the expression is a sum of terms of the form ${\lVert By\rVert^{p}_{\ell_{p}}}$ and can thus be upper bounded term-wise by ${\lVert B\rVert^{p}_{\ell_{p}\to\ell_{p}}}\cdot{\lVert z_{k}\rVert^{p}_{\ell_{p}}}$ which sums to ${\lVert B\rVert^{p}_{\ell_{q}\to\ell_{p}}}$. To handle the case of $q>p$, we will use a convexity argument:

[TABLE]

It remains to show that ${\lVert v_{k}\rVert^{q/p}_{\ell_{q/p}}}$ is precisely ${\lVert Bz_{k}\rVert^{q}_{\ell_{q}}}/{\lVert z_{k}\rVert^{q}_{\ell_{p}}}$.

[TABLE]

Thus we have established $\mathchoice{\lVert A\otimes B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\otimes B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\otimes B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\otimes B\rVert_{\ell_{p}\to\ell_{q}}}\leq\mathchoice{\lVert A\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\rVert_{\ell_{p}\to\ell_{q}}}{\lVert A\rVert_{\ell_{p}\to\ell_{q}}}\cdot\mathchoice{\lVert B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert B\rVert_{\ell_{p}\to\ell_{q}}}{\lVert B\rVert_{\ell_{p}\to\ell_{q}}}$. Lastly, the claim follows by observing that the statement is equivalent to the statement obtained by replacing the counting norms with expectation norms.

We finally establish super constant NP-Hardness of approximating $p\to q$, proving Item˜(1).

• **Proof Item˜(1):  ** Fix $2<p\leq q<\infty$. Proposition˜4.4.4 states that there exists $c=c(p,q)>1$ such that any polynomial time algorithm approximating the $p\to q$ of an $n\times n$-matrix $A$ within a factor of $c$ will imply $\textsf{NP}\subseteq\textsf{BPP}$. Using Theorem˜4.4.11, for any integer $k\in{\mathbb{N}}$ and $N=n^{k}$, any polynomial time algorithm approximating the $p\to q$ of an $N\times N$-matrix $A^{\otimes k}$ within a factor of $c^{k}$ implies that NP admits a randomized algorithm running in time $\operatorname{\operatorname{poly}}(N)=n^{O(k)}$. Under $\textsf{NP}\not\subseteq\textsf{BPP}$, any constant factor approximation algorithm is ruled out by setting $k$ to be a sufficiently large constant. For any $\varepsilon>0$, setting $k=\log^{1/\varepsilon}n$ rules out an approximation factor of $c^{k}=2^{O(\log^{1-\varepsilon}N)}$ unless $\textsf{NP}\subseteq\textsf{BPTIME}\left(2^{\log^{O(1)}n}\right)$.

By duality, the same statements hold for $1<p\leq q<2$. When $2<p\leq q$ and $q$ is an even integer, all reductions become deterministic due to Proposition˜4.4.10.

4.4.5 A Simple Proof of Hardness for the Case $2\notin[q,p]$

In this section, we prove Item˜(2). That is, we show how to prove an almost-polynomial factor hardness for approximating $p\to q$ in the non-hypercontractive case when $2>p\geq q$ (and the case $p\geq q>2$ follows by duality). This result is already known from the work of Bhaskara and Vijayaraghavan [BV11]. We show how to obtain a more modular proof, composing our previous results with a simple embedding argument. However, while the reduction in [BV11] was deterministic, we will only give a randomized reduction below.

As in [BV11], we start with a strong hardness for the $p\to p$, obtained in Item˜(1). While the reduction in [BV11] relied on special properties of the instance for $\ell_{p}\to\ell_{p}$, we can simply use the following embedding result of Schechtman [Sch87] (phrased in a way convenient for our application).

Theorem 4.4.12** (Schechtman [Sch87], Theorem 5).**

Let $q<p<2$ and $\varepsilon>0$. Then, there exists a polynomial time samplable distribution $\mathcal{D}$ on random matrices in ${\mathbb{R}}^{m\times n}$ with $m=\Omega_{\varepsilon}(n^{3})$, such that with probability $1-o(1)$, we have for every $x\in R^{n}$, $\mathchoice{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}{\lVert Bx\rVert_{\ell_{q}}}~=~(1\pm\varepsilon)\cdot\mathchoice{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}{\lVert x\rVert_{\ell_{p}}}$.

In fact the distribution $\mathcal{D}$ is based on $p$-stable distributions. While the theorem in [Sch87] does not mention the high probability bound or samplability, it is easy to modify the proof to obtain these properties. We provide a proof sketch below for completeness. We note that Schechtman obtains a stronger bound of $O(n^{1+p/q})$ on the dimension $m$ of the $\ell_{q}$ space, which requires a more sophisticated argument using “Lewis weights”. However, we only state weaker $O(n^{3})$ bound above, which suffices for our purposes and is easier to convert to a samplable distribution.

We first prove the following hardness result for approximating $p\to q$ in the reverse-hypercontractive case, using Theorem˜4.4.12.

Theorem 4.4.13**.**

For any $p,q$ such that $1<q\leq p<2$ or $2<q\leq p<\infty$ and $\epsilon>0$, there is no polynomial time algorithm that approximates the $p\to q$ of an $n\times n$ matrix within a factor $2^{\log^{1-\epsilon}n}$ unless $\textsf{NP}\subseteq\textsf{BPTIME}\left(2^{(\log n)^{O(1)}}\right)$.

• **Proof:  ** We consider the case $1<q\leq p<2$ (the other case follows via duality). Item˜(1) gives a reduction from SAT on $n$ variables, approximating the $p\to p$ of matrices $A\in{\mathbb{R}}^{N\times N}$ with $N=2^{(\log n)^{O(1/\varepsilon)}}$, within a factor $2^{(\log N)^{1-\varepsilon}}$. Sampling a matrix $B$ from the distribution $\mathcal{D}$ given by Theorem˜4.4.12 (with dimension $N$) gives that it is also hard to approximate $\mathchoice{\lVert BA\rVert_{p\to q}}{\lVert BA\rVert_{p\to q}}{\lVert BA\rVert_{p\to q}}{\lVert BA\rVert_{p\to q}}\approx\mathchoice{\lVert A\rVert_{p\to p}}{\lVert A\rVert_{p\to p}}{\lVert A\rVert_{p\to p}}{\lVert A\rVert_{p\to p}}$, within a factor $2^{(\log N)^{1-\varepsilon}}$.

We now give a sketch of the proof of Theorem˜4.4.12 including the samplability condition. The key idea is to embed the space $\ell_{p}^{n}$ into the infinite-dimensional space $L_{q}$ (for $0\leq q\leq p<2$) using $p$-stable random variables. The corresponding subspace of $L_{q}$ can then be embedded into $\ell_{q}^{m}$ if the random variables (elements of $L_{q}$) constructed in the previous space are bounded in $L_{\infty}$ norm. This is the content of the following claim.

Claim 4.4.14** (Schechtman [Sch87], Proposition 4).**

Let $\varepsilon>0$ and $\Omega$ be an efficiently samplable probability space and let $V$ be an $n$-dimensional subspace of $L_{q}(\Omega)$, such that

[TABLE]

Then there exists a polynomial time samplable distribution $\mathcal{D}$ over linear operators $T:L_{q}(\Omega)\to{\mathbb{R}}^{m}$ for $m=C(\varepsilon,q)\cdot n\cdot M^{q}$ such that with probability $1-o(1)$, we have that for every $f\in V$, $\mathchoice{\lVert Tf\rVert_{\ell_{q}}}{\lVert Tf\rVert_{\ell_{q}}}{\lVert Tf\rVert_{\ell_{q}}}{\lVert Tf\rVert_{\ell_{q}}}=(1\pm\varepsilon)\cdot\mathchoice{\lVert f\rVert_{L_{q}}}{\lVert f\rVert_{L_{q}}}{\lVert f\rVert_{L_{q}}}{\lVert f\rVert_{L_{q}}}$.

• **Proof Sketch:  ** The linear operator is simply defined by sampling $x_{1},\ldots,x_{m}\sim\Omega$ independently, and taking

[TABLE]

The proof then follows by concentration bounds for $L_{\infty}$-bounded random variables, and a union bound over an epsilon net for the space $V$.   $\Box$

The problem then reduces to constructing an embedding of $\ell_{p}^{n}$ into $L_{q}$, which is bounded in $L_{\infty}$ norm. While a simple embedding can be constructed using $p$-stable distributions, Schechtman uses a clever reweighting argument to control the $L_{\infty}$ norm. We show below that a simple truncation argument can also be used to obtain a somewhat crude bound on the $L_{\infty}$ norm, which suffices for our purposes and yields an easily samplable distribution.

We collect below the relevant facts about $p$-stable random variables needed for our argument, which can be found in many well-known references, including [Ind06, AK06b].

Fact 4.4.15**.**

For all $p\in(0,2)$, there exist (normalized) $p$-stable random variables $Z$ satisfying the following properties:

For $Z_{1},\ldots,Z_{n}$ iid copies of $Z$, and for all $a\in{\mathbb{R}}^{n}$, the random variable

[TABLE]

has distribution identical to $Z$. 2. 2.

For all $q<p$, we have

[TABLE] 3. 3.

There exists a constant $C_{p}$ such that for all $t>0$,

[TABLE] 4. 4.

$Z$* can be sampled by choosing $\theta\in_{R}[-\pi/2,\pi/2]$, $r\in_{R}[0,1]$, and taking*

[TABLE]

We now define an embedding of $\ell_{p}^{n}$ into $L_{q}$ with bounded $L_{\infty}$, using truncated $p$-stable random variables. Let $Z=(Z_{1},\ldots,Z_{n})$ be a vector of iid $p$-stable random variables as above, and let $B$ be a parameter to be chosen later. We consider the random variables

[TABLE]

For all $a\in{\mathbb{R}}^{n}$, we define the (linear) embedding

[TABLE]

By the properties of $p$-stable distributions, we know that $\mathchoice{\lVert\left\langle a,Z\right\rangle/C_{p,q}\rVert_{L_{q}}}{\lVert\left\langle a,Z\right\rangle/C_{p,q}\rVert_{L_{q}}}{\lVert\left\langle a,Z\right\rangle/C_{p,q}\rVert_{L_{q}}}{\lVert\left\langle a,Z\right\rangle/C_{p,q}\rVert_{L_{q}}}=\mathchoice{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}$ for all $a\in{\mathbb{R}}^{n}$. By the following claim, we can choose $B$ so that the second term only introduces a small error.

Claim 4.4.16**.**

For all $\varepsilon>0$, there exists $B=O_{p,q,\varepsilon}(n^{1/p})$ such that for the embedding $\phi$ defined above

[TABLE]

• **Proof:  ** By triangle inequality, it suffices to bound $\mathchoice{\lVert\Delta(Z)\cdot\left\langle a,Z\right\rangle\rVert_{L_{q}}}{\lVert\Delta(Z)\cdot\left\langle a,Z\right\rangle\rVert_{L_{q}}}{\lVert\Delta(Z)\cdot\left\langle a,Z\right\rangle\rVert_{L_{q}}}{\lVert\Delta(Z)\cdot\left\langle a,Z\right\rangle\rVert_{L_{q}}}$ by $\varepsilon\cdot C_{p,q}\cdot\mathchoice{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}$. Let $\delta>0$ be such that $(1+\delta)\cdot q<p$. Using the fact that $\Delta(Z)$ is Boolean and Hölder’s inequality, we observe that

[TABLE]

Thus, choosing $B=O_{\varepsilon,p,q}(n^{1/p})$ such that

[TABLE]

proves the claim.

Using the value of $B$ as above, we now observe a bound on $\mathchoice{\lVert\phi(a)\rVert_{L_{\infty}}}{\lVert\phi(a)\rVert_{L_{\infty}}}{\lVert\phi(a)\rVert_{L_{\infty}}}{\lVert\phi(a)\rVert_{L_{\infty}}}$.

Claim 4.4.17**.**

Let $B=O_{\varepsilon,p,q}(n^{1/p})$ be chosen as above. Then, we have that

[TABLE]

• **Proof:  ** By the choice of $B$, we have that $\mathchoice{\lVert\left\langle a,Y\right\rangle\rVert_{L_{q}}}{\lVert\left\langle a,Y\right\rangle\rVert_{L_{q}}}{\lVert\left\langle a,Y\right\rangle\rVert_{L_{q}}}{\lVert\left\langle a,Y\right\rangle\rVert_{L_{q}}}\geq(1-\varepsilon)\mathchoice{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}$. Thus, we can assume that $\mathchoice{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}{\lVert a\rVert_{\ell_{p}}}\leq 2$. Hölder’s inequality then gives for all such $a$,

[TABLE]

which proves the claim.

Using the above bound on $M$ in ˜4.4.14 gives a bound of $m=O_{\varepsilon,p,q}(n^{q+1})=O_{\varepsilon,p,q}(n^{3})$. Moreover, the distribution over embeddings is efficiently samplable, since it obtained by truncating $p$-stable random variables. This completes the proof of Theorem˜4.4.12.

4.5 Analyzing the Approximation Ratio via Rounding

We will show that $\mathsf{CP}(A)$ is a good approximation to $\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}$ by using an appropriate generalization of Krivine’s rounding procedure. Before stating the generalized procedure, we shall give a more detailed summary of Krivine’s procedure.

4.5.1 Krivine’s Rounding Procedure

Krivine’s procedure centers around the classical random hyperplane rounding. In this context, we define the random hyperplane rounding procedure on an input pair of matrices $U\in{\mathbb{R}}^{m\times\ell},~V\in{\mathbb{R}}^{n\times\ell}$ as outputting the vectors $\operatorname{\operatorname{sgn}}[U{\bf g}]$ and $\operatorname{\operatorname{sgn}}[V{\bf g}]$ where ${\bf g}\in{\mathbb{R}}^{\ell}$ is a vector with i.i.d. standard Gaussian coordinates ($f[v]$ denotes entry-wise application of a scalar function $f$ to a vector $v$. We use the same convention for matrices.). The so-called Grothendieck identity states that for vectors $u,v\in{\mathbb{R}}^{\ell}$,

[TABLE]

where $\widehat{u}$ denotes $u/\mathchoice{\lVert u\rVert_{2}}{\lVert u\rVert_{2}}{\lVert u\rVert_{2}}{\lVert u\rVert_{2}}$. This implies the following equality which we will call the hyperplane rounding identity:

[TABLE]

where for a matrix $U$, we use $\widehat{U}$ to denote the matrix obtained by replacing the rows of $U$ by the corresponding unit (in $\ell_{2}$ norm) vectors. Krivine’s main observation is that for any matrices $U,V$, there exist matrices $\phi(\widehat{U}),\psi(\widehat{V})$ with unit vectors as rows, such that

[TABLE]

where $c=\sinh^{-1}(1)\cdot 2/\pi$. Taking $\widehat{U},\widehat{V}$ to be the optimal solution to $\mathsf{CP}(A)$, it follows that

[TABLE]

The proof of Krivine’s observation follows from simulating the Taylor series of a scalar function using inner products. We will now describe this more concretely.

Observation 4.5.1** (Krivine).**

Let $f:[-1,1]\to{\mathbb{R}}$ be a scalar function satisfying $f(\rho)=\sum_{k\geq 1}f_{k}\,\rho^{k}$ for an absolutely convergent series $(f_{k})$. Let $\mathrm{abs}\left(f\right)(\rho):=\sum_{k\geq 1}|f_{k}|\,\rho^{k}$ and further for vectors $u,v\in{\mathbb{R}}^{\ell}$ of $\ell_{2}$-length at most $1$, let

[TABLE]

Then for any $U\in{\mathbb{R}}^{m\times\ell},~V\in{\mathbb{R}}^{n\times\ell}$,  $S_{L}(f,\sqrt{c_{f}}\cdot\widehat{U})$ and $S_{R}(f,\sqrt{c_{f}}\cdot\widehat{V})$ have $\ell_{2}$-unit vectors as rows, and

[TABLE]

where $S_{L}(f,W)$ for a matrix $W$, is applied to row-wise and $c_{f}:=(\mathrm{abs}\left(f\right)^{-1})(1)$.

• **Proof:  ** Using the facts $\left\langle y^{1}\otimes y^{2},y^{3}\otimes y^{4}\right\rangle=\left\langle y^{1},y^{3}\right\rangle\cdot\left\langle y^{2},y^{4}\right\rangle$ and

$\left\langle y^{1}\oplus y^{2},y^{3}\oplus y^{4}\right\rangle=\left\langle y^{1},y^{3}\right\rangle+\left\langle y^{2},y^{4}\right\rangle$, we have

• –

$\left\langle S_{L}(f,u),S_{R}(f,v)\right\rangle=f(\left\langle u,v\right\rangle)$

• –

$\mathchoice{\lVert S_{L}(f,u)\rVert_{2}}{\lVert S_{L}(f,u)\rVert_{2}}{\lVert S_{L}(f,u)\rVert_{2}}{\lVert S_{L}(f,u)\rVert_{2}}=\sqrt{\mathrm{abs}\left(f\right)({\lVert u\rVert^{2}_{2}})}$

• –

$\mathchoice{\lVert S_{R}(f,v)\rVert_{2}}{\lVert S_{R}(f,v)\rVert_{2}}{\lVert S_{R}(f,v)\rVert_{2}}{\lVert S_{R}(f,v)\rVert_{2}}=\sqrt{\mathrm{abs}\left(f\right)({\lVert v\rVert^{2}_{2}})}$

The claim follows.

Before stating our full rounding procedure, we first discuss a natural generalization of random hyperplane rounding, and much like in Krivine’s case this will guide the final procedure.

4.5.2 Generalizing Random Hyperplane Rounding – HölderDual Rounding

Fix any convex bodies $B_{1}\subset{\mathbb{R}}^{m}$ and $B_{2}\subset{\mathbb{R}}^{k}$. Suppose that we would like a strategy that for given vectors $y\in{\mathbb{R}}^{m},~x\in{\mathbb{R}}^{n}$, outputs $\overline{y}\in B_{1},~\overline{x}\in B_{2}$ so that $y^{T}A\,x=\left\langle A,y\,x^{T}\right\rangle$ is close to $\left\langle A,\overline{y}\,\overline{x}^{T}\right\rangle$ for all $A$. A natural strategy is to take

[TABLE]

In the special case where $B$ is the unit $\ell_{p}$ ball, there is a closed form for an optimal solution to $\max_{\widetilde{x}\in B}\left\langle\widetilde{x},x\right\rangle$, given by $\Psi_{\!p^{*}}(x)/{\lVert x\rVert^{p^{*}-1}_{p^{*}}}$, where $\Psi_{\!p^{*}}(x):=\operatorname{\operatorname{sgn}}[x]\circ|[x]|^{p^{*}-1}$. Note that for $p=\infty$, this strategy recovers the random hyperplane rounding procedure. We shall call this procedure, Gaussian HölderDual Rounding or HölderDual Rounding for short.

Just like earlier, we will first understand the effect of HölderDual Rounding on a solution pair $U,V$. For $\rho\in[-1,1]$, let ${\bf g}_{1}\!\sim_{\rho}{\bf g}_{2}$ denote $\rho$-correlated standard Gaussians, i.e., ${\bf g}_{1}=\rho\,{\bf g}_{2}+\sqrt{1-\rho^{2}}\,{\bf g}_{3}$  where $({\bf g}_{2},{\bf g}_{3})\sim{\mathcal{N}\left(0,\mathrm{I}_{2}\right)}$, and let

[TABLE]

We will work towards a better understanding of $\widetilde{f}_{a,\,b}(\cdot)$ in later sections. For now note that we have for vectors $u,v\in{\mathbb{R}}^{\ell}$,

[TABLE]

Thus given matrices $U,V$, we obtain the following generalization of the hyperplane rounding identity for HölderDual Rounding :

[TABLE]

4.5.3 Generalized Krivine Transformation and the Full Rounding Procedure

We are finally ready to state the generalized version of Krivine’s algorithm. At a high level the algorithm simply applies HölderDual Rounding to a transformed version of the optimal convex program solution pair $U,V$. Analogous to Krivine’s algorithm, the transformation is a type of “inverse” of Eq.˜4.16.

1. (Inversion 1)

Let $(U,V)$ be the optimal solution to $\mathsf{CP}(A)$, and let $(u^{i})_{i\in[m]}$ and $(v^{j})_{j\in[n]}$ respectively denote the rows of $U$ and $V$. 2. (Inversion 2)

Let $c_{a,b}:=\left(\mathrm{abs}\left(\widetilde{f}^{-1}_{a,\,b}\right)\right)^{-1}\!\!(1)$ and let

[TABLE]

1. (Hölder-Dual 1)

Let ${\bf g}\sim{\mathcal{N}\left(0,\mathrm{I}\right)}$ be an infinite dimensional i.i.d. Gaussian vector. 2. (Hölder-Dual 2)

Return $y:=\Psi_{\!q}(\phi(U)\,{\bf g})/{\lVert\phi(U)\,{\bf g}\rVert^{b}_{q}}$ and $x:=\Psi_{\!p^{*}}(\psi(V)\,{\bf g})/{\lVert\psi(V)\,{\bf g}\rVert^{a}_{p^{*}}}$.

Remark 4.5.2**.**

Note that $\mathchoice{\lVert\Psi_{\!r}(\overline{x})\rVert_{r^{*}}}{\lVert\Psi_{\!r}(\overline{x})\rVert_{r^{*}}}{\lVert\Psi_{\!r}(\overline{x})\rVert_{r^{*}}}{\lVert\Psi_{\!r}(\overline{x})\rVert_{r^{*}}}={\lVert\overline{x}\rVert^{r-1}_{r}}$ and so the returned solution pair always lie on the unit $\ell_{q^{*}}$ and $\ell_{p}$ spheres respectively.

Remark 4.5.3**.**

Like in [AN04] the procedure above can be made algorithmic by observing that there always exist $\phi^{\prime}(U)\in{\mathbb{R}}^{m\times(m+n)}$ and $\psi^{\prime}(V)\in{\mathbb{R}}^{m\times(m+n)}$, whose rows have the exact same lengths and pairwise inner products as those of $\phi(U)$ and $\psi(V)$ above. Moreover they can be computed without explicitly computing $\phi(U)$ and $\psi(V)$ by obtaining the Gram decomposition of

[TABLE]

and normalizing the rows of the decomposition according to the definition of $\phi(\cdot)$ and $\psi(\cdot)$ above. The entries of $M$ can be computed in polynomial time with exponentially (in $m$ and $n$) good accuracy by implementing the Taylor series of $\widetilde{f}^{-1}_{a,\,b}$ upto $\mathrm{poly}(m,n)$ terms (Taylor series inversion can be done upto $k$ terms in time $\mathrm{poly}(k)$).

Remark 4.5.4**.**

Note that the $2$-norm of the $i$-th row (resp. $j$-th row) of $\phi(U)$ (resp. $\psi(V)$) is ${\lVert u^{i}\rVert^{1/b}_{2}}$ (resp. ${\lVert v^{j}\rVert^{1/a}_{2}}$).

We commence the analysis by defining some convenient normalized functions, and we will also show that $c_{a,b}$ above is well-defined.

4.5.4 Auxiliary Functions

Let  $f_{p,\,q}(\rho):=\widetilde{f}_{p,\,q}(\rho)/(\gamma_{p^{*}}^{p^{*}}\,\gamma_{q}^{q})$,   $\widetilde{h}_{a,\,b}:=\mathrm{abs}\left(\widetilde{f}^{-1}_{a,\,b}\right)$,  and  $h_{a,\,b}:=\mathrm{abs}\left(f^{-1}_{a,\,b}\right)$. Also note that $h^{-1}_{a,\,b}(\rho)=\widetilde{h}^{-1}_{a,\,b}(\rho)/(\gamma_{p^{*}}^{p^{*}}\,\gamma_{q}^{q})$.

Well Definedness.

By Lemma˜4.7.7, $f^{-1}_{a,\,b}(\rho)$ and $h_{a,\,b}(\rho)$ are well-defined for $\rho\in[-1,1]$. By (M1) in Corollary˜4.6.6,   $\widetilde{f}^{-1}_{1}=1$ and hence $h_{a,\,b}(1)\geq 1$ and $h_{a,\,b}(-1)\leq-1$. Combining this with the fact that $h_{a,\,b}(\rho)$ is continuous and strictly increasing on $[-1,1]$, implies that $h^{-1}_{a,\,b}(x)$ is well-defined on $[-1,1]$.

We can now proceed with the analysis.

4.5.5 $1/(h^{-1}_{p,\,q}(1)\cdot\gamma_{p^{*}}\,\gamma_{q})$

Bound on Approximation Factor

For any vector random variable ${\bf X}$ in a universe $\Omega$, and scalar valued functions $f_{1}:\Omega\to{\mathbb{R}}$ and $f_{2}:\Omega\to(0,\infty)$. Let $\lambda=\mathchoice{{\mathbb{E}}\left[f_{1}({\bf X})\right]}{{\mathbb{E}}[f_{1}({\bf X})]}{{\mathbb{E}}[f_{1}({\bf X})]}{{\mathbb{E}}[f_{1}({\bf X})]}/\mathchoice{{\mathbb{E}}\left[f_{2}({\bf X})\right]}{{\mathbb{E}}[f_{2}({\bf X})]}{{\mathbb{E}}[f_{2}({\bf X})]}{{\mathbb{E}}[f_{2}({\bf X})]}$. Now we have

[TABLE]

Thus we have

[TABLE]

which allows us to consider the numerator and denominator separately. We begin by proving the equality that the above algorithm was designed to satisfy:

Lemma 4.5.5**.**

$\mathchoice{{\mathbb{E}}\left[\Psi_{\!q}(\phi(U)\,{\bf g})~\Psi_{\!p^{*}}(\psi(V)\,{\bf g})^{T}\right]}{{\mathbb{E}}[\Psi_{\!q}(\phi(U)\,{\bf g})~\Psi_{\!p^{*}}(\psi(V)\,{\bf g})^{T}]}{{\mathbb{E}}[\Psi_{\!q}(\phi(U)\,{\bf g})~\Psi_{\!p^{*}}(\psi(V)\,{\bf g})^{T}]}{{\mathbb{E}}[\Psi_{\!q}(\phi(U)\,{\bf g})~\Psi_{\!p^{*}}(\psi(V)\,{\bf g})^{T}]}~=~c_{a,b}\cdot(\widetilde{U}\widetilde{V}^{T})$**

• **Proof:  **

[TABLE]

It remains to upper bound the denominator which we do using a straightforward convexity argument.

Lemma 4.5.6**.**

$\mathchoice{{\mathbb{E}}\left[{\lVert\phi(U)\,{\bf g}\rVert^{b}_{q}}\cdot{\lVert\psi(V)\,{\bf g}\rVert^{a}_{p^{*}}}\right]}{{\mathbb{E}}[{\lVert\phi(U)\,{\bf g}\rVert^{b}_{q}}\cdot{\lVert\psi(V)\,{\bf g}\rVert^{a}_{p^{*}}}]}{{\mathbb{E}}[{\lVert\phi(U)\,{\bf g}\rVert^{b}_{q}}\cdot{\lVert\psi(V)\,{\bf g}\rVert^{a}_{p^{*}}}]}{{\mathbb{E}}[{\lVert\phi(U)\,{\bf g}\rVert^{b}_{q}}\cdot{\lVert\psi(V)\,{\bf g}\rVert^{a}_{p^{*}}}]}\leq\gamma_{p^{*}}^{a}\,\gamma_{q}^{b}\,.$**

• **Proof:  **

[TABLE]

We are now ready to prove our approximation guarantee.

Lemma 4.5.7**.**

Consider any $1\leq q\leq 2\leq p\leq\infty$. Then,

[TABLE]

• **Proof:  **

[TABLE]

We next begin the primary technical undertaking for the analysis of our algorithm, namely proving upper bounds on $h^{-1}_{p,\,q}(1)$.

4.6 Hypergeometric Representation of $f_{a,\,b}(x)$

In this section, we show that $f_{a,\,b}(\rho)$ can be represented using the Gaussian hypergeometric function ${}_{2}F_{1}$. The result of this section can be thought of as a generalization of the so-called Grothendieck identity for hyperplane rounding which simply states that

[TABLE]

We believe the result of this section and its proof technique to be of independent interest in analyzing generalizations of hyperplane rounding to convex bodies other than the hypercube.

Recall that $\widetilde{f}_{a,\,b}(\rho)$ is defined as follows:

[TABLE]

where $a=p^{*}-1$ and $b=q-1$. Our starting point is the simple observation that the above expectation can be viewed as the noise correlation (under the Gaussian measure) of the functions $\widetilde{f}^{\,(a)}(\tau):=\operatorname{\operatorname{sgn}}{\tau}\cdot|\tau|^{a}$ and $\widetilde{f}^{\,(b)}(\tau):=\operatorname{\operatorname{sgn}}{\tau}\cdot|\tau|^{b}$. Elementary Hermite analysis then implies that it suffices to understand the Hermite coefficients of $\widetilde{f}^{\,(a)}$ and $\widetilde{f}^{\,(b)}$ individually, in order to understand the Taylor coefficients of $f_{a,\,b}$. To understand the Hermite coefficients of $\widetilde{f}^{\,(a)}$ and $\widetilde{f}^{\,(b)}$ individually, we use a generating function approach. More specifically, we derive an integral representation for the generating function of the (appropriately normalized) Hermite coefficients which fortunately turns out to be closely related to a well studied special function called the parabolic cylinder function.

We can write: $\widetilde{f}_{a,\,b}(\rho)=\mathchoice{\underset{{\bf g}_{1}\sim\rho\,{\bf g}_{2}}{\mathbb{E}}\left[\widetilde{f}^{\,(a)}({\bf g}_{1})\cdot\widetilde{f}^{\,(b)}({\bf g}_{2})\right]}{{\mathbb{E}}_{{\bf g}_{1}\sim\rho\,{\bf g}_{2}}[\widetilde{f}^{\,(a)}({\bf g}_{1})\cdot\widetilde{f}^{\,(b)}({\bf g}_{2})]}{{\mathbb{E}}_{{\bf g}_{1}\sim\rho\,{\bf g}_{2}}[\widetilde{f}^{\,(a)}({\bf g}_{1})\cdot\widetilde{f}^{\,(b)}({\bf g}_{2})]}{{\mathbb{E}}_{{\bf g}_{1}\sim\rho\,{\bf g}_{2}}[\widetilde{f}^{\,(a)}({\bf g}_{1})\cdot\widetilde{f}^{\,(b)}({\bf g}_{2})]}$, where $\widetilde{f}^{\,(c)}(\tau):=\operatorname{\operatorname{sgn}}(\tau)\cdot\left\lvert\tau\right\rvert^{c}$ for $c\in\{a,b\}$. Now we note that $f_{a,\,b}(\rho)$ is the noise correlation of $\widetilde{f}^{\,(a)}$ and $\widetilde{f}^{\,(b)}$. Thus, we can relate the Taylor coefficients of $f_{a,\,b}(\rho)$, to the Hermite coefficients of $\widetilde{f}^{\,(a)}$ and $\widetilde{f}^{\,(b)}$.

Claim 4.6.1** (Coefficients of $\widetilde{f}_{a,\,b}(\rho)$).**

For $\rho\in[-1,1]$, we have:

[TABLE]

where $\widehat{f}^{\,\,(a)}_{i}$ and $\widehat{f}^{\,\,(b)}_{j}$ are the $i$-th and $j$-th Hermite coefficients of $\widetilde{f}^{\,(a)}$ and $\widetilde{f}^{\,(b)}$, respectively. Moreover, $\widehat{f}^{\,\,(a)}_{2k}=\widehat{f}^{\,\,(b)}_{2k}=0$ for $k\geq 0$.

• **Proof:  ** We observe that both $\widetilde{f}^{\,(a)}$ and $\widetilde{f}^{\,(b)}$ are odd functions and hence ˜4.2.14 implies that $\widehat{f}^{\,\,(a)}_{2k}=\widehat{f}^{\,\,(b)}_{2k}=0$ for all $k\geq 0$ – as $\widetilde{f}^{\,(a)}(\tau)\cdot H_{2k}(\tau)$ is an odd function of $\tau$.

[TABLE]

4.6.1 Hermite Coefficients of $\widetilde{f}^{\,(a)}$ and $\widetilde{f}^{\,(b)}$ via Parabolic Cylinder Functions

In this subsection, we use the generating function of Hermite polynomials to obtain an integral representation for the generating function of the ($\sqrt{k!}$ normalized) odd Hermite coefficients of $\widetilde{f}^{\,(a)}$ (and similarly of $\widetilde{f}^{\,(b)}$) is closely related to a special function called the parabolic cylinder function. We then use known facts about the relation between parabolic cylinder functions and confluent hypergeometric functions, to show that the Hermite coefficients of $\widetilde{f}^{\,(c)}$ can be obtained from the Taylor coefficients of a confluent hypergeometric function.

Generating Function of Hermite Coefficients and its Confluent Hypergeometric Representation

Using the generating function of (appropriately normalized) Hermite polynomials, we derive an integral representation for the generating function of the (appropriately normalized) Hermite coefficients of $\widetilde{f}^{\,(a)}$ (and similarly $\widetilde{f}^{\,(b)}$):

Lemma 4.6.2**.**

For $c\in\{a,b\}$, let $\widehat{f}^{\,\,(c)}_{k}$ denote the $k$-th Hermite coefficient of $\widetilde{f}^{\,(c)}(\tau):=\operatorname{\operatorname{sgn}}{(\tau)}\cdot|\tau|^{c}$. Then we have the following identity:

[TABLE]

• **Proof:  ** We observe that for, $\widetilde{f}^{\,(c)}$ is an odd function and hence ˜4.2.14 implies that $\widetilde{f}^{\,(c)}(\tau)\cdot H_{2k}(\tau)$ is an odd function and $\widetilde{f}^{\,(c)}(\tau)\cdot H_{2k+1}(\tau)$ is an even function. This implies for any $k\geq 0$, that $\widehat{f}^{\,\,(c)}_{2k}=0$ and

[TABLE]

Thus we have

[TABLE]

where the exchange of summation and integral in the second equality follows by Fubini’s theorem. We include this routine verification for the sake of completeness. As a consequence of Fubini’s theorem, if $(f_{k}:{\mathbb{R}}\to{\mathbb{R}})_{k}$ is a sequence of functions such that $\sum_{k\geq 0}\int_{0}^{\infty}|f_{k}|<\infty$, then $\sum_{k\geq 0}\int_{0}^{\infty}f_{k}=\int_{0}^{\infty}\sum_{k\geq 0}f_{k}\,.$ Now for any fixed $k$, we have

[TABLE]

Setting $f_{k}(\tau):=\tau^{c}\cdot\mathrm{e}^{-\tau^{2}/2}\cdot H_{2k+1}(\tau)\cdot\lambda^{2k+1}/\sqrt{(2k+1)!}\,\,,$ we get that $\sum_{k\geq 0}\int_{0}^{\infty}|f_{k}|<\infty$. This completes the proof.

Finally using known results about parabolic cylinder functions, we are able to relate the aforementioned integral representation to a confluent hypergeometric function (whose Taylor coefficients are known).

Lemma 4.6.3**.**

For $\lambda\in[-1,1]$ and real valued $c>-1$, we have

[TABLE]

• **Proof:  ** We prove this by using the Corollary˜4.2.25 with $a=c+\frac{1}{2}$. We note that $\alpha>-\frac{1}{2}$ and ${}_{1}F_{1}\left(\cdot,\cdot,-\lambda^{2}/2\right)$ is an even function of $\lambda$. So combining the two, we get:

[TABLE]

4.6.2 Taylor Coefficients of $\widetilde{f}_{a,\,b}(x)$ and Hypergeometric

Representation

By ˜4.6.1, we are left with understanding the function whose power series is given by a weighted coefficient-wise product of a certain pair of confluent hypergeometric functions. This turns out to be precisely the Gaussian hypergeometric function, as we will see below.

Observation 4.6.4**.**

*Let $f_{k}:=[\tau^{k}]\,{}_{1}F_{1}(a_{1},3/2,\tau)$ and $h_{k}:=[\tau^{k}]\,{}_{1}F_{1}(b_{1},3/2,\tau)$. Further let

$\mu_{k}:=f_{k}\cdot h_{k}\cdot(2k+1)!/4^{k}$. Then for $\rho\in[-1,1]$,*

[TABLE]

• **Proof:  ** The claim is equivalent to showing that $\mu_{k}=(a_{1})_{k}\,(b_{1})_{k}/((3/2)_{k}\,k!)$. Since we have $f_{k}=(a_{1})_{k}/((3/2)_{k}\,k!)$ and $h_{k}=(b_{1})_{k}/((3/2)_{k}\,k!)$, it is sufficient to show that $(2k+1)!/4^{k}=(3/2)_{k}\cdot k!$. Indeed, we have:

[TABLE]

We are finally equipped to put everything together.

Theorem 4.6.5**.**

For any $a,b\in(-1,\infty)$ and $\rho\in[-1,1]$, we have

[TABLE]

It follows that the $(2k+1)$-th Taylor coefficient of $f_{a,\,b}(\rho)$ is

[TABLE]

• **Proof:  ** The claim follows by combining ˜4.6.1, Lemmas˜4.6.2 and 4.6.3, and ˜4.6.4.

This hypergeometric representation immediately yields some non-trivial coefficient and monotonicity properties:

Corollary 4.6.6**.**

For any $a,b\in[0,1]$, the function $f_{a,\,b}:[-1,1]\to{\mathbb{R}}$ satisfies

1. (M1)

$[\rho]\,f_{a,\,b}(\rho)=1$*  and  $[\rho^{3}]\,f_{a,\,b}(\rho)=(1-a)(1-b)/6$.* 2. (M2)

All Taylor coefficients are non-negative. Thus, $f_{a,\,b}(\rho)$ is increasing on $[-1,1]$. 3. (M3)

All Taylor coefficients are decreasing in $a$ and in $b$. Thus, for any fixed $\rho\in[-1,1]$,  $f_{a,\,b}(\rho)$ is decreasing in $a$ and in $b$. 4. (M4)

Note that $f_{a,\,b}(0)=0$ and by (M1) and (M2), $f_{a,\,b}(1)\geq 1$. By continuity, $f_{a,\,b}([0,1])$ contains $[0,1]$. Combining this with (M3) implies that for any fixed $\rho\in[0,1]$, $f^{-1}_{a,\,b}(\rho)$ is increasing in $a$ and in $b$.

4.7 $\sinh^{-1}(1)/(1+\varepsilon_{0})$  Bound on $h^{-1}_{a,\,b}(1)$

In this section we show that $p=\infty,q=1$ (the Grothendieck case) is roughly the extremal case for the value of $h^{-1}_{a,\,b}(1)$, i.e., we show that for any $1\leq q\leq 2\leq p\leq\infty$, $h^{-1}_{a,\,b}(1)\geq\sinh^{-1}(1)/(1+\varepsilon_{0})$ (recall that $h^{-1}_{0,\,0}(1)=\sinh^{-1}(1)$). While we were unable to establish as much, we conjecture that $h^{-1}_{a,\,b}(1)\geq\sinh^{-1}(1)$. Section˜4.7.1 details some of the challenges involved in establishing that $\sinh^{-1}(1)$ is the worst case, and presents our approach to establish an approximate bound, which will be formally proved in Section˜4.7.2.

4.7.1 Behavior of The Coefficients of $f^{-1}_{a,\,b}(z)$.

Krivine’s upper bound on the real Grothendieck constant, Haagerup’s upper bound [Haa81] on the complex Grothendieck constant and the work of Naor and Regev [NR14, BdOFV14] on the optimality of Krivine schemes are all closely related to our work in that each of the aforementioned papers needs to lower bound $(\mathrm{abs}\left(f^{-1}\right))^{-1}(1)$ for an appropriate odd function $f$ (the work of Briet et al. [BdOFV14] on the rank-constrained Grothendieck problem is also a generalization of Krivine’s and Haagerup’s work, however they did not derive a closed form upper bound on $(\mathrm{abs}\left(f^{-1}\right))^{-1}(1)$ in their setting). In Krivine’s setting $f=\sin^{-1}x$, implying $(\mathrm{abs}\left(f^{-1}\right))^{-1}=\sinh^{-1}$ and hence the bound is immediate. In our setting, as well as in citeHaagerup81 and [NR14, BdOFV14], $f$ is given by its Taylor coefficients and is not known to have a closed form. In [NR14], all coefficients of $f^{-1}$ after the third are negligible and so one doesn’t incur much loss by assuming that $\mathrm{abs}\left(f^{-1}\right)(\rho)=c_{1}\rho+c_{3}\rho^{3}$. In [Haa81], the coefficient of $\rho$ in $f^{-1}(\rho)$ is $1$ and every subsequent coefficient is negative, which implies that $\mathrm{abs}\left(f^{-1}\right)(\rho)=2\rho-f^{-1}(\rho)$. Note that if the odd coefficients of $f^{-1}$ are alternating in sign like in Krivine’s setting, then $\mathrm{abs}\left(f^{-1}\right)(\rho)=-i\cdot f^{-1}(i\rho)$. These structural properties of the coefficients help their analyses.

In our setting there does not appear to be such a strong relation between $(\mathrm{abs}\left(f^{-1}\right))$ and $f^{-1}$. Consider $f(\rho)=f_{a,\,a}(\rho)$. For certain $a\in(0,1)$, the sign pattern of the coefficients of $f^{-1}$ is unlike that of [Haa81] or $\sin\rho$. In fact empirical results suggest that the odd coefficients of $f$ alternate in sign up to some term $K=K(a)$, and subsequently the coefficients are all non-positive (where $K(a)\to\infty$ as $a\to 0$), i.e., the sign pattern appears to be interpolating between that of $\sin\rho$ and that of $f^{-1}(\rho)$ in the case of Haagerup [Haa81].

Another source of difficulty is that for a fixed $a$, the coefficients of $f^{-1}$ (with and without magnitude) are not necessarily monotone in $k$, and moreover for a fixed $k$, the $k$-th coefficient of $f^{-1}$ is not necessarily monotone in $a$.

A key part of our approach is noting that certain milder assumptions on the coefficients are sufficient to show that $\sinh^{-1}(1)$ is the worst case. The proof crucially uses the monotonicity of $f_{a,\,b}(\rho)$ in $a$ and $b$. The conditions are as follows:

Let $f^{\,-1}_{k}:=[\rho^{k}]\,f^{-1}_{a,\,b}(\rho)$. Then

1. (C1)

$f^{\,-1}_{k}\leq 1/k!$  if  $k\!\pmod{4}\equiv 1$. 2. (C2)

$f^{\,-1}_{k}\leq 0$  if  $k\!\pmod{4}\equiv 3$.

To be more precise, we were unable to establish that the above conditions hold for all $k$ (however we conjecture that it is true for all $k$), and instead use Mathematica to verify it for the fist few coefficients. We additionally show that the coefficients of $f^{-1}_{a,\,b}$ decay exponentially. Combining this exponential decay with a robust version of the previously advertised claim yields that $h^{-1}_{a,\,b}(1)\geq\sinh^{-1}(1)/(1+\varepsilon_{0})$.

We next proceed to prove the claim that the aforementioned conditions are sufficient to show that $\sinh^{-1}(1)$ is the worst case. We will need the following definition. For an odd positive integer $t$, let

[TABLE]

Lemma 4.7.1**.**

If $t$ is an odd integer such that (C1) and (C2) are satisfied for all $k<t$, and $\rho=\sinh^{-1}(1-2h_{err}(t,\delta))$  for some $\delta\geq\rho$, then $h_{a,\,b}(\rho)\leq 1$.

• **Proof:  ** We have,

[TABLE]

Thus, we obtain:

Theorem 4.7.2**.**

For any $1\leq q\leq 2\leq p\leq\infty$, let $a:=p^{*}-1,b=q-1$. Then for any $m,n\in{\mathbb{N}}$ and $A\in{\mathbb{R}}^{m\times n}$,   $\mathsf{CP}(A)/\mathchoice{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}{\lVert A\rVert_{p\to q}}~\leq~1/(h^{-1}_{a,\,b}(1)\cdot\gamma_{q}\,\gamma_{p^{*}})$   and moreover

$h^{-1}_{1,\,b}(1)=h^{-1}_{a,\,1}(1)=1$.

• -

$h^{-1}_{a,\,b}(1)\geq\sinh^{-1}(1)/(1+\varepsilon_{0})$*  where $\varepsilon_{0}=0.00863$.*

• **Proof:  ** The first inequality follows from Lemma˜4.5.7. As for the next item, If $p=2$ or $q=2$  (i.e., $a=1$ or $b=1$) we are trivially done since $h^{-1}_{a,\,b}(\rho)=\rho$ in that case (since for $k\geq 1$,  $(0)_{k}=0$). So we may assume that $a,b\in[0,1)$.

We are left with proving the final part of the claim. Now using Mathematica we verify (exactly)333 We generated $f^{\,-1}_{k}$ as a polynomial in $a$ and $b$ and maximized it over $a,b\in[0,1]$ using the Mathematica “Maximize” function which is exact for polynomials. that $(C1)$ and $(C2)$ are true for $k\leq 29$. Now let $\delta=\sinh^{-1}(0.974203)$. Then by Lemma˜4.7.7 (which states that $f^{\,-1}_{k}$ decays exponentially and will be proven in the subsequent section),

[TABLE]

Now by Lemma˜4.7.1 we know $h^{-1}_{a,\,b}(1)\geq\sinh^{-1}(1-2h_{err}(31,\delta))$. Thus,

$h^{-1}_{a,\,b}(1)\geq\sinh^{-1}(0.974202)\geq\sinh^{-1}(1)/(1+\varepsilon_{0})$ for $\varepsilon_{0}=0.00863$, which completes the proof.

4.7.2 Bounding Inverse Coefficients

In this section we prove that $f^{\,-1}_{k}$ decays as $1/c^{k}$ for some $c=c(a,b)>1$, proving Lemma˜4.7.7. Throughout this section we assume $1\leq p^{*},q<2$, and $a=p^{*}-1,~b=q-1$ (i.e., $a,b\in[0,1)$). Via the power series representation, $f_{a,\,b}(z)$ can be analytically continued to the unit complex disk. Let $f^{-1}_{a,\,b}(z)$ be the inverse of $f_{a,\,b}(z)$ and recall $f^{\,-1}_{k}$ denotes its $k$-th Taylor coefficient.

We begin by stating a standard identity from complex analysis that provides a convenient contour integral representation of the Taylor coefficients of the inverse of a function. We include a proof for completeness.

Lemma 4.7.3** (Inversion Formula).**

There exists $\delta>0$, such that for any odd $k$,

[TABLE]

where $C^{+}_{\delta}$ denotes the first quadrant quarter circle of radius $\delta$ with counter-clockwise orientation.

• **Proof:  ** Via the power series representation, $f_{a,\,b}(z)$ can be analytically continued to the unit complex disk. Thus by inverse function theorem for holomorphic functions, there exists $\delta_{0}\in(0,1]$ such that $f_{a,\,b}(z)$ has an analytic inverse in the open disk $|z|<\delta_{0}$. So for $\delta\in(0,\delta_{0})$, $f_{a,\,b}(C_{\delta})$ is a simple-closed curve with winding number $1$ (where $C_{\delta}$ is the complex circle of radius $\delta$ with the usual counter-clockwise orientation). Thus, by Cauchy’s integral formula we have:

[TABLE]

where the second equality follows from substituting $w=f_{a,\,b}(z)$.

Now by ˜4.7.5,   $z/f_{a,\,b}(z)^{k}$  is holomorphic on the open set $|z|\in(0,1)$, which contains $C_{\delta}$. Hence, by the fundamental theorem of contour integration we have

[TABLE]

So we get,

[TABLE]

where the second equality follows since $f^{\,-1}_{k}$ is purely real. Lastly, we complete the proof of the claim by using the fact that for odd $k$,  $f_{a,\,b}(z)^{-k}$ is odd and that $\overline{f_{a,\,b}(z)}=f_{a,\,b}(\overline{z})$.

We next state a standard bound on the magnitude of a contour integral that we will use in our analysis.

Fact 4.7.4** (ML-inequality).**

If $f$ is a complex valued continuous function on a contour $\Gamma$ and $|f(z)|$ is bounded by $M$ for every $z\in\Gamma$, then

[TABLE]

where $\ell(\Gamma)$ is the length of $\Gamma$.

Unfortunately the integrand in Eq.˜4.17 can be very large for small $\delta$, and we cannot use the ML-inequality as is. To fix this, we modify the contour of integration (using Cauchy’s integral theorem) so that the imaginary part of the integral vanishes when restricted to the sections close to the origin, and the integrand is small in magnitude on the sections far from the origin (thus allowing us to use the ML-inequality). To do this we will need some preliminaries.

$f_{a,\,b}(z)$ is defined on the closed complex unit disk. The domain is analytically extended to the region $\mathcal{C}\setminus((-\infty,-1)\cup(1,\infty))$, using the Euler-type integral representation of the hypergeometric function.

[TABLE]

where $\mathrm{B}(\tau_{1},\tau_{2})$ is the beta function and

[TABLE]

Fact 4.7.5**.**

For any $a_{1}>0$, ${}_{2}F_{1}(a_{1},b_{1},c_{1},z)$ has no non-zero roots in the region $\mathcal{C}\setminus(1,\infty)$. This implies that if $p^{*}<2$,  $f^{+}_{a,\,b}(z)$ has no non-zero roots in the region $\mathcal{C}\setminus((-\infty,-1)\cup(1,\infty))$.

We are now equipped to expand the contour. Our choice of contour is inspired by that of Haagerup [Haa81] which he used in deriving an upper bound on the complex Grothendieck constant. The contour we choose has some differences for technical reasons related to the region to which hypergeometric functions can be analytically extended. The analysis is quite different from that of Haagerup since the functions in consideration behave differently. In fact the inverse function Haagerup considers has polynomially decaying coefficients while the class of inverse functions we consider have coefficients that have decay between exponential and factorial.

Observation 4.7.6** (Expanding Contour).**

For any $\alpha\geq 1$ and $\varepsilon>0$, let $P(\alpha,\varepsilon)$ be the four-part curve (see LABEL:contour) given by

the line segment $\delta~\rightarrow~(1-\varepsilon)$,

• -

the line segment $(1-\varepsilon)~\rightarrow~(\sqrt{\alpha-\varepsilon}+i\sqrt{\varepsilon})$ (henceforth referred to as $L_{\alpha,\varepsilon}$),

• -

the arc along $C^{+}_{\alpha}$ starting at $(\sqrt{\alpha-\varepsilon}+i\sqrt{\varepsilon})$ and ending at $i\alpha$ (henceforth referred to as $C^{+}_{\alpha,\varepsilon}$),

• -

the line segment $i\alpha~\rightarrow~i\delta$.

By Cauchy’s integral theorem, combining Lemma˜4.7.3 with ˜4.7.5 yields that for odd $k$,

[TABLE]

We will next see that the imaginary part of our contour integral vanishes on section of $P(\alpha,\varepsilon)$. Applying ML-inequality to the remainder of the contour, combined with lower bounds on $|f^{+}_{a,\,b}(z)|$  (proved below the fold in Section˜4.7.2), allows us to derive an exponentially decaying upper bound on $|f^{\,-1}_{k}|$.

Lemma 4.7.7**.**

For any $1\leq p^{*},q<2$, there exists $\varepsilon>0$ such that

[TABLE]

• **Proof:  ** For a contour $P$, we define $V(P)$ as

[TABLE]

As is evident from the integral representation, $f^{+}_{a,\,b}(z)$ is purely imaginary if $z$ is purely imaginary, and as is evident from the power series, $f_{a,\,b}(z)$ is purely real if $z$ lies on the real interval $[-1,1]$. This implies that $V(\delta\rightarrow(1-\varepsilon))=V(i\alpha\rightarrow i\delta)=0$.

Now combining ˜4.7.4 (ML-inequality) with Lemma˜4.7.9 and Lemma˜4.7.12 (which state that the integrand is small in magnitude over $C^{+}_{6,\varepsilon}$ and $L_{6,\varepsilon}$ respectively), we get that for sufficiently small $\varepsilon>0$,

[TABLE]

Lower bounds on $|f^{+}_{a,\,b}(z)|$ Over $C^{+}_{\alpha,\varepsilon}$ and $L_{\alpha,\varepsilon}$

In this section we show that for sufficiently small $\varepsilon$, $|f^{+}_{a,\,b}(z)|>1$ over $L_{\alpha,\varepsilon}$ (regardless of the value of $\alpha$, Lemma 4.7.12), and over $C^{+}_{\alpha,\varepsilon}$ when $\alpha$ is a sufficiently large constant (Lemma 4.7.9).

We will first show the claim for $C^{+}_{\alpha,\varepsilon}$ by relating $|f^{+}_{a,\,b}(z)|$ to $|z|$. While the asymptotic behavior of hypergeometric functions for $|z|\to\infty$ has been extensively studied (see for instance [Loz03]), it appears that our desired estimates aren’t immediate consequences of prior work for two reasons. Firstly, we require relatively precise estimates for moderately large but constant $|z|$. Secondly, due to the expressive power of hypergeometric functions, the estimates we derive can only be true for hypergeometric functions parameterized in a specific range. Indeed, our proof crucially uses the fact that $a,b\in[0,1)$. Our approach is to use the Euler-type integral representation of $f^{+}_{a,\,b}(z)$ which as a reminder to the reader is as follows:

[TABLE]

where $\mathrm{B}(x,y)$ is the beta function and

[TABLE]

We start by making the simple observation that the integrand of $\mathrm{I}(z)$ is always in the positive complex quadrant — an observation that will come in handy multiple times in this section, in dismissing the possibility of cancellations. This is the part of our proof that makes the most crucial use of the assumption that $0\leq a<1$ (equivalently $1\leq p^{*}<2$).

Observation 4.7.8**.**

Let $z=re^{i\theta}$ be such that either one of the following two cases is satisfied:

1. (A)

$r<1$* and $\theta=0$.* 2. (B)

$\theta\in(0,\pi/2]$.

Then for any $0\leq a\leq 1$ and any $t\in{\mathbb{R}}^{+}$,

[TABLE]

• **Proof:  ** The claim is clearly true when $\theta=0$ and $r<1$. It is also clearly true when $\theta=\pi/2$. Thus we may assume $\theta\in(0,\pi/2)$.

[TABLE]

Moreover since $\arg(-tz^{2})=2\theta-\pi\in(-\pi,0)$, we have $\arg(1-tz^{2})>2\theta-\pi$. Thus we have,

[TABLE]

We now show $|f^{+}_{a,\,b}(z)|$ is large over $C^{+}_{\alpha,\varepsilon}$. The main idea is to move from a complex integral to a real integral with little loss, and then estimate the real integral. To do this, we use ˜4.7.8 to argue that the magnitude of $\mathrm{I}(z)$ is within $\sqrt{2}$ of the integral of the magnitude of the integrand.

Lemma 4.7.9** ( $|f^{+}_{a,\,b}(z)|$ is large over $C^{+}_{\alpha,\varepsilon}$ ).**

Assume $a,b\in[0,1)$ and consider any $z\in\mathcal{C}$ with $|z|\geq 6$. Then $|f^{+}_{a,\,b}(z)|>1$.

• **Proof:  ** We start with a useful substitution.

[TABLE]

where

[TABLE]

We next exploit the observation that the integrand is always in the positive complex quadrant by showing that $|\mathrm{I}(z)|$ is at most a factor of $\sqrt{2}$ away from the integral obtained by replacing the integrand with its magnitude.

[TABLE]

We now break the integral into two parts and analyze them separately. We start by analyzing the part that’s large when $b\rightarrow 0$.

[TABLE]

We now analyze the part that’s large when $b\rightarrow 1$.

[TABLE]

Combining the two estimates above yields that if $r>\sqrt{2}$,

[TABLE]

Lastly, the proof follows by using the following estimate:

Fact 4.7.10**.**

Via Mathematica, for $0\leq b<1$ we have

[TABLE]

Remark 4.7.11**.**

The preceding proof can be used to derive the precise asymptotic behavior of $|f^{+}_{a,\,b}(z)|$ in $r$. Specifically, it grows as  $r^{a}\log r$  if  $a=b$  and as  $r^{\,\max\{a,b\}}$  if  $a\neq b$.

We now show that $|f^{+}_{a,\,b}(z)|>1$ over $L_{\alpha,\varepsilon}$. To do this, it is insufficient to assume that $|z|\geq 1$ since there exist points $z$ (for instance $z=i$) of unit length such that $|f^{+}_{a,\,b}(z)|<1$. To show the claim, we observe that $|f^{+}_{a,\,b}(z)|$ is large when $z$ is close to the real line and use the fact that $L_{\alpha,\varepsilon}$ is close to the real line. Formally, we show that if $z$ is of length at least $1$ and is sufficiently close to the real line, $|f^{+}_{a,\,b}(z)|$ is close to $f^{+}_{a,\,b}(1)$. Lastly, we use the power series representation of the hypergeometric function to obtain a sufficiently accurate lower bound on $f^{+}_{a,\,b}(1)$.

Lemma 4.7.12** ( $|f^{+}_{a,\,b}(z)|$ is large over $L_{\alpha,\varepsilon}$ ).**

Assume $a,b\in[0,1)$ and consider any $\gamma\geq 1-\varepsilon_{1}$. Let $\varepsilon_{2}:=\sqrt{\varepsilon_{1}}$ and $z:=\gamma(1+i\varepsilon_{1})$. Then for $\varepsilon_{1}>0$ sufficiently small, $|f^{+}_{a,\,b}(z)|>1$.

• **Proof:  ** Below the fold we will show

[TABLE]

But we know ($\mathrm{LHS},~\mathrm{RHS}$ refer to Eq.˜4.18)

[TABLE]

Also by Corollary˜4.6.6 : (M1), (M2),  $f_{a,\,b}(1)\geq 1+(1-a)(1-b)/6>1$. Thus for $\varepsilon_{1}$ sufficiently small, we must have $|f^{+}_{a,\,b}(z)|>1$.

We now show Eq.˜4.18, by comparing integrands point-wise. To do this, we will assume the following closeness estimate that we will prove below the fold:

[TABLE]

We will also need the following inequality. Since $\gamma\geq 1-\varepsilon_{1}=1-\varepsilon_{2}^{2}$, for any $0\leq s\leq 1-\varepsilon_{2}$, we have

[TABLE]

Given, these estimates, we can complete the proof of Eq.˜4.18 as follows:

[TABLE]

It remains to establish Eq.˜4.19, which we will do by considering the numerator and reciprocal of the denominator separately and subsequently using the fact that $\Re(z_{1}z_{2})=\Re(z_{1})\Re(z_{2})-\Im(z_{1})\Im(z_{2})$. In doing this, we need to show that the respective real parts are large and respective imaginary parts are small for which the following simple facts will come in handy.

Fact 4.7.13**.**

Let $z=re^{i\theta}$ be such that $\Re{z}\geq 0$  (i.e. $-\pi/2\leq\theta\leq\pi/2$). Then for any $0\leq\alpha\leq 1$,

[TABLE]

Fact 4.7.14**.**

Let $z=re^{-i\theta}$ be such that $\Re{z}\geq 0,\Im{z}\leq 0$  (i.e. $0\leq\theta\leq\pi/2$). Then for any $0\leq\alpha\leq 1$,

[TABLE]

We are now ready to prove the claimed properties of the reciprocal of the denominator from Eq.˜4.19. For any $0\leq s\leq 1-\varepsilon_{2}$ we have,

[TABLE]

Similarly,

[TABLE]

Combining Section˜4.7.2 and Section˜4.7.2 with the fact that $\Re(z_{1}z_{2})=\Re(z_{1})\Re(z_{2})-\Im(z_{1})\Im(z_{2})$ yields,

[TABLE]

This completes the proof.

Challenges of Proving (C1) and (C2) for all $k$

For certain values of $a$ and $b$, the inequalities in (C1) and (C2) leave very little room for error. In particular, when $a=b=0$, (C1) holds at equality and (C2) has $1/k!$ additive slack. In this special case, it would mean that one cannot analyze the contour integral (for the $k$-th coefficient of $f^{-1}_{a,\,b}(\rho)$) by using ML-inequality on any section of the contour that is within a distance of $\mathrm{exp}(k)$ from the origin. Analytic approaches would require extremely precise estimates on the value of the contour integral on parts close to the origin. Other challenges to naive approaches come from the lack of monotonicity properties for $f^{\,-1}_{k}$ (both in $k$ and in $a,b$ - see Section˜4.7.1)

4.8 Factorization of Linear Operators

Let $X,Y,E$ be Banach spaces and let $A:X\to Y$ be a continuous linear operator. We say that $A$ factorizes through $E$ if there exist continuous operators $C:X\to E$ and $B:E\to Y$ such that $A=BC$. Factorization theory has been a major topic of study in functional analysis, going as far back as Grothendieck’s famous “Resume” [Gro56]. It has many striking applications, like the isomorphic characterization of Hilbert spaces and $L_{p}$ spaces due to Kwapień [Kwa72a, Kwa72b], connections to type and cotype through the work of Kwapień [Kwa72a], Rosenthal [Ros73], Maurey [Mau74] and Pisier [Pis80], connections to Sidon sets through the work of Pisier [Pis86], characterization of weakly compact operators due to Davis et al. [DFJP74], connections to the theory of $p$-summing operators through the work of Grothendieck [Gro56], Pietsch [Pie67] and Lindenstrauss and Pelczynski [LP68].

Let $\Phi(A)$ denote

[TABLE]

where the infimum runs over all Hilbert spaces $H$. We say $A$ factorizes through a Hilbert space if $\Phi(A)<\infty$. Further, let

[TABLE]

where the supremum runs over continuous operators $A:X\to Y$. As a quick example of the power of factorization theorems, observe that if $\mathrm{I}:X\to X$ is the identity operator on a Banach space $X$ and $\Phi(\mathrm{I})<\infty$, then $X$ is isomorphic to a Hilbert space and moreover the distortion (Banach-Mazur distance) is at most $\Phi(\mathrm{I})$ (i.e., there exists an invertible operator $T:X\to H$ for some Hilbert space $H$ such that $\mathchoice{\lVert T\rVert_{X\to H}}{\lVert T\rVert_{X\to H}}{\lVert T\rVert_{X\to H}}{\lVert T\rVert_{X\to H}}\cdot\mathchoice{\lVert T^{-1}\rVert_{H\to X}}{\lVert T^{-1}\rVert_{H\to X}}{\lVert T^{-1}\rVert_{H\to X}}{\lVert T^{-1}\rVert_{H\to X}}\leq\Phi(\mathrm{I})$). In fact (as observed by Maurey), Kwapień gave an isomorphic characterization of Hilbert spaces by proving a factorization theorem.

In this section we will show that our approximation results imply improved bounds on $\Phi(\ell_{p}^{n},\ell_{q}^{m})$ for certain values of $p$ and $q$. Before doing so, we first summarize prior work which will require the definitions of type and cotype:

Definition 4.8.1**.**

The Type-2 constant of a Banach space $X$, denoted by $T_{2}(X)$, is the smallest constant $C$ such that for every finite sequence of vectors $\{x^{i}\}$ in $X$,

[TABLE]

where $\varepsilon_{i}$ is an independent Rademacher random variable. We say $X$ is of Type-2 if $T_{2}(X)<\infty$.

Definition 4.8.2**.**

The Cotype-2 constant of a Banach space $X$, denoted by $C_{2}(X)$, is the smallest constant $C$ such that for every finite sequence of vectors $\{x^{i}\}$ in $X$,

[TABLE]

where $\varepsilon_{i}$ is an independent Rademacher random variable. We say $X$ is of Cotype-2 if $C_{2}(X)<\infty$.

Remark 4.8.3**.**

It is known that $C_{2}(X^{*})\leq T_{2}(X)$.

• -

It is known that for $p\geq 2$, we have $T_{2}(\ell_{p}^{n})=\gamma_{p}$ (while $C_{2}(\ell_{p}^{n})\to\infty$  as  $n\to\infty$) and for $q\leq 2$,  $C_{2}(\ell_{q}^{n})=\max\{2^{1/q-1/2},1/\gamma_{q}\}$ (while  $T_{2}(\ell_{q}^{n})\to\infty$  as  $n\to\infty$).

We say $X$ is Type-2 (resp. Cotype-2) if $T_{2}(X)<\infty$ (resp. $C_{2}(X)<\infty$). $T_{2}(X)$ and $C_{2}(X)$ can be regarded as measures of the “closeness” of $X$ to a Hilbert space. Some notable manifestations of this correspondence are:

$T_{2}(X)=C_{2}(X)=1$ if and only if $X$ is isometric to a Hilbert space.

• -

Kwapień [Kwa72a]:  $X$ is of Type-2 and Cotype-2 if and only if it is isomorphic to a Hilbert space.

• -

Figiel, Lindenstrauss and Milman [FLM77]:  If $X$ is a Banach space of Cotype-2, then any $n$-dimensional subspace of $X$ has an $m=\Omega(n)$-dimensional subspace with Banach-Mazur distance at most $2$ from $\ell_{2}^{m}$.

Maurey observed that a more general factorization result underlies Kwapień’s work:

Theorem 4.8.4** (Kwapień-Maurey).**

Let $X$ be a Banach space of Type-2 and $Y$ be a Banach space of Cotype-2. Then any operator $T:X\to Y$ factorizes through a Hilbert space. Moreover, $\Phi(X,Y)\leq T_{2}(X)C_{2}(Y)$.

Surprisingly Grothendieck’s work which predates the work of Kwapień and Maurey, established that $\Phi(\ell_{\infty}^{n},\ell_{1}^{m})\leq K_{G}$ for all $m,n\in{\mathbb{N}}$, which is not implied by the above theorem since $T_{2}(\ell_{\infty}^{n})\to\infty$ as $n\to\infty$. Pisier [Pis80] unified the above results for the case of approximable operators by proving the following:

Theorem 4.8.5** (Pisier).**

*Let $X,Y$ be Banach spaces such that $X^{*},Y$ are of Cotype-2. Then any approximable operator $T:X\to Y$ factorizes through a Hilbert space. Moreover,

$\Phi(T)\leq(2\,C_{2}(X^{*})C_{2}(Y))^{3/2}$.*

In the next section we show that for any $p^{*},q\in[1,2]$, any $m,n\in{\mathbb{N}}$

[TABLE]

which improves upon Pisier’s bound and for certain ranges of $(p,q)$, improves upon $K_{G}$ as well as the bound of Kwapień-Maurey.

4.8.1 Integrality Gap Implies Factorization Upper Bound

Known upper bounds on $\Phi(X,Y)$ involve Hahn-Banach separation arguments. In this section we see that for a special class of Banach spaces admitting a convex programming relaxation, $\Phi(X,Y)$ is bounded by the integrality gap of the relaxation as an immediate consequence of Convex programming duality (which of course uses a separation argument under the hood). A very similar observation had already been made by Tropp [Tro09] in the special case of $X=\ell_{\infty}^{n},Y=\ell_{1}^{m}$ with a slightly different convex program.

We start by restating the relaxation in a more general setup, and stating its dual. To this end, let $\mathcal{F}_{X},\subset{\mathbb{R}}^{n},~\mathcal{F}_{Y}\subset{\mathbb{R}}^{m}$ be convex sets. Also let

[TABLE]

Given an input matrix $A\in{\mathbb{R}}^{m\times n}$, we shall give a convex programming relaxation for the following problem:

[TABLE]

The relaxation $\mathsf{CP}(A)$ (due to Nesterov et al. [NWY00]) is as follows:

[TABLE]

For a vector $s$, let $D_{s}$ denote the diagonal matrix with $s$ as diagonal entries. Let

[TABLE]

The dual program $\mathrm{DP}(A)$ is as follows:

[TABLE]

Strong duality is satisfied, i.e. $\mathrm{DP}(A)=\mathsf{CP}(A)$, and a proof can be found in Theorem 13.2.3 of [NWY00]. Assume $\sqrt{\mathcal{F}}_{X}$ and $\sqrt{\mathcal{F}}_{Y}$ are convex and let $\mathchoice{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}$ and $\mathchoice{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}}}$ respectively denote the norms they induce. For Banach spaces $X$ over ${\mathbb{R}}^{n}$, $Y$ over ${\mathbb{R}}^{m}$ and an operator $A:X\to Y$, we define

[TABLE]

where the infimum runs over diagonal matrices $D_{1},D_{2}$ and $B\in{\mathbb{R}}^{m\times n}$. Clearly, $\Phi(A)\leq\Phi_{3}(A)$ and therefore $\Phi(X,Y)\leq\Phi_{3}(X,Y)$.

Henceforth, we fix $X$ and $Y$ to be the Banach spaces $({\mathbb{R}}^{n},\mathchoice{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}})$ and $({\mathbb{R}}^{m},\mathchoice{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}^{\,\,\ast}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}^{\,\,\ast}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}^{\,\,\ast}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}^{\,\,\ast}}})$ respectively. As was the approach of Grothendieck, we give an upper bound on $\Phi(X,Y)$ by giving an upper bound on $\Phi_{3}(X,Y)$. We do this by showing

Lemma 4.8.6**.**

For any $A:X\to Y$,    $\Phi_{3}(A)~\leq~\mathrm{DP}(A)/\mathchoice{\lVert A\rVert_{\sqrt{\mathcal{F}}_{X}\to\sqrt{\mathcal{F}}_{Y}^{\,*}}}{\lVert A\rVert_{\sqrt{\mathcal{F}}_{X}\to\sqrt{\mathcal{F}}_{Y}^{\,*}}}{\lVert A\rVert_{\sqrt{\mathcal{F}}_{X}\to\sqrt{\mathcal{F}}_{Y}^{\,*}}}{\lVert A\rVert_{\sqrt{\mathcal{F}}_{X}\to\sqrt{\mathcal{F}}_{Y}^{\,*}}}$.

• **Proof:  ** Consider an optimal solution to $\mathrm{DP}(A)$. We will show

[TABLE]

by taking $D_{1}:=D_{s}^{1/2}$,  $D_{2}:=D_{t}^{1/2}$ and $B:=\left(D_{s}^{1/2}\right)^{\dagger}A\left(D_{t}^{1/2}\right)^{\dagger}$ (where for a diagonal matrix $D$, $D^{\dagger}$ only inverts the non-zero diagonal entries and zero-entries remain the same). Note that $s_{i}=0$ (resp. $t_{i}=0$) implies the $i$-th row (resp. $i$-th column) of $A$ is all zeroes, since otherwise one can find a $2\times 2$ principal submatrix (of the block matrix in the relaxation) that is not PSD. This implies that $D_{1}BD_{2}=A$.

It remains to show that $\mathchoice{\lVert D_{2}\rVert_{X\to 2}}{\lVert D_{2}\rVert_{X\to 2}}{\lVert D_{2}\rVert_{X\to 2}}{\lVert D_{2}\rVert_{X\to 2}}\cdot\mathchoice{\lVert B\rVert_{2\to 2}}{\lVert B\rVert_{2\to 2}}{\lVert B\rVert_{2\to 2}}{\lVert B\rVert_{2\to 2}}\cdot\mathchoice{\lVert D_{1}\rVert_{2\to Y}}{\lVert D_{1}\rVert_{2\to Y}}{\lVert D_{1}\rVert_{2\to Y}}{\lVert D_{1}\rVert_{2\to Y}}\leq~\mathrm{DP}(A)$. Now we have,

[TABLE]

Similarly, since  $\mathchoice{\lVert D_{1}\rVert_{2\to Y}}{\lVert D_{1}\rVert_{2\to Y}}{\lVert D_{1}\rVert_{2\to Y}}{\lVert D_{1}\rVert_{2\to Y}}=\mathchoice{\lVert D_{1}\rVert_{Y^{*}\to 2}}{\lVert D_{1}\rVert_{Y^{*}\to 2}}{\lVert D_{1}\rVert_{Y^{*}\to 2}}{\lVert D_{1}\rVert_{Y^{*}\to 2}}$  we have

[TABLE]

Thus it suffices to show $\mathchoice{\lVert B\rVert_{2\to 2}}{\lVert B\rVert_{2\to 2}}{\lVert B\rVert_{2\to 2}}{\lVert B\rVert_{2\to 2}}\leq 1$ since

[TABLE]

We have,

[TABLE]

4.8.2 Improved Factorization Bounds for Certain $\ell_{p}^{n},\ell_{q}^{m}$

Let $1\leq q\leq 2\leq p\leq\infty$. Then taking $\mathcal{F}_{X}$ to be the $\ell_{p/2}^{n}$ unit ball and $\mathcal{F}_{Y}$ to be the $\ell_{q^{*}/2}^{m}$ unit ball, we have $\sqrt{\mathcal{F}}_{X}$ and $\sqrt{\mathcal{F}}_{Y}$ are respectively the unit balls in $\ell_{p}^{n}$ and $\ell_{q^{*}}^{m}$. Therefore, $X$ and $Y$ as defined above are the spaces $\ell_{p}^{n}$ and $\ell_{q}^{m}$ respectively. Hence, we obtain

Theorem 4.8.7** ($\ell_{p}^{n}\to\ell_{q}^{m}$ factorization).**

If $1\leq q\leq 2\leq p\leq\infty$, then for any $m,n\in{\mathbb{N}}$ and $\varepsilon_{0}=0.00863$,

[TABLE]

.

This improves upon Pisier’s bound and for a certain range of $(p,q)$, improves upon $K_{G}$ as well as the bound of Kwapień-Maurey.

On a slightly unrelated note, straightforward observations imply that the integrality gap of $\mathsf{CP}(A)$ for any pair of convex sets $\mathcal{F}_{X},\mathcal{F}_{Y}$ is $K_{G}$ (Grothendieck’s constant). This provides a class of Banach space pairs for which $K_{G}$ is an upper bound on the factorization constant, and it would be interesting to get a better understanding of how this class compares to that of Pisier. We include a proof of this in the next section.

4.8.3 $K_{G}$ Bound on Approximation Ratio

In this subsection, we prove that for any pair of convex sets $\mathcal{F}_{X}$ and $\mathcal{F}_{Y}$ such that $\sqrt{\mathcal{F}}_{X}$ and $\sqrt{\mathcal{F}}_{Y}$ are convex, the approximation ratio is bounded by $K_{G}$. As in the previous section, we fix $X$ and $Y$ to be the Banach spaces $({\mathbb{R}}^{n},\mathchoice{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{X}}})$ and $({\mathbb{R}}^{m},\mathchoice{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}^{\,\,\ast}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}^{\,\,\ast}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}^{\,\,\ast}}}{\lVert\cdot\rVert_{\sqrt{\mathcal{F}}_{Y}^{\,\,\ast}}})$ respectively.

Lemma 4.8.8**.**

For any $A:X\to Y$,    $\mathsf{CP}(A)/\mathchoice{\lVert A\rVert_{X\to Y}}{\lVert A\rVert_{X\to Y}}{\lVert A\rVert_{X\to Y}}{\lVert A\rVert_{X\to Y}}~\leq~K_{G}$.

• **Proof:  ** Let B:=\frac{1}{2}\left[\begin{array}[]{cc}0&A\\ A^{T}&0\end{array}\right]. The main intuition of the proof is to decompose $x\in\sqrt{\mathcal{F}}_{X}$ as $x=|[x]|\circ\operatorname{\operatorname{sgn}}{[x]}$ (where $\circ$ denotes Hadamard/entry-wise multiplication), and then use Grothendieck’s inequality on $\operatorname{\operatorname{sgn}}{[x]}$ and $\operatorname{\operatorname{sgn}}{[y]}$. Another simple observation is that for any convex set $\mathcal{F}$, the feasible set we optimize over is invariant under factoring out the magnitudes of the diagonal entries. In other words,

[TABLE]

We will apply the above fact for $\mathcal{F}=\mathcal{F}_{X}\oplus\mathcal{F}_{Y}$. Let $\sqrt{\mathcal{F}}_{X}^{+}$ denote $\sqrt{\mathcal{F}}_{X}\cap{\mathbb{R}}_{\geq 0}^{n}$ (analogous for $\sqrt{\mathcal{F}}_{Y}^{+}$). Now simple algebraic manipulations yield

[TABLE]