The power of sum-of-squares for detecting hidden structures

TL;DR

This paper demonstrates that spectral algorithms based on low-degree polynomial matrices can match the power of sum-of-squares semidefinite programs for a wide range of planted problems, providing new insights and bounds.

Contribution

It establishes the equivalence in power between low-degree polynomial eigenvalues and SoS algorithms for many planted problems, and derives new lower bounds for tensor and sparse PCA.

Findings

Eigenvalues of degree-d polynomial matrices match SoS guarantees for many problems.

New nearly-tight SoS lower bounds for tensor and sparse PCA.

First evidence that surpassing current algorithms for sparse PCA may require exponential time.

Abstract

We study planted problems---finding hidden structures in random noisy inputs---through the lens of the sum-of-squares semidefinite programming hierarchy (SoS). This family of powerful semidefinite programs has recently yielded many new algorithms for planted problems, often achieving the best known polynomial-time guarantees in terms of accuracy of recovered solutions and robustness to noise. One theme in recent work is the design of spectral algorithms which match the guarantees of SoS algorithms for planted problems. Classical spectral algorithms are often unable to accomplish this: the twist in these new spectral algorithms is the use of spectral structure of matrices whose entries are low-degree polynomials of the input variables. We prove that for a wide class of planted problems, including refuting random constraint satisfaction problems, tensor and sparse PCA, densest-k-subgraph, community detection in stochastic block models, planted clique, and others, eigenvalues of degree-d matrix polynomials are as powerful as SoS semidefinite programs of roughly degree d. For such problems it is therefore always possible to match the guarantees of SoS without solving a large semidefinite program. Using related ideas on SoS algorithms and low-degree matrix polynomials (and inspired by recent work on SoS and the planted clique problem by Barak et al.), we prove new nearly-tight SoS lower bounds for the tensor and sparse principal component analysis problems. Our lower bounds for sparse principal component analysis are the first to suggest that going beyond existing algorithms for this problem may require sub-exponential time.