SoS Degree Reduction with Applications to Clustering and Robust Moment Estimation

TL;DR

This paper introduces a framework to reduce the degree of sum-of-squares proofs, enabling faster algorithms for clustering and robust moment estimation without sacrificing statistical guarantees.

Contribution

The authors develop a general degree reduction framework for sum-of-squares proofs and demonstrate its effectiveness in speeding up algorithms for clustering and robust moment estimation.

Findings

Algorithms run in d^{O(ell)} time, improving over previous d^{O(ell^2)}.

Maintains statistical guarantees of prior algorithms.

Significantly faster for high-dimensional data with large moments.

Abstract

We develop a general framework to significantly reduce the degree of sum-of-squares proofs by introducing new variables. To illustrate the power of this framework, we use it to speed up previous algorithms based on sum-of-squares for two important estimation problems, clustering and robust moment estimation. The resulting algorithms offer the same statistical guarantees as the previous best algorithms but have significantly faster running times. Roughly speaking, given a sample of $n$ points in dimension $d$, our algorithms can exploit order-$\ell$ moments in time $d^{O(\ell)}\cdot n^{O(1)}$, whereas a naive implementation requires time $(d\cdot n)^{O(\ell)}$. Since for the aforementioned applications, the typical sample size is $d^{\Theta(\ell)}$, our framework improves running times from $d^{O(\ell^2)}$ to $d^{O(\ell)}$.