Finding planted cliques using gradient descent

TL;DR

This paper demonstrates that gradient descent and related Hamiltonian-minimizing Markov chains can recover planted cliques of size proportional to the square root of the number of nodes when initialized from the full graph, but not from the empty set.

Contribution

It introduces a Hamiltonian-based optimization approach that succeeds in finding large planted cliques using gradient descent, addressing a gap in black-box optimization methods.

Findings

Gradient descent recovers planted cliques of size  \,  \,  n when initialized from the full graph.

Initialization from the empty set does not enable finding sub-linear planted cliques.

The approach is robust under a natural contamination model.

Abstract

The planted clique problem is a paradigmatic model of statistical-to-computational gaps: the planted clique is information-theoretically detectable if its size $k\ge 2\log_2 n$ but polynomial-time algorithms only exist for the recovery task when $k= \Omega(\sqrt{n})$. By now, there are many algorithms that succeed as soon as $k = \Omega(\sqrt{n})$. Glaringly, however, no black-box optimization method, e.g., gradient descent or the Metropolis process, has been shown to work. In fact, Chen, Mossel, and Zadik recently showed that any Metropolis process whose state space is the set of cliques fails to find any sub-linear sized planted clique in polynomial time if initialized naturally from the empty set. We show that using the method of Lagrange multipliers, namely optimizing the Hamiltonian given by the sum of the objective function and the clique constraint over the space of all subgraphs, succeeds. In particular, we prove that Markov chains which minimize this Hamiltonian (gradient descent and a low-temperature relaxation of it) succeed at recovering planted cliques of size $k = \Omega(\sqrt{n})$ if initialized from the full graph. Importantly, initialized from the empty set, the relaxation still does not help the gradient descent find sub-linear planted cliques. We also demonstrate robustness of these Markov chain approaches under a natural contamination model.