Strong Low Degree Hardness for the Number Partitioning Problem

TL;DR

This paper demonstrates a fundamental computational barrier for the number partitioning problem, showing that low degree algorithms cannot achieve near-optimal solutions, implying brute-force search is nearly optimal within certain runtime bounds.

Contribution

The paper introduces a nearly tight algorithmic barrier for NPP using low degree algorithms, linking solution landscape properties with computational hardness.

Findings

Low degree algorithms fail to surpass certain accuracy thresholds.

The analysis applies to NPP with independent, bounded density inputs.

Brute-force search is nearly unimprovable within polynomial to exponential runtimes.

Abstract

In the number partitioning problem (NPP) one aims to partition a given set of $N$ real numbers into two subsets with approximately equal sum. The NPP is a well-studied optimization problem and is famous for possessing a statistical-to-computational gap: when the $N$ numbers to be partitioned are i.i.d. standard gaussian, the optimal discrepancy is $2^{-\Theta(N)}$ with high probability, but the best known polynomial-time algorithms only find solutions with a discrepancy of $2^{-\Theta(\log^2 N)}$. This gap is a common feature in optimization problems over random combinatorial structures, and indicates the need for a study that goes beyond worst-case analysis. We provide evidence of a nearly tight algorithmic barrier for the number partitioning problem. Namely we consider the family of low coordinate degree algorithms (with randomized rounding into the Boolean cube), and show that degree $D$ algorithms fail to solve the NPP to accuracy beyond $2^{-\widetilde O(D)}$. According to the low degree heuristic, this suggests that simple brute-force search algorithms are nearly unimprovable, given any allotted runtime between polynomial and exponential in $N$. Our proof combines the isolation of solutions in the landscape with a conditional form of the overlap gap property: given a good solution to an NPP instance, slightly noising the NPP instance typically leaves no good solutions near the original one. In fact our analysis applies whenever the $N$ numbers to be partitioned are independent with uniformly bounded density.