Hardness of High-Dimensional Linear Classification

TL;DR

This paper proves exponential lower bounds in dimension for the Maximum Halfspace Discrepancy problem, revealing fundamental computational hardness in high-dimensional linear classification.

Contribution

It establishes the first exponential in dimension lower bounds for high-dimensional linear classification problems, based on reductions from well-known hardness conjectures.

Findings

Matching lower bounds of rac{Omega(n^d)}{Omega(1/varepsilon^d)} based on Affine Degeneracy testing.

Lower bounds conditioned on k-Sum conjecture, rac{Omega(n^{d/2})}{Omega(1/varepsilon^{d/2})}.

Unconditional lower bounds in models restricted to sidedness queries.

Abstract

We establish new exponential in dimension lower bounds for the Maximum Halfspace Discrepancy problem, which models linear classification. Both are fundamental problems in computational geometry and machine learning in their exact and approximate forms. However, only $O(n^d)$ and respectively $\tilde O(1/\varepsilon^d)$ upper bounds are known and complemented by polynomial lower bounds that do not support the exponential in dimension dependence. We close this gap up to polylogarithmic terms by reduction from widely-believed hardness conjectures for Affine Degeneracy testing and $k$-Sum problems. Our reductions yield matching lower bounds of $\tilde\Omega(n^d)$ and respectively $\tilde\Omega(1/\varepsilon^d)$ based on Affine Degeneracy testing, and $\tilde\Omega(n^{d/2})$ and respectively $\tilde\Omega(1/\varepsilon^{d/2})$ conditioned on $k$-Sum. The first bound also holds unconditionally if the computational model is restricted to make sidedness queries, which corresponds to a widely spread setting implemented and optimized in many contemporary algorithms and computing paradigms.