Lower Bounds for Approximate Sign Rank

TL;DR

This paper establishes new bounds on approximate sign-rank, linking it to geometric properties and VC dimension, and analyzes specific matrices like Hadamard, advancing understanding of complexity measures in computational geometry and learning theory.

Contribution

It introduces novel bounds and geometric theorems for approximate sign-rank, connecting it to VC dimension and analyzing specific matrices, which were not previously well-understood.

Findings

Every sign matrix with approximate sign-rank d contains a large monochromatic rectangle.

Lower bound of Ω(√d/ log d) on approximate sign-rank of large-margin half-spaces.

Approximate sign-rank of Hadamard matrix is at most m^{O(√m) log(1/ε)} and at least Ω_ε(m).

Abstract

We prove new upper and lower bounds on $\epsilon$-approximate sign-rank, a relaxation of sign-rank introduced by Chornomaz, Moran, and Waknine (STOC 2025). We show that every $m \times n$ sign matrix with approximate sign-rank $d$ contains a monochromatic rectangle of size $d^{-O(d)}m \times d^{-O(d^2)}n$, paralleling classical results for exact sign-rank. As an application, we establish a lower bound of $\Omega(\sqrt{d/\log d})$ on the $\epsilon$-approximate sign-rank of large-margin $d$-dimensional half-spaces. Prior to our work, the only general lower bound technique known for approximate sign-rank yielded bounds of strength $\epsilon^{-1} - 1$, which are constant for fixed $\epsilon$. A key ingredient is a new geometric theorem on hyperplane avoidance: for any set of $n$ points in general position in $\mathbb{R}^d$, there exist $d$ subsets, each of size $d^{-O(d)} n$, such that no hyperplane simultaneously splits all of them. The proof combines the Forster-Barthe isotropic position theorem with the Bourgain-Tzafriri restricted invertibility principle. We also study the relationship between approximate sign-rank and VC dimension. We prove a lower bound on approximate sign-rank in terms of VC dimension, and exhibit concept classes of VC dimension $2$ with large approximate sign-rank. Finally, we study the approximate sign-rank of the $2^m \times 2^m$ Hadamard matrix $H_m$. The sign-rank of $H_m$ is known to be $\Omega(\sqrt{2^m})$ by Forster's classic theorem. Contrasting this, we adapt an argument of Alman and Williams to show that the approximate sign-rank of $H_m$ is at most $m^{O(\sqrt{m} \log(1/\epsilon))}$, and hence the Hadamard matrix does not witness polynomial-strength lower bounds for approximate sign-rank. Using our VC dimension bound, we prove that the approximate sign-rank of $H_m$ is at least $\Omega_\epsilon(m)$.