A Fine-Grained Understanding of Uniform Convergence for Halfspaces

TL;DR

This paper provides a detailed analysis of uniform convergence for halfspaces, revealing different behaviors for homogeneous and inhomogeneous cases and establishing tight bounds and thresholds.

Contribution

It offers the first fine-grained, nearly complete characterization of uniform convergence for halfspaces, including tight bounds and structural thresholds.

Findings

Inhomogeneous halfspaces have population error bounds of Θ(d ln(n/d)/n).

Homogeneous halfspaces in R^2 have O(1/n) error in the realizable case.

A bandwise, log-free deviation bound is established for homogeneous halfspaces, with a matching lower bound.

Abstract

We study the fine-grained uniform convergence behavior of halfspaces beyond worst-case VC bounds. For inhomogeneous halfspaces in $\mathbb{R}^d$ with $d\ge 2$, we show that standard first-order VC bounds are essentially tight: even consistent hypotheses can incur population error $\Theta(d\ln(n/d)/n)$, and in the agnostic setting the deviation scales as $\sqrt{\tau\ln(1/\tau)}$ at true error $\tau$. In contrast, homogeneous halfspaces in $\mathbb{R}^2$ exhibit a markedly different behavior. In the realizable case, every hypothesis consistent with the sample has error $O(1/n)$. In the agnostic case, we prove a bandwise, log-free deviation bound on each dyadic risk band via a critical-wedge localization argument. Unioning over bands incurs only a $\ln\ln n$ overhead, and we establish a matching lower bound showing this overhead is unavoidable. Together, these results give a fine-grained and nearly complete picture of uniform convergence for halfspaces, revealing sharp dimensional and structural thresholds.