Private Learning of Halfspaces: Simplifying the Construction and Reducing the Sample Complexity

TL;DR

This paper introduces a differentially private learning algorithm for halfspaces over finite grids, improving sample complexity and simplifying the construction compared to previous methods.

Contribution

It presents a new private learner for halfspaces with reduced sample complexity and a novel iterative algorithm for solving linear feasibility problems.

Findings

Achieves sample complexity of approximately d^{2.5} * 2^{log^*|G|}

Improves previous results by a factor of d^2

Provides an iterative private algorithm for linear feasibility

Abstract

We present a differentially private learner for halfspaces over a finite grid $G$ in $\mathbb{R}^d$ with sample complexity $\approx d^{2.5}\cdot 2^{\log^*|G|}$, which improves the state-of-the-art result of [Beimel et al., COLT 2019] by a $d^2$ factor. The building block for our learner is a new differentially private algorithm for approximately solving the linear feasibility problem: Given a feasible collection of $m$ linear constraints of the form $Ax\geq b$, the task is to privately identify a solution $x$ that satisfies most of the constraints. Our algorithm is iterative, where each iteration determines the next coordinate of the constructed solution $x$.