Private Center Points and Learning of Halfspaces

TL;DR

This paper introduces a differentially private algorithm for learning halfspaces over finite domains, utilizing a new private method for locating approximate center points, and establishes lower bounds for private convex hull point finding.

Contribution

It presents a novel private learner for halfspaces based on a private center point algorithm and explores the relationship between these problems, also providing lower bounds for private convex hull point detection.

Findings

Sample complexity for private halfspace learning is polynomial in dimension and polylogarithmic in domain size.

A new differentially private algorithm for approximate center points in high dimensions.

Lower bounds for private convex hull point finding under different privacy models.

Abstract

We present a private learner for halfspaces over an arbitrary finite domain $X\subset \mathbb{R}^d$ with sample complexity $mathrm{poly}(d,2^{\log^*|X|})$. The building block for this learner is a differentially private algorithm for locating an approximate center point of $m>\mathrm{poly}(d,2^{\log^*|X|})$ points -- a high dimensional generalization of the median function. Our construction establishes a relationship between these two problems that is reminiscent of the relation between the median and learning one-dimensional thresholds [Bun et al.\ FOCS '15]. This relationship suggests that the problem of privately locating a center point may have further applications in the design of differentially private algorithms. We also provide a lower bound on the sample complexity for privately finding a point in the convex hull. For approximate differential privacy, we show a lower bound of $m=\Omega(d+\log^*|X|)$, whereas for pure differential privacy $m=\Omega(d\log|X|)$.