Privately Learning Decision Lists and a Differentially Private Winnow

TL;DR

This paper introduces new differentially private algorithms for learning decision lists and halfspaces, achieving near-optimal sample complexity and mistake bounds in PAC and online models, respectively.

Contribution

It presents the first efficient private algorithms for decision lists in PAC and online models, matching non-private performance guarantees.

Findings

Efficient private PAC algorithm for decision lists with minimal sample overhead.

Private Winnow algorithm for halfspaces with polylogarithmic mistake bounds.

Private online learning of decision lists matching non-private guarantees.

Abstract

We give new differentially private algorithms for the classic problems of learning decision lists and large-margin halfspaces in the PAC and online models. In the PAC model, we give a computationally efficient algorithm for learning decision lists with minimal sample overhead over the best non-private algorithms. In the online model, we give a private analog of the influential Winnow algorithm for learning halfspaces with mistake bound polylogarithmic in the dimension and inverse polynomial in the margin. As an application, we describe how to privately learn decision lists in the online model, qualitatively matching state-of-the art non-private guarantees.