Improved Bounds for Pure Private Agnostic Learning: Item-Level and User-Level Privacy

TL;DR

This paper advances the theoretical understanding of private agnostic learning by providing improved bounds on the number of users needed under item-level and user-level privacy, with near-optimal results for general concept classes.

Contribution

It introduces new upper bounds for user requirements in pure private agnostic learning, improving upon previous results and addressing threshold learning under user privacy.

Findings

Achieves near-optimal bounds for item-level privacy in general concept classes.

Provides tighter upper bounds for user-level privacy compared to prior work.

Develops a nearly tight user complexity algorithm for learning thresholds.

Abstract

Machine Learning has made remarkable progress in a wide range of fields. In many scenarios, learning is performed on datasets involving sensitive information, in which privacy protection is essential for learning algorithms. In this work, we study pure private learning in the agnostic model -- a framework reflecting the learning process in practice. We examine the number of users required under item-level (where each user contributes one example) and user-level (where each user contributes multiple examples) privacy and derive several improved upper bounds. For item-level privacy, our algorithm achieves a near optimal bound for general concept classes. We extend this to the user-level setting, rendering a tighter upper bound than the one proved by Ghazi et al. (2023). Lastly, we consider the problem of learning thresholds under user-level privacy and present an algorithm with a nearly tight user complexity.