Multiclass versus Binary Differentially Private PAC Learning

TL;DR

This paper introduces a reduction from multiclass to binary differentially private PAC learning, resulting in a more sample-efficient multiclass learner with improved dependence on key parameters.

Contribution

It presents a novel reduction method that leverages binary private PAC learners to efficiently learn multiclass classifiers with better sample complexity bounds.

Findings

Polynomial dependence on multiclass Littlestone dimension

Poly-logarithmic dependence on number of classes

Exponential improvement over previous learners

Abstract

We show a generic reduction from multiclass differentially private PAC learning to binary private PAC learning. We apply this transformation to a recently proposed binary private PAC learner to obtain a private multiclass learner with sample complexity that has a polynomial dependence on the multiclass Littlestone dimension and a poly-logarithmic dependence on the number of classes. This yields an exponential improvement in the dependence on both parameters over learners from previous work. Our proof extends the notion of $\Psi$-dimension defined in work of Ben-David et al. [JCSS '95] to the online setting and explores its general properties.