On Convex Optimization with Semi-Sensitive Features

TL;DR

This paper investigates differentially private empirical risk minimization when only some features are sensitive, providing improved bounds that scale polylogarithmically with the sensitive domain size, advancing privacy-utility trade-offs.

Contribution

It introduces a semi-sensitive DP setting for ERM, generalizing Label DP, and offers tighter bounds on excess risk with better scalability in sensitive domain size.

Findings

Error scales polylogarithmically with sensitive domain size

Improved upper and lower bounds on excess risk

Generalization of Label DP setting

Abstract

We study the differentially private (DP) empirical risk minimization (ERM) problem under the semi-sensitive DP setting where only some features are sensitive. This generalizes the Label DP setting where only the label is sensitive. We give improved upper and lower bounds on the excess risk for DP-ERM. In particular, we show that the error only scales polylogarithmically in terms of the sensitive domain size, improving upon previous results that scale polynomially in the sensitive domain size (Ghazi et al., 2021).