Classification Under Local Differential Privacy with Model Reversal and Model Averaging

TL;DR

This paper introduces new methods for classification under local differential privacy, including model reversal and averaging, which improve accuracy while maintaining privacy guarantees, supported by theoretical analysis and empirical validation.

Contribution

It proposes novel techniques tailored for LDP, such as model reversal and averaging, to enhance classification utility without compromising privacy.

Findings

Significant accuracy improvements demonstrated on real-world datasets

Theoretical excess risk bounds established under LDP

Model reversal effectively salvages underperforming classifiers

Abstract

Local differential privacy (LDP) has become a central topic in data privacy research, offering strong privacy guarantees by perturbing user data at the source and removing the need for a trusted curator. However, the noise introduced by LDP often significantly reduces data utility. To address this issue, we reinterpret private learning under LDP as a transfer learning problem, where the noisy data serve as the source domain and the unobserved clean data as the target. We propose novel techniques specifically designed for LDP to improve classification performance without compromising privacy: (1) a noised binary feedback-based evaluation mechanism for estimating dataset utility; (2) model reversal, which salvages underperforming classifiers by inverting their decision boundaries; and (3) model averaging, which assigns weights to multiple reversed classifiers based on their estimated utility. We provide theoretical excess risk bounds under LDP and demonstrate how our methods reduce this risk. Empirical results on both simulated and real-world datasets show substantial improvements in classification accuracy.