Regression with Label Differential Privacy

TL;DR

This paper introduces an optimal label differential privacy mechanism for regression models, utilizing a randomized response on bins approach, with an efficient algorithm and strong experimental validation.

Contribution

It proposes a novel optimal label DP mechanism for regression, based on a global prior and a randomized response on bins, with an efficient bin optimization algorithm.

Findings

The proposed mechanism is optimal under certain regression loss functions.

Experimental results show the method's effectiveness across multiple datasets.

The approach outperforms existing privacy-preserving regression techniques.

Abstract

We study the task of training regression models with the guarantee of label differential privacy (DP). Based on a global prior distribution on label values, which could be obtained privately, we derive a label DP randomization mechanism that is optimal under a given regression loss function. We prove that the optimal mechanism takes the form of a "randomized response on bins", and propose an efficient algorithm for finding the optimal bin values. We carry out a thorough experimental evaluation on several datasets demonstrating the efficacy of our algorithm.