Locally Private Estimation with Public Features

TL;DR

This paper studies locally differentially private learning with the addition of public features, proposing an estimator that optimally combines public and private data for improved accuracy.

Contribution

It introduces semi-feature LDP, analyzes its impact on convergence rates, and proposes HistOfTree, an estimator that achieves optimal rates by leveraging both public and private features.

Findings

HistOfTree attains mini-max optimal convergence rate.

Empirical results show superior performance of HistOfTree.

Flexible feature protection strategies are effective in practice.

Abstract

We initiate the study of locally differentially private (LDP) learning with public features. We define semi-feature LDP, where some features are publicly available while the remaining ones, along with the label, require protection under local differential privacy. Under semi-feature LDP, we demonstrate that the mini-max convergence rate for non-parametric regression is significantly reduced compared to that of classical LDP. Then we propose HistOfTree, an estimator that fully leverages the information contained in both public and private features. Theoretically, HistOfTree reaches the mini-max optimal convergence rate. Empirically, HistOfTree achieves superior performance on both synthetic and real data. We also explore scenarios where users have the flexibility to select features for protection manually. In such cases, we propose an estimator and a data-driven parameter tuning strategy, leading to analogous theoretical and empirical results.