Integrating Feature Correlation in Differential Privacy with Applications in DP-ERM

TL;DR

This paper introduces CorrDP, a relaxed differential privacy framework that accounts for feature correlations, improving privacy utility trade-offs in DP-ERM tasks.

Contribution

It proposes a correlation-aware differential privacy model that relaxes privacy for insensitive features considering their correlations, with algorithms and utility guarantees.

Findings

CorrDP outperforms standard DP in experiments with insensitive features.

Algorithms incorporate distance-dependent noise for better utility.

Estimation of correlation distance maintains privacy-utility balance.

Abstract

Standard differential privacy imposes uniform privacy constraints across all features, overlooking the inherent distinction between sensitive and insensitive features in practice. In this paper, we introduce a relaxed definition of differential privacy that accounts for such heterogeneity, allowing certain features to be treated as insensitive even when correlated with sensitive ones. We propose a correlation-aware framework, $\textsf{CorrDP}$, which relaxes privacy for insensitive features while accounting for their correlations with sensitive features, with the correlations quantified using total variation distance. We design algorithms for differentially private empirical risk minimization (DP-ERM) under the $\textsf{CorrDP}$ framework, incorporating distance-dependent noise into gradients for improved theoretical utility guarantees. When the correlation distance is unknown, we estimate it from the dataset and show that it achieves a comparable privacy-utility guarantee. We perform experiments on synthetic and real-world datasets and show that $\textsf{CorrDP}$-based DP-ERM algorithms consistently outperform the standard DP framework in the presence of insensitive features.