Differentially Private Truncation of Unbounded Data via Public Second Moments

TL;DR

This paper introduces a method called PMT that uses public second-moment information to enable differential privacy on unbounded data, improving model accuracy and stability.

Contribution

The paper proposes a novel public-moment-guided truncation technique that enhances differential privacy by conditioning data transformations on public second moments.

Findings

PMT improves DP model accuracy on synthetic data.

PMT enhances stability and robustness of DP models.

Theoretical bounds confirm improved error rates.

Abstract

Data privacy is important in the AI era, and differential privacy (DP) is one of the golden solutions. However, DP is typically applicable only if data have a bounded underlying distribution. We address this limitation by leveraging second-moment information from a small amount of public data. We propose Public-moment-guided Truncation (PMT), which transforms private data using the public second-moment matrix and applies a principled truncation whose radius depends only on non-private quantities: data dimension and sample size. This transformation yields a well-conditioned second-moment matrix, enabling its inversion with a significantly strengthened ability to resist the DP noise. Furthermore, we demonstrate the applicability of PMT by using penalized and generalized linear regressions. Specifically, we design new loss functions and algorithms, ensuring that solutions in the transformed space can be mapped back to the original domain. We have established improvements in the models' DP estimation through theoretical error bounds, robustness guarantees, and convergence results, attributing the gains to the conditioning effect of PMT. Experiments on synthetic and real datasets confirm that PMT substantially improves the accuracy and stability of DP models.