Hush! Protecting Secrets During Model Training: An Indistinguishability Approach

TL;DR

This paper introduces a new approach to protect secrets in model training by bounding the probability of secret reconstruction, offering better utility than differential privacy in certain settings.

Contribution

It proposes an alternative secret protection definition and an algorithm that outperforms DP-SGD by solving a linear program and using weighted Poisson sampling.

Findings

Our algorithm significantly outperforms DP-SGD baseline.

The new protection method maintains higher utility while safeguarding secrets.

Empirical results demonstrate effective secret protection with improved utility.

Abstract

We consider the problem of secret protection, in which a business or organization wishes to train a model on their own data, while attempting to not leak secrets potentially contained in that data via the model. The standard method for training models to avoid memorization of secret information is via differential privacy (DP). However, DP requires a large loss in utility or a large dataset to achieve its strict privacy definition, which may be unnecessary in our setting where the data curator and data owner are the same entity. We propose an alternate definition of secret protection that instead of targeting DP, instead targets a bound on the posterior probability of secret reconstruction. We then propose and empirically evaluate an algorithm for model training with this secret protection definition. Our algorithm solves a linear program to assign weights to examples based on the desired per-secret protections, and then performs Poisson sampling using these weights. We show our algorithm significantly outperforms the baseline of running DP-SGD on the whole dataset.