Supervised Machine Learning with Plausible Deniability

TL;DR

This paper explores the privacy implications of machine learning models, demonstrating that models can be associated with plausible training data through specific learning rules, thus providing a form of deniability about their actual training data.

Contribution

It introduces the concept of plausible deniability in ML training data privacy and shows how models can be reconstructed from random data using tailored learning rules.

Findings

Models can be associated with multiple training datasets via specific learning rules.

Plausible deniability allows denying the actual training data used.

Open source tools demonstrate how to find such learning rules.

Abstract

We study the question of how well machine learning (ML) models trained on a certain data set provide privacy for the training data, or equivalently, whether it is possible to reverse-engineer the training data from a given ML model. While this is easy to answer negatively in the most general case, it is interesting to note that the protection extends over non-recoverability towards plausible deniability: Given an ML model $f$, we show that one can take a set of purely random training data, and from this define a suitable ``learning rule'' that will produce a ML model that is exactly $f$. Thus, any speculation about which data has been used to train $f$ is deniable upon the claim that any other data could have led to the same results. We corroborate our theoretical finding with practical examples, and open source implementations of how to find the learning rules for a chosen set of raining data.