Formalizing Distribution Inference Risks

TL;DR

This paper introduces a formal, broad definition of property inference attacks in machine learning, highlighting their potential risks and extending previous concepts to include new types of inferences about training data distributions.

Contribution

It provides a formal, generic framework for property inference attacks, encompassing previous methods and introducing new attack types, with experimental insights into associated risks.

Findings

The new definition captures existing and novel property inference attacks.

Experiments demonstrate the potential privacy risks of property inference.

The framework extends understanding of what can be inferred about training distributions.

Abstract

Property inference attacks reveal statistical properties about a training set but are difficult to distinguish from the primary purposes of statistical machine learning, which is to produce models that capture statistical properties about a distribution. Motivated by Yeom et al.'s membership inference framework, we propose a formal and generic definition of property inference attacks. The proposed notion describes attacks that can distinguish between possible training distributions, extending beyond previous property inference attacks that infer the ratio of a particular type of data in the training data set. In this paper, we show how our definition captures previous property inference attacks as well as a new attack that reveals the average degree of nodes of a training graph and report on experiments giving insight into the potential risks of property inference attacks.