Correlation inference attacks against machine learning models

TL;DR

This paper investigates how machine learning models can leak information about the correlations in their training data through novel inference attacks, revealing potential privacy risks.

Contribution

It introduces both model-less and model-based correlation inference attacks and demonstrates their effectiveness against common models on tabular datasets.

Findings

Models leak correlations between input variables.

Correlation information can be used for attribute inference.

Attacks work with minimal assumptions and black-box access.

Abstract

Despite machine learning models being widely used today, the relationship between a model and its training dataset is not well understood. We explore correlation inference attacks, whether and when a model leaks information about the correlations between the input variables of its training dataset. We first propose a model-less attack, where an adversary exploits the spherical parametrization of correlation matrices alone to make an informed guess. Second, we propose a model-based attack, where an adversary exploits black-box model access to infer the correlations using minimal and realistic assumptions. Third, we evaluate our attacks against logistic regression and multilayer perceptron models on three tabular datasets and show the models to leak correlations. We finally show how extracted correlations can be used as building blocks for attribute inference attacks and enable weaker adversaries. Our results raise fundamental questions on what a model does and should remember from its training set.