Range Membership Inference Attacks

TL;DR

This paper introduces range membership inference attacks (RaMIAs), a new method to better assess privacy risks in machine learning models by detecting if training data falls within a specified range, not just exact matches.

Contribution

It proposes RaMIAs as a novel, more comprehensive privacy auditing tool that captures privacy loss beyond traditional membership inference attacks.

Findings

RaMIAs outperform MIAs in detecting privacy leaks across data types

RaMIAs provide a more accurate measure of privacy loss

The method is applicable to tabular, image, and language data

Abstract

Machine learning models can leak private information about their training data. The standard methods to measure this privacy risk, based on membership inference attacks (MIAs), only check if a given data point \textit{exactly} matches a training point, neglecting the potential of similar or partially overlapping memorized data revealing the same private information. To address this issue, we introduce the class of range membership inference attacks (RaMIAs), testing if the model was trained on any data in a specified range (defined based on the semantics of privacy). We formulate the RaMIAs game and design a principled statistical test for its composite hypotheses. We show that RaMIAs can capture privacy loss more accurately and comprehensively than MIAs on various types of data, such as tabular, image, and language. RaMIA paves the way for more comprehensive and meaningful privacy auditing of machine learning algorithms.