Bayesian Inference of Training Dataset Membership

TL;DR

This paper introduces a Bayesian inference method for membership inference that efficiently determines dataset membership in trained models using post-hoc metrics, enhancing privacy analysis without extensive retraining.

Contribution

It presents a novel, interpretable Bayesian approach for membership inference that relies on post-hoc metrics, avoiding the need for shadow models or internal model access.

Findings

Effective in synthetic datasets for distinguishing members from non-members

Capable of detecting distribution shifts in data

Provides a practical, interpretable alternative to existing methods

Abstract

Determining whether a dataset was part of a machine learning model's training data pool can reveal privacy vulnerabilities, a challenge often addressed through membership inference attacks (MIAs). Traditional MIAs typically require access to model internals or rely on computationally intensive shadow models. This paper proposes an efficient, interpretable and principled Bayesian inference method for membership inference. By analyzing post-hoc metrics such as prediction error, confidence (entropy), perturbation magnitude, and dataset statistics from a trained ML model, our approach computes posterior probabilities of membership without requiring extensive model training. Experimental results on synthetic datasets demonstrate the method's effectiveness in distinguishing member from non-member datasets. Beyond membership inference, this method can also detect distribution shifts, offering a practical and interpretable alternative to existing approaches.