Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA

TL;DR

This paper unifies leading membership inference attacks under an exponential-family framework, introduces BaVarIA as a Bayesian variance inference attack, and demonstrates its superior or comparable performance across multiple datasets and settings.

Contribution

It reveals a unified exponential-family perspective of MIAs, introduces BaVarIA with Bayesian variance estimation, and shows improved attack performance especially in low-shadow-model regimes.

Findings

BaVarIA matches or outperforms LiRA and RMIA in various datasets.

BaVarIA provides stable performance without hyperparameter tuning.

The framework connects RMIA and LiRA as parts of a spectrum of model complexity.

Abstract

Membership inference attacks (MIAs) are becoming standard tools for auditing the privacy of machine learning models. The leading attacks -- LiRA (Carlini et al., 2022) and RMIA (Zarifzadeh et al., 2024) -- appear to use distinct scoring strategies, while the recently proposed BASE (Lassila et al., 2025) was shown to be equivalent to RMIA, making it difficult for practitioners to choose among them. We show that all three are instances of a single exponential-family log-likelihood ratio framework, differing only in their distributional assumptions and the number of parameters estimated per data point. This unification reveals a hierarchy (BASE1-4) that connects RMIA and LiRA as endpoints of a spectrum of increasing model complexity. Within this framework, we identify variance estimation as the key bottleneck at small shadow-model budgets and propose BaVarIA, a Bayesian variance inference attack that replaces threshold-based parameter switching with conjugate normal-inverse-gamma priors. BaVarIA yields a Student-t predictive (BaVarIA-t) or a Gaussian with stabilized variance (BaVarIA-n), providing stable performance without additional hyperparameter tuning. Across 12 datasets and 7 shadow-model budgets, BaVarIA matches or improves upon LiRA and RMIA, with the largest gains in the practically important low-shadow-model and offline regimes.