Privacy Auditing Synthetic Data Release through Local Likelihood Attacks

TL;DR

This paper introduces Gen-LRA, a novel no-box membership inference attack that effectively detects privacy leakage in synthetic data by exploiting overfitting, with strong theoretical backing and superior empirical performance.

Contribution

The paper presents Gen-LRA, a new local likelihood ratio attack that is computationally efficient, theoretically grounded, and outperforms existing methods in privacy auditing of synthetic data.

Findings

Gen-LRA consistently outperforms existing MIAs across diverse datasets and models.

Theoretical analysis shows Gen-LRA's score correlates with local overfitting, enabling provable detection.

Empirical results demonstrate Gen-LRA's effectiveness at low false positive rates.

Abstract

Auditing the privacy leakage of synthetic data is an important but unresolved problem. Existing privacy auditing frameworks for synthetic data rely on heuristics and unrealistic assumptions about model access, offering limited ability to describe or detect the privacy exposure of training data through synthetic data release. In this paper, we study designing membership inference attacks (MIAs) that specifically exploit the observation that tabular generative models tend to significantly overfit to certain regions of the training distribution. We propose \emph{Generative Likelihood Ratio Attack} (Gen-LRA), a novel, computationally efficient No-Box MIA that, with no assumption of model knowledge or access, formulates its attack by evaluating the influence a test observation has on a surrogate model's estimate of a local likelihood ratio over the synthetic data. We develop a theoretical framework for the attack: we show that the Gen-LRA score admits a closed-form characterization as a localized density-ratio statistic, and we prove that under a general model of local overfitting it produces a provable mean-score gap between members and non-members, yielding testable predictions for when the attack should succeed. We validate these predictions in a controlled simulation study and assess Gen-LRA against a comprehensive benchmark spanning diverse datasets, generative model architectures, and attack parameters. Across metrics, Gen-LRA consistently dominates competing MIAs, with especially strong gains at low false positive rates. These results underscore Gen-LRA's effectiveness as a privacy auditing tool for the release of synthetic data, and highlight the significant privacy risks posed by generative model overfitting in real-world applications.

Tables

Table 1. Table 1 . Attack ranking performance across evaluation metrics over all 1,525 runs. Higher Top-k values are better, while lower mean rank indicates stronger performance. Gen-LRA achieves the highest Top-1 rate on every metric at more than 2 × 2\times that of any baseline.

			TPR at Fixed FPR
Method	Statistic	AUC	0.0	0.001	0.01	0.1
Gen LRA $k=10$	Top-1	38.6%	42.1%	41.5%	39.8%	46.2%
	Top-3	71.7%	78.5%	75.6%	73.1%	76.7%
	Mean Rank (std)	2.83 (2.12)	2.79 (2.05)	2.76 (2.02)	2.78 (2.04)	2.54 (2.0)
Classifier	Top-1	16.0%	9.6%	9.1%	11.6%	9.6%
	Top-3	45.7%	35.4%	31.9%	35.2%	35.7%
	Mean Rank (std)	4.29 (2.57)	5.22 (2.16)	5.04 (2.36)	4.79 (2.47)	4.83 (2.48)
DCR	Top-1	9.0%	17.4%	15.0%	12.2%	8.3%
	Top-3	31.2%	44.1%	37.4%	37.9%	32.2%
	Mean Rank (std)	4.93 (2.34)	4.67 (2.3)	4.77 (2.44)	4.63 (2.39)	4.86 (2.3)
DCR-Diff	Top-1	12.0%	18.4%	16.4%	16.3%	16.3%
	Top-3	43.5%	60.3%	51.8%	50.0%	46.2%
	Mean Rank (std)	4.11 (2.21)	3.78 (2.04)	4.05 (2.35)	4.2 (2.49)	4.2 (2.48)
DOMIAS	Top-1	10.0%	12.7%	11.8%	11.3%	11.8%
	Top-3	36.8%	52.1%	43.4%	36.8%	41.0%
	Mean Rank (std)	4.42 (2.19)	4.22 (2.08)	4.56 (2.45)	5.03 (2.67)	4.63 (2.59)
DPI	Top-1	10.7%	2.4%	1.9%	4.0%	6.5%
	Top-3	38.9%	13.0%	12.4%	20.6%	29.9%
	Mean Rank (std)	4.5 (2.34)	6.79 (1.51)	5.94 (1.95)	5.18 (1.93)	4.97 (2.22)
LOGAN	Top-1	12.7%	16.2%	13.8%	11.3%	11.0%
	Top-3	28.7%	48.9%	41.7%	30.8%	32.1%
	Mean Rank (std)	5.51 (2.79)	4.35 (2.2)	4.71 (2.53)	5.3 (2.59)	5.19 (2.62)
Local Neighborhood	Top-1	2.4%	6.6%	5.8%	6.7%	4.4%
	Top-3	16.0%	32.9%	30.3%	29.9%	20.8%
	Mean Rank (std)	5.79 (1.99)	5.4 (2.09)	5.26 (2.28)	5.11 (2.3)	5.41 (2.14)
MC	Top-1	4.2%	12.5%	11.3%	8.5%	6.3%
	Top-3	22.5%	43.8%	35.7%	30.3%	25.3%
	Mean Rank (std)	5.61 (2.3)	4.71 (2.14)	4.91 (2.41)	5.11 (2.41)	5.32 (2.35)

Table 2. Table 2 . Mean performance with standard deviations across the top 100 runs for each metric. Bold indicates the highest value in each column. Despite outliers that inflate attacks that target near-memorization, Gen-LRA achieves the highest mean on every metric.

		TPR at Fixed FPR
Method	AUC	0.0	0.01	0.1
Gen-LRA	0.589 (0.03)	0.051 (0.03)	0.053 (0.03)	0.211 (0.05)
Classifier	0.557 (0.05)	0.012 (0.02)	0.021 (0.02)	0.144 (0.06)
DCR	0.579 (0.04)	0.029 (0.04)	0.040 (0.04)	0.175 (0.06)
DCR-Diff	0.570 (0.04)	0.037 (0.04)	0.044 (0.04)	0.171 (0.05)
DOMIAS	0.546 (0.04)	0.024 (0.02)	0.024 (0.02)	0.140 (0.04)
DPI	0.526 (0.03)	0.001 (0.00)	0.013 (0.01)	0.114 (0.02)
LOGAN	0.493 (0.02)	0.013 (0.01)	0.015 (0.01)	0.105 (0.03)
Local N.	0.538 (0.04)	0.013 (0.02)	0.024 (0.02)	0.132 (0.04)
MC	0.548 (0.04)	0.016 (0.02)	0.024 (0.02)	0.134 (0.05)

Table 3. Table 3 . List of Datasets included in Section 5.

Dataset	OpenML ID	N-size	Classes	Cat. Feat.	Num Feat.
GesturePhaseSegmentationProcessed	4538	9873	5	1	32
MiceProtein	40966	1080	8	5	77
PhishingWebsites	4534	11055	2	31	0
adult	1590	48842	2	9	6
analcatdata_authorship	40983	4839	2	1	5
bank-marketing	1461	45211	2	10	7
banknote-authentication	1462	1372	2	1	4
blood-transfusion-service-center	1464	748	2	1	4
car	40975	1728	4	7	0
churn	40701	5000	2	5	16
climate-model-simulation-crashes	1467	540	2	1	20
cmc	23	1473	3	8	2
connect-4	40668	67557	3	43	0
credit-approval	29	690	2	10	6
credit-g	31	1000	2	14	7
diabetes	37	768	2	1	8
electricity	151	45312	2	2	7
eucalyptus	43924	736	5	15	5
kc1	1067	2109	2	1	21
kc2	1063	522	2	1	21
kr-vs-kp	3	3196	2	37	0
letter	6	20000	26	1	16
mfeat-morphological	18	2000	10	1	6
numerai28.6	23517	96320	2	1	21
optdigits	28	5620	10	1	64
pc3	1044	10936	3	4	24
pendigits	32	10992	10	1	16
phoneme	1489	5404	2	1	5
satimage	182	6430	6	1	36
segment	40984	2310	7	1	19
sick	38	3772	2	23	7
spambase	44	4601	2	1	57
steel-plates-fault	40983	4839	2	1	5
texture	40499	5500	11	1	40
tic-tac-toe	50	958	2	10	0
vehicle	54	846	4	1	18

Table 4. Table 4 . Mean (std) LRA detection performance by encoding strategy, top 100 runs per metric, averaged over datasets, generative methods, and seeds. Rows sorted by AUC-ROC. Bold marks best encoding per metric.

Encoding	AUC-ROC	TPR@FPR=0	TPR@FPR=0.001	TPR@FPR=0.01	TPR@FPR=0.1
Ordinal	0.616 (0.053)	0.011 (0.010)	0.013 (0.012)	0.043 (0.028)	0.221 (0.079)
VAE	0.594 (0.049)	0.009 (0.010)	0.012 (0.012)	0.039 (0.027)	0.198 (0.064)
PCA	0.575 (0.046)	0.010 (0.008)	0.012 (0.009)	0.031 (0.021)	0.171 (0.056)

Table 5. Table 5. Mean AUC-ROC at different k k values for Gen-LRA for top 300 highest runs.

Model	$k=1$	$k=5$	$k=10$	$k=25$	$k=50$	$k=100$
AdsGAN	0.57 (0.00)	0.58	0.57 (0.00)	0.57 (0.01)	0.58 (0.01)	0.61 (0.00)
ARF	0.60 (0.02)	0.58 (0.02)	0.59 (0.01)	0.58 (0.01)	0.58 (0.01)	0.58 (0.01)
CTGAN	0.60 (0.01)	0.58 (0.00)	0.59 (0.02)	0.60 (0.00)	0.58 (0.01)	0.58 (0.01)
Tab-DDPM	0.60 (0.02)	0.60 (0.01)	0.60 (0.02)	0.61 (0.00)	0.60 (0.01)	0.60 (0.01)
N-Flow	0.61 (0.00)	0.57 (0.01)	0.59 (0.00)	0.59 (0.00)	0.59 (0.00)	0.60 (0.01)
RTF	0.59 (0.02)	0.59 (0.02)	0.59 (0.02)	0.60 (0.02)	0.59 (0.02)	0.59 (0.02)
TabSyn	0.62 (0.03)	0.61 (0.02)	0.60 (0.02)	0.60 (0.01)	0.59 (0.01)	0.58 (0.01)
TVAE	0.60 (0.00)	0.59 (0.00)	0.57 (0.00)	0.57 (0.00)	0.58 (0.01)	0.58 (0.01)

Table 6. Table 6 . Median with interquartile ranges for MIAs across the top 100 runs for each metric. Distance-based baselines (DCR, DCR-Diff) exhibit medians well below their means, with IQRs touching zero, indicating that their strong-run performance is driven by a small set of near-exact memorization outliers. Gen-LRA’s median exceeds its own mean and its IQR is bounded away from zero, with median AUC (0.594) exceeding the third quartile of almost every baseline.

		TPR at Fixed FPR
Method	AUC	0.0	0.001	0.01	0.1
Gen-LRA	0.594 [0.574, 0.609]	0.055 [0.034, 0.067]	0.055 [0.034, 0.067]	0.055 [0.038, 0.072]	0.214 [0.175, 0.237]
Classifier	0.559 [0.524, 0.597]	0.005 [0.000, 0.017]	0.005 [0.001, 0.018]	0.015 [0.007, 0.027]	0.133 [0.105, 0.181]
DCR	0.579 [0.561, 0.600]	0.011 [0.000, 0.053]	0.012 [0.002, 0.053]	0.022 [0.011, 0.063]	0.178 [0.125, 0.212]
DCR-Diff	0.576 [0.541, 0.591]	0.024 [0.008, 0.063]	0.025 [0.008, 0.063]	0.031 [0.013, 0.072]	0.169 [0.131, 0.205]
DOMIAS	0.546 [0.523, 0.568]	0.019 [0.009, 0.034]	0.019 [0.009, 0.035]	0.020 [0.008, 0.032]	0.132 [0.110, 0.172]
DPI	0.528 [0.511, 0.543]	0.000 [0.000, 0.000]	0.001 [0.001, 0.001]	0.011 [0.010, 0.013]	0.113 [0.101, 0.123]
LOGAN	0.495 [0.474, 0.507]	0.007 [0.002, 0.020]	0.006 [0.002, 0.020]	0.010 [0.002, 0.026]	0.107 [0.082, 0.123]
Local Neighborhood	0.534 [0.505, 0.557]	0.005 [0.000, 0.012]	0.006 [0.001, 0.015]	0.014 [0.010, 0.029]	0.118 [0.102, 0.145]
MC	0.553 [0.522, 0.571]	0.006 [0.000, 0.023]	0.006 [0.002, 0.024]	0.017 [0.009, 0.031]	0.121 [0.100, 0.165]

Equations

$$\mathcal{A}(x^{\star })=\mathbb{I}\left[f(x^{\star })>\gamma \right],$$

$$\mathcal{I}(x^{\star },F,\theta )=\underset{ϵ\to 0}{\lim }\frac{\theta ((1-ϵ)F+ϵ{\delta }_{x^{\star }})-\theta (F)}{ϵ},$$

$$\hat{\mathcal{I}}(x^{\star },\mathcal{D},\theta )=\theta (\mathcal{D}\cup \{x^{\star }\})-\theta (\mathcal{D}).$$

$$\hat{\mathcal{I}}(x^{\star };R,S):=\log \hat{p}(S\mid R\cup \{x^{\star }\})-\log \hat{p}(S\mid R).$$

$$\mathcal{I}(g(x^{\star }),g(R),g(S))=\mathcal{I}(x^{\star },R,S).$$

$$\hat{\mathcal{I}}(x^{\star };R,S)=\sum _{s\in S_k(x^{\star })}\left[\log {\hat{p}}_{R\cup \{x^{\star }\}}(s)-\log {\hat{p}}_R(s)\right].$$

$$\hat{\mathcal{I}}(x^{\star };R,S)=\frac{1}{m+1}\sum _{s\in S_k(x^{\star })}\left[\frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}-1\right]+O_p(m^{-2}).$$

$$p_G(s)=p(s)+\frac{ϵ}{|T|}\sum _{x_i\in T}{\varphi }_{\rho }(s;x_i)+r(s),$$

$${\mu }_1-{\mu }_0\gtrsim \frac{kϵ}{(m+1)|T|}\cdot \Psi (h,\rho ,x^{\star }),$$

$$\Psi (h,\rho ,x^{\star })={\mathbb{E}}_s\left[\frac{K_h(s-x^{\star }){\varphi }_{\rho }(s;x^{\star })}{{\hat{p}}_R(s)}\right],$$

$$s_i\sim \{\begin{array}{ll}P& \text{with probability }1-ϵ,\\ \mathcal{N}(x_j,{\rho }^2{I}_d),x_j\sim \text{Unif}(T)& \text{with probability }ϵ.\end{array}$$

$$\tilde{p}(g(A))=\frac{p(A)}{|J(A)|},$$

$$\tilde{p}(\tilde{S}\mid \tilde{R}\cup \{{\tilde{x}}^{\star }\})$$

$$=\frac{p(S,R\cup \{x^{\star }\})/|J(S,R\cup \{x^{\star }\})|}{p(R\cup \{x^{\star }\})/|J(R\cup \{x^{\star }\})|}.$$

$$\tilde{p}(\tilde{S}\mid \tilde{R})$$

$$\frac{\tilde{p}(\tilde{S}\mid \tilde{R}\cup \{{\tilde{x}}^{\star }\})}{\tilde{p}(\tilde{S}\mid \tilde{R})}$$

$$\cdot \underset{=1}{\underbrace{\frac{|J(R\cup \{x^{\star }\})|\cdot |J(S,R)|}{|J(S,R\cup \{x^{\star }\})|\cdot |J(R)|}}}.$$

$$\frac{\tilde{p}(\tilde{S}\mid \tilde{R}\cup \{{\tilde{x}}^{\star }\})}{\tilde{p}(\tilde{S}\mid \tilde{R})}=\frac{p(S\mid R\cup \{x^{\star }\})}{p(S\mid R)}.$$

$${\hat{p}}_{R\cup \{x^{\star }\}}(s)$$

$$=\frac{m}{m+1}{\hat{p}}_R(s)+\frac{1}{m+1}{K}_h(s-x^{\star }).$$

$$\frac{{\hat{p}}_{R\cup \{x^{\star }\}}(s)}{{\hat{p}}_R(s)}$$

$$=1+\frac{1}{m+1}\left[\frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}-1\right].$$

$$u_s:=\frac{1}{m+1}\left[\frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}-1\right].$$

$$\log (1+u)=u-\frac{u^2}{2}+O(u^3)$$

$$\log \frac{{\hat{p}}_{R\cup \{x^{\star }\}}(s)}{{\hat{p}}_R(s)}=\frac{1}{m+1}\left[\frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}-1\right]+O(m^{-2}).$$

$$\hat{\mathcal{I}}(x^{\star };R,S)$$

$$=\frac{1}{m+1}\sum _{s\in S_k(x^{\star })}\left[\frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}-1\right]+O(k{m}^{-2})$$

$$\hat{\mathcal{I}}(x^{\star };R,S)\approx \frac{1}{m+1}\sum _{s\in S_k(x^{\star })}\left[\frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}-1\right].$$

$$\mathbb{E}[\hat{\mathcal{I}}(x^{\star };R,S)]\approx \frac{k}{m+1}\cdot {\mathbb{E}}_{s\sim p_G}\left[g(s,x^{\star })|s\in S_k(x^{\star })\right].$$

$${\mathbb{E}}_{s\sim p_G}\left[\frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}\right]$$

$$+\underset{\text{Piece B: memorization sum}}{\underbrace{\frac{ϵ}{|T|}\sum _{i=1}^{|T|}\int \frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}{\varphi }_{\rho }(s;x_i)ds}}$$

$$+\underset{\text{Piece C: residual}}{\underbrace{\int \frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}r(s)ds}}.$$

$$f(x^{\star },x_i):=\int \frac{K_h(s-x^{\star })}{{\hat{p}}_R(s)}{\varphi }_{\rho }(s;x_i)ds,$$

$$\sum _{i=1}^{|T|}f(x^{\star },x_i)=\underset{=\Psi (h,\rho ,x^{\star })}{\underbrace{f(x^{\star },x_j)}}+\underset{=B(x^{\star })}{\underbrace{\sum _{i\ne j}f(x^{\star },x_i)}},$$

Full text

We propose Generative Likelihood Ratio Attack (Gen-LRA), a novel, computationally efficient No-Box MIA that, with no assumption of model knowledge or access, formulates its attack by evaluating the influence a test observation has on a surrogate model’s estimate of a local likelihood ratio over the synthetic data. We develop a theoretical framework for the attack: we show that the Gen-LRA score admits a closed-form characterization as a localized density-ratio statistic, and we prove that under a general model of local overfitting it produces a provable mean-score gap between members and non-members, yielding testable predictions for when the attack should succeed. We validate these predictions in a controlled simulation study and assess Gen-LRA against a comprehensive benchmark spanning diverse datasets, generative model architectures, and attack parameters. Across metrics, Gen-LRA consistently dominates competing MIAs, with especially strong gains at low false positive rates. These results underscore Gen-LRA’s effectiveness as a privacy auditing tool for the release of synthetic data, and highlight the significant privacy risks posed by generative model overfitting in real-world applications.

membership inference attacks, synthetic data, privacy auditing, generative models, tabular data, density estimation, likelihood ratio

††conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY††isbn: 978-1-4503-XXXX-X/2018/06††ccs: Security and privacy Domain-specific security and privacy architectures††ccs: Computing methodologies Machine learning approaches††ccs: Security and privacy Data anonymization and sanitization

1. Introduction

Real-world tabular data is often privacy-sensitive to the individual observations that compose these samples, hindering their ability to be shared in open-science efforts that can aid in new research and improve reproducibility. A promise of generative modeling is that models trained on sensitive data can produce samples that preserve the privacy of the training set while maintaining much of its intrinsic statistical information, enabling responsible release to a third party. In practice, a wide array of methodologies have been proposed to accomplish synthetic data release involving modifying loss functions (Abadi et al., 2016; Wang et al., 2022), creating new generative model architectures (Yoon et al., 2019, 2020a), and studying data release strategies (Hardt et al., 2012; Gupta et al., 2012; Takagi et al., 2021) to provide differential privacy guarantee. In another direction, a variety of methods have been proposed that maximize the fidelity of synthetic data and argue that privacy is satisfied through ad-hoc similarity metrics (Zhao et al., 2021; Guillaudeux et al., 2022; Liu et al., 2023; Solatorio and Dupriez, 2023).

To audit the empirical privacy of synthetic data generators, membership inference attacks (MIAs) have recently been extended from traditional machine learning models to synthetic tabular data. Here, privacy auditing is framed as an adversarial game: given specific constraints defined by a threat model, an attacker attempts to determine whether a test observation belongs to a model’s training dataset exploiting some notion of model failure (Shokri et al., 2017; Chen et al., 2020; Carlini et al., 2021). A successful attack represents a concrete privacy breach with clear real-world implications, where other similarity-based metrics have been shown to fail to capture privacy risk (Platzer and Reutterer, 2021; Ganev and Cristofaro, 2023; Ward et al., 2024).

While promising, MIAs for generative models and synthetic data release have seen limited success. Previous work has often relied on distance- or density-based heuristics, or has made additional assumptions about model query access that are unrealistic to the release setting and do not scale computationally to modern architectures. Moreover, most existing attacks are proposed without a theoretical account of what they measure or when they should succeed, making it difficult to assess whether empirical performance reflects a principled design or a coincidence of benchmark choice. We address both gaps in this work.

We focus on studying membership inference for the release of synthetic data in a No-Box Threat Model (Houssiau et al., 2022). In this setting, we make no adversarial assumptions of knowledge about model architecture, access, or training parameters, mimicking real-world scenarios of parties following best practices for releasing synthetic data in domains like healthcare and finance. Under this threat model, we derive a powerful MIA called Generative Likelihood Ratio Attack (Gen-LRA), which constructs an influence function formulated from likelihood ratio estimation to target privacy leakage that occurs through model overfitting. We develop a theoretical framework that characterizes what Gen-LRA computes and predicts when it succeeds, validate these predictions in a controlled simulation study, and demonstrate empirical dominance across a large-scale benchmark of 1,525 synthetic dataset configurations.

Contributions:

1. (1)

A novel attack. We introduce Gen-LRA, a No-Box MIA that formulates membership inference as the influence of a test point on a surrogate model’s estimate of the likelihood ratio over a local subset of synthetic data. To our knowledge, this is the first MIA for tabular synthetic data to explicitly adapt an influence-function framework to the No-Box setting. 2. (2)

Theoretical framework. We develop three theoretical results supporting Gen-LRA: (i) an invariance property showing that the log-likelihood-ratio score is insensitive to invertible reparametrizations of the data; (ii) a closed-form characterization showing that Gen-LRA is, to leading order, a localized density-ratio statistic, making explicit what the attack measures and why it differs from prior density-based MIAs; and (iii) a mean-score-gap result that, under a general model of local overfitting, produces a provable separation between members and non-members and identifies the structural quantities that govern the attack’s success. 3. (3)

Simulation validation. We employ a controlled simulation that directly instantiates the overfitting model, validating our theoretical results and characterizing when membership inference is and is not feasible. We find Gen-LRA is uniquely tunable across overfitting regimes, and the attack outperforms existing methods particularly when a generator approximately memorizes its training data. 4. (4)

Empirical benchmark. We show that Gen-LRA broadly outperforms competing MIAs across a benchmark of 35 datasets, 9 generative architectures, and 9 attacks, comprising 1,525 unique synthetic dataset configurations and over 10,000 attack runs. Gen-LRA is the top-ranked attack on more than $2\times$ as many runs as any baseline across both AUC and TPR at all fixed false positive rates (Table 1), and achieves the highest mean values across the top 100 highest-leakage runs on every metric (Table 2). We additionally perform ablations across encoding strategies, surrogate density estimators, and locality and bandwidth choices, finding that Gen-LRA is robust across these design decisions.

2. Membership Inference Attacks Preliminaries

In this work, we study the Membership Inference Attack Game in the context of synthetic data release. The objective of this game is to determine whether a particular data point was included in the original training dataset by examining the outputs of a generative model. We first introduce the formal definition of the Membership Inference Attack Game modeled after (Shokri et al., 2017).

2.1. Membership Inference Attack Game

Definition (Membership Inference Attack Game)

The game proceeds between a challenger $\mathcal{C}$ and an adversary $\mathcal{A}$ as follows:

1. (1)

The challenger samples a training dataset $T=\{x_{i}\}_{i=1}^{n}$ from the population distribution $x_{i}\sim\mathbb{P}$ over the domain $\mathcal{X}=\mathcal{X}_{1}\times\cdots\times\mathcal{X}_{d}$, where each attribute domain $\mathcal{X}_{j}$ is either numeric (i.e., $\mathcal{X}_{j}\subseteq\mathbb{R}$) or categorical (i.e., $\mathcal{X}_{j}$ is a finite discrete set), and uses $T$ to train a tabular generative model $G\leftarrow\mathcal{T}(T)$. The generative model $G$ produces synthetic dataset $S$. 2. (2)

The challenger flips a bit $b\in\{0,1\}$. If $b=0$, the challenger samples a test observation $x^{\star}\in\mathcal{X}$ from the population distribution $\mathbb{P}$. Otherwise, the challenger selects the test observation $x^{\star}$ from the training set $T$. 3. (3)

The challenger sends the test observation $x^{\star}$ to the adversary $\mathcal{A}$. 4. (4)

The adversary has access to some information defined by a threat model and uses this information to output a guess $\hat{b}\leftarrow\mathcal{A}(x^{\star})$. 5. (5)

The output of the game is $1$ if $\hat{b}=b$, and [math] otherwise. The adversary wins if $\hat{b}=b$, i.e., if it correctly identifies whether $x^{\star}$ was part of the training set $T$ or a sampled point from $\mathbb{P}$.

Adversary’s goal and capabilities.

The adversary $\mathcal{A}$ aims to determine whether a specific data point $x^{\star}$ was part of the original training dataset $T$ or was drawn from the population distribution $\mathbb{P}$. The adversary can use any available information to construct a scoring function that classifies the membership of $x^{\star}$ defined by a threat model. The performance of this classifier, evaluated with binary classification metrics, measures the privacy leakage of the training data for $G$ through $S$. Formally, a membership inference attack can be expressed as

[TABLE]

where $\mathbb{I}$ is the indicator function, $f(x^{\star})$ is a scoring function of $x^{\star}$, and $\gamma$ is an adjustable decision threshold.

2.2. Threat Model

In this work, we consider a “No-Box” (Houssiau et al., 2022) threat model in which the adversary has no access to the internal structure, parameters, or sampling mechanism of the generative model. The attack must instead be constructed using only two observed datasets: the released synthetic dataset $S\sim G(T)$, and an independently collected reference dataset $R\sim\mathbb{P}$ drawn from the same underlying population. The adversary is not granted access to knowledge of the model implementation nor can they issue queries to a trained generator. This reflects deployment scenarios in which organizations release synthetic data for downstream analysis while keeping all model knowledge confidential. The synthetic dataset $S$ serves as the only potential leakage surface, and the reference set $R$ provides a statistical anchor for the population. A reference dataset is commonly assumed in No-Box attacks on synthetic data (Chen et al., 2020; Houssiau et al., 2022; van Breugel et al., 2023; Ward et al., 2024) as well as in MIAs for supervised learning models (Carlini et al., 2021; Ye et al., 2022; Sablayrolles et al., 2019), and represents a scenario in which an adversary may be able to find comparable data in the real world—for example via open-source datasets, paid collection, or domain knowledge.

Three considerations motivate our focus on this threat model. First, it is the most realistic setting for synthetic data release. Shadow-box attacks (Stadler et al., 2022; Houssiau et al., 2022; Meeus et al., 2024) rely on the adversary knowing the generator’s architecture or implementation in order to train surrogate models, but a defender following release best practices can trivially neutralize such attacks by simply withholding this information (Golob et al., 2024). The estimated privacy leakage of these attacks is therefore not conditioned on the knowledge an adversary would realistically have. Second, shadow-box and white-box approaches do not scale computationally to modern tabular generators. Diffusion-based and transformer-based architectures can take hours to train for a single model, and attacks that require fitting hundreds of surrogate models per auditing run become infeasible as the generator’s training cost grows. Third, No-Box attacks study leakage through the synthetic data itself rather than through any particular model, making them model-agnostic. This property makes them useful both as a practical privacy-auditing tool that a third party can run on released synthetic data, and as a research tool for understanding the privacy behavior of generative models independent of their specific architectures.

3. Related Works

3.1. Assessing Overfitting in Tabular Generative Models

Several measures have been developed to assess the fitness of tabular synthetic data, particularly from a privacy perspective. These metrics generally aim to measure the similarity between the training and synthetic datasets, with the ideal outcome being that the synthetic data is neither too similar to the training data nor too different. A widely used metric for this purpose is the Distance to Closest Record111DCR in the similarity-metric case compares a training point to a synthetic point. Chen et al. (2020) propose an MIA in which the scoring function is a distance computation for a test point and a synthetic point; in all other sections of the paper we use DCR to refer to the MIA. (Park et al., 2018; Liu et al., 2023; Lu et al., 2019; Yale et al., 2019; Zhao et al., 2021; Guillaudeux et al., 2022), which measures the distance from each training point to its nearest neighbor in the synthetic dataset and averages across training points. Another commonly used metric is the Identical Matching Score (IMS) (Lu et al., 2019; AI, 2020, 2021), which measures the proportion of identical records between the training and synthetic datasets. While these measures can be useful for describing overfitness from a distribution-level quality or model-generalization perspective, they do not characterize privacy risk: there is no assumed threat model and they are not evaluated against non-member examples.

Exploiting overfitting as a source of privacy leakage has been documented in specific generators. van Breugel et al. (2023) showed that TVAE (Xu et al., 2019) overfits to minority-class examples in a medical training dataset, leaking their privacy. Ward et al. (2024) found that Tab-DDPM (Kotelnikov et al., 2022) heavily replicates training records from certain demographic subgroups when generating synthetic data for the Adult dataset. These findings motivate our approach of designing an attack that explicitly targets the local overfitting behavior of tabular generative models (see Section 4).

3.2. MIAs for Tabular Generative Models

Membership inference attacks explicitly characterize the empirical privacy risk of a machine learning model (Song and Mittal, 2020; Yeom et al., 2018). MIAs were originally developed for attacking supervised learning classifiers (Shokri et al., 2017), where the general idea is to query a model with different observations to learn patterns in its class-probability outputs. Membership is then inferred by comparing the outputs of the model to outputs from reference models in various ways (Carlini et al., 2021; Long et al., 2020; Sablayrolles et al., 2019; Watson et al., 2022; Ye et al., 2022; Zarifzadeh et al., 2024).

To adapt to the structural differences of generative models, a range of MIAs for tabular generators have been proposed that employ different threat models and strategies (Chen et al., 2020; Hayes et al., 2017; Hilprecht et al., 2019; Houssiau et al., 2022; Meeus et al., 2024; Stadler et al., 2022; van Breugel et al., 2023; Ward et al., 2024). Gen-LRA is most closely related to DOMIAS (van Breugel et al., 2023) and to a line of work that extends query-based attacks to tabular generative models (Houssiau et al., 2022; Meeus et al., 2024; Stadler et al., 2022).

Relation to DOMIAS

DOMIAS follows the same threat-model assumptions as Gen-LRA and similarly defines its scoring function in (1) as a density ratio, $\frac{p_{S}(x^{*})}{p_{R}(x^{*})}$. However, its theoretical foundation is limited because it effectively tests the wrong membership hypothesis: rather than testing whether the inclusion of $x^{*}$ in the reference data influences the released synthetic dataset, it tests only whether $x^{*}$ lies in a region where the synthetic distribution is denser than the reference distribution. In contrast, Gen-LRA directly targets the membership inference problem by measuring the effect of specifically including $x^{*}$ on $R$, angling with the true hypothesis of interest. Moreover, while DOMIAS relies on a single-point density estimate, Gen-LRA aggregates evidence over a local region, allowing it to capture diffuse memorization signals and incorporate substantially more information. Our theoretical analysis, simulations, and empirical benchmarks all support this distinction: because DOMIAS tests the wrong hypothesis, its performance degrades substantially in a variety of overfitting regimes, whereas Gen-LRA remains robust and consistently outperforms it.

Relation to query-based attacks

(Stadler et al., 2022), Houssiau et al. (2022), and Meeus et al. (2024) propose query-based attacks on tabular generators that additionally assume the adversary has knowledge of the implementation of the target model. In these methods, an attacker trains many versions of the model on $R\cup\{x^{\star}\}$ and $R$, generates many synthetic datasets, represents each synthetic dataset by summary statistics or histograms, and trains a classifier to distinguish between the two regimes.

Gen-LRA improves on these attacks in two ways. First, they are unsuitable for auditing privacy in a release setting because they are trivially defeated if the defender chooses not to disclose the architecture; indeed, Golob et al. (2024) has shown significant privacy leakage arising from architectural disclosure, leading to the recommendation that data-releasing parties disclose as little model information as possible. Gen-LRA makes no assumption about model implementation. Second, query-based attacks are computationally expensive: they require training $(N_{\text{test}}+1)\times N_{\text{surrogate}}$ separate models to audit a single generator, which is impractical as large diffusion and language model architectures become more common. Gen-LRA requires only $N_{\text{test}}+1$ density estimators to be fit, which is substantially cheaper.

4. Generative Likelihood Ratio Attack

In this section, we propose Generative Likelihood Ratio Attack (Gen-LRA), a membership inference attack designed to detect membership leakage in synthetic data through a statistical notion of likelihood influence. Unlike prior MIAs that evaluate the density or distance of a test point itself, Gen-LRA frames membership inference as a measurement of how much the test point $x^{\star}$ influences an estimate of the likelihood of $S$. The central idea is that if $x^{\star}$ belonged to the training set and the generative model is overfit, then adding $x^{\star}$ to a surrogate density estimator should increase the estimated likelihood of the synthetic dataset.

We develop Gen-LRA in six steps. Section 4.1 reviews empirical influence functions, which provide the theoretical foundation for the attack. Section 4.2 defines the Gen-LRA score as a log-likelihood-ratio influence function. Section 4.3 justifies the log-likelihood-ratio form via Neyman-Pearson optimality and an invariance property. Sections 4.4 and 4.5 establish the two main theoretical results of the paper: a closed-form characterization of the Gen-LRA score, and a mean-score-gap result quantifying the attack’s power under a general overfitting model. Section 4.6 describes the practical implementation, with each design choice motivated by the preceding theory.

4.1. Empirical Influence Functions

Influence functions, originally developed in the field of Robust Statistics (Hampel, 1974; Cook and Weisberg, 1986), measure how statistical estimates change when the underlying data distribution is perturbed. The influence function for an estimator $\theta$ applied to a distribution $F$ is

[TABLE]

where $\delta_{x^{\star}}$ is the Dirac measure placing unit mass at $x^{\star}$. This quantity captures the sensitivity of $\theta$ to infinitesimal perturbations of $F$ at $x^{\star}$.

In the empirical setting with finite samples $\mathcal{D}=\{x_{1},\ldots,x_{n}\}$, the empirical influence function is defined by evaluating how an estimate changes when a point is added:

[TABLE]

For supervised learning models, this difference is typically measured on loss or empirical risk (Koh and Liang, 2017). In an MIA for tabular generative models with no model access, however, measures of loss are not available. Rather than examining how $x^{\star}$ affects model parameters, Gen-LRA considers the influence of $x^{\star}$ on the likelihood assigned to generated samples $S$ by a surrogate estimator.

4.2. Likelihood Influence as an Attack Surface

Recalling that $T,R\sim\mathbb{P}$ and that our goal is to infer whether $x^{\star}\in T$ given $S$ and $R$, we hypothesize that if the generator is overfit 222We use overfitting for the generator’s behavior of placing excess mass near training points relative to the true population, and memorization for its geometric manifestation in the synthetic data. Both notions are formalized in Section 4.5. then $S$ carries signal of $x^{\star}$ specifically when $x^{\star}\in T$. Gen-LRA measures this overfitness by formalizing an influence function on the estimated log-likelihood of $S$ under two surrogate models: one fit on the reference dataset $R$, and another fit on the augmented dataset $R\cup\{x^{\star}\}$:

[TABLE]

Intuitively (see Figure 1), if adding $x^{\star}$ to $R$ substantially increases the estimated likelihood of $S$, this suggests $x^{\star}$ contributed to the generative process. If the likelihood is unchanged or decreases, it implies $x^{\star}\notin T$. Using $\hat{\mathcal{I}}(x^{\star};R,S)$ as the scoring function $f$ in the membership prediction rule (1) defines the Gen-LRA attack.

4.3. Justification of the Log-Likelihood Ratio

The influence function in (4) is one of many possible functionals of $\hat{p}(S\mid R\cup\{x^{\star}\})$ and $\hat{p}(S\mid R)$. Two properties, one statistical and one geometric, justify the choice of the log-likelihood ratio.

Neyman-Pearson optimality

The membership inference game is fundamentally a binary hypothesis test: $H_{0}:x^{\star}\sim P$ against $H_{1}:x^{\star}\in T$. By the Neyman-Pearson lemma, the likelihood ratio test is uniformly most powerful among all tests at a given false positive rate. This principle has long been central to the MIA literature: Shokri et al. (2017) frame membership inference as a classification problem whose Bayes-optimal solution reduces to a likelihood ratio, Sablayrolles et al. (2019) derive the Bayes-optimal MIA strategy and show it takes a likelihood-ratio form, and Ye et al. (2022) argue for calibrated likelihood-ratio statistics as a general design principle.

Carlini et al. (2021) make this argument operational, demonstrating that MIAs grounded explicitly in likelihood ratios (rather than ad-hoc distance, confidence, or loss statistics) achieve substantially stronger guarantees in the supervised-learning setting, particularly at the low false positive rates that matter most for practical privacy auditing. Subsequent work has extended this principle to enhanced and low-cost variants (Ye et al., 2022; Zarifzadeh et al., 2024). Gen-LRA adapts this design principle to the No-Box tabular synthetic data setting: the scoring function in (4) is a log-likelihood ratio over $S$ under two hypotheses about the data distribution that generated it, making it the natural Neyman-Pearson analogue for the attack surface available to a No-Box adversary.

Invariance to reparametrization

A second advantage of the log-likelihood-ratio form is that it is invariant to invertible transformations of the data.

Theorem 4.1 (Invariance of the population influence function).

Let $S$ and $R$ be sets of samples and $x^{\star}$ a test point, with probability distributions on $\mathcal{X}\subseteq\mathbb{R}^{d}$. For any continuously differentiable invertible function $g:\mathcal{X}\to\mathcal{X}$ with non-vanishing Jacobian, the population-level log-likelihood-ratio influence function is invariant:

[TABLE]

The proof (Appendix A.1) follows from the change-of-variables formula and cancellation of Jacobian terms. This invariance is practically important: tabular data is typically preprocessed through one-hot encoding, scaling, or learned embeddings, and an attack whose score depends on these arbitrary choices would be unreliable. Competing attacks for example based on distance heuristics such as Distance to Closest Record do not satisfy this property.

4.4. Closed-Form Characterization of the Attack Score

Having justified the log-likelihood-ratio form, we now analyze what Gen-LRA computes. To make this analysis concrete, we specialize to a particular choice of surrogate density estimator and localization strategy. Throughout the remainder of this section, we take $\hat{p}_{R}$ to be a Gaussian Kernel Density Estimator (KDE) fit on the reference set $R$, and we localize the attack statistic to the $k$-nearest synthetic neighbors of $x^{\star}$ rather than summing over all of $S$.

Both of these choices are justified empirically and discussed in full in Section 4.6; briefly, Gaussian KDEs admit a closed-form expansion that makes the subsequent analysis tractable and empirically outperform more expressive density estimators (Section 7.3), while localization concentrates the statistic on the region where overfitting signal can exist.

Formally, let $\hat{p}_{R}$ denote a Gaussian KDE fit on reference set $R$ with $|R|=m$ and bandwidth $h$, let $K_{h}(u)=h^{-d}K(u/h)$ be the Gaussian kernel in $\mathbb{R}^{d}$, and let $S_{k}(x^{\star})\subseteq S$ be the set of $k$ synthetic points in $S$ nearest to $x^{\star}$. The Gen-LRA score is then

[TABLE]

Our first theoretical result characterizes this quantity in closed form.

Theorem 4.2 (Closed-form approximation).

Suppose $\hat{p}_{R}(s)\geq c>0$ for all $s\in S_{k}(x^{\star})$. Then, for any fixed $k$ or any $k=o(m)$, the Gen-LRA score satisfies

[TABLE]

The proof (Appendix A.2) follows from an exact identity for the augmented KDE together with a Taylor expansion of the logarithm. Theorem 4.2 shows that Gen-LRA is, to leading order, a sum of localized density-ratio-like quantities. The dominant term $K_{h}(s-x^{\star})/\hat{p}_{R}(s)$ is large precisely when a synthetic point $s$ is close to $x^{\star}$ (numerator) and lies in a region of low reference density (denominator). This is the fingerprint of memorization: training-induced synthetic points that cluster near $x^{\star}$ in regions the population would not naturally populate.

Comparison to DOMIAS

DOMIAS (van Breugel et al., 2023) uses the scoring function $\hat{p}_{S}(x^{\star})/\hat{p}_{R}(x^{\star})$—a single density ratio evaluated at $x^{\star}$. Equation 7 shows that Gen-LRA evaluates an analogous ratio at $k$ synthetic points correlated with memorization of $x^{\star}$, rather than at $x^{\star}$ itself. Gen-LRA thus aggregates evidence over a region where the memorization signal is expected to concentrate, while DOMIAS uses only a single-point estimate. This provides a formal explanation for why Gen-LRA and DOMIAS differ despite following the same threat model.

Role of localization

The closed form in (7) makes clear why localization is essential. The summand is appreciable only at synthetic points $s$ near $x^{\star}$; points far from $x^{\star}$ contribute $K_{h}(s-x^{\star})\approx 0$ and thus only the $-1$ baseline term, which adds noise without signal. Restricting the sum to the $k$-nearest synthetic neighbors of $x^{\star}$ discards these noise-only terms and concentrates the statistic on the region where signal can exist. The parameter $k$ thus serves as a bias-variance trade-off: larger $k$ reduces variance by averaging more terms, while smaller $k$ reduces bias by excluding neighbors outside the memorization region.

4.5. Power Under a Local Overfitting Model

Theorem 4.2 characterizes what Gen-LRA computes; it does not by itself guarantee that the computation separates members from non-members. We now establish that under a general model of local overfitting, the expected Gen-LRA score is provably larger for members than for non-members, and we characterize the conditions under which the gap is largest.

Definition 4.3 (Local overfitting).

A generator $G$ trained on $T$ exhibits local overfitting at scale $\rho$ with strength $\epsilon$ if its induced density $p_{G}$ decomposes as

[TABLE]

where $p$ is the population distribution, $|T|$ denotes the cardinality of the training set, $\phi_{\rho}(\cdot\,;x_{i})$ is a non-negative excess mass function concentrated within distance $\rho$ of $x_{i}$ with $\int\phi_{\rho}(s;x_{i})\,ds=1$, $\epsilon\in[0,1]$ is the overfitting strength, and $r(s)$ is a residual satisfying $\int r(s)\,ds=-\epsilon$.

The formulation is deliberately general: $\phi_{\rho}$ can be any concentrated non-negative function, a sharp spike for exact replication, a diffuse bump for noisy memorization, or an asymmetric shape reflecting the generator’s inductive bias, and $r$ absorbs any displacement of mass from elsewhere in the distribution. The decomposition does not require a specific generator or model; it is purely a description of its output density.

Theorem 4.4 (Mean score gap).

Let $\mu_{1}=\mathbb{E}[\hat{\mathcal{I}}(x^{\star};R,S)\mid x^{\star}\in T]$ and $\mu_{0}=\mathbb{E}[\hat{\mathcal{I}}(x^{\star};R,S)\mid x^{\star}\sim P]$ denote the expected Gen-LRA scores for members and non-members, respectively. Under Definition 4.3 and the regularity conditions of Theorem 4.2,

[TABLE]

where the signal term is

[TABLE]

and $\Psi(h,\rho,x^{\star})>0$ whenever the support of $\phi_{\rho}(\cdot;x^{\star})$ overlaps with the kernel $K_{h}(\cdot-x^{\star})$.

We defer the proof to Appendix A.3. The argument proceeds by substituting the overfitting decomposition (8) into the leading-order expression from Theorem 4.2, and showing that the contributions from the population density $p$ and the residual $r$ cancel in the difference $\mu_{1}-\mu_{0}$, leaving the self-overlap $\Psi$ as the dominant signal.

Theorem 4.4 states that Gen-LRA has nontrivial power against any generator exhibiting local overfitting with $\epsilon>0$. The size of the mean gap depends on three interpretable quantities, each governing a distinct aspect of the attack’s behavior:

1. (1)

Linear dependence on the overfitting strength $\epsilon$. The mean gap of Gen-LRA scales linearly with $\epsilon$: generators that overfit more aggressively are more vulnerable to Gen-LRA, and generators that do not, for example those trained with strong differential privacy guarantees, to which $\phi_{\rho}$ must be nearly flat, should yield low-quality attacks. 2. (2)

Localization to the memorization region. The score in (7) approximates $\Psi$ by summing the integrand over the $k$ nearest synthetic neighbors of $x^{\star}$, so $k$ controls how many synthetic points are included in the sum. When $k$ is smaller than the number of synthetic points the generator places within distance $\sim\rho$ of $x^{\star}$, the sum ignores signal; when $k$ is larger, the additional neighbors lie where $K_{h}(s-x^{\star})$ is negligible and contribute only additional noise. The optimal $k$ therefore depends on how the generator distributes its excess mass across synthetic points. 3. (3)

Bandwidth-scale matching. The signal term $\Psi(h,\rho,x^{\star})$ is a kernel-overlap integral between $K_{h}$ and the $\phi_{\rho}$, and is non-negligible only in a neighborhood of $x^{\star}$ on the scale of $\rho$. The bandwidth $h$ controls the resolution at which the KDE recovers the local density ratio $p_{R\cup\{x^{\star}\}}/p_{R}$ in this neighborhood: $h\gg\rho$ oversmooths and washes out the perturbation from adding $x^{\star}$, while $h\ll\rho$ produces an estimate of $\hat{p}_{R}$ that adds noise to the score. Because the attack signal is local, estimators with explicit control over a local smoothing scale are better attack instruments than estimators optimized for global density-estimation accuracy.

We study the empirical behavior of these quantities under controlled conditions in Section 5, and connect them to Gen-LRA’s performance on real generators in Section 6.

4.6. Implementation Choices

The theoretical results above motivate each component of the Gen-LRA implementation (full pseudocode is given in Alg. 1).

Surrogate model

We use Gaussian KDEs as the surrogate density estimator. This choice is motivated by Theorem 4.2: the closed-form expansion depends on Gaussian-kernel algebra, making Gen-LRA with KDEs analytically tractable and computationally cheap to run. KDEs are widely available and, per the remark following Theorem 4.4, their locally structured bias profile aligns with the task of detecting memorization at scale $\rho\approx h$. We include an ablation and discussion where we study replacing KDEs with a flow-based deep learning method (see Section 7.3), but find KDE empirically outperforms it. We set the bandwidth using Silverman’s rule per results we find in Section 5. An ablation on encoding strategies for heterogeneous tabular data appears in Appendix C.1; we that ordinal encoding for categorical variables outperforms other methods and thus choose it as a default.

Choice of k

The choice of $k$ governs a bias-variance trade-off (Section 4.4), and it plays different roles in the two settings Gen-LRA is used in. A realistic adversary cannot tune $k$ against ground-truth membership labels and must commit to a single value; the ablation in Appendix C.3 shows that $k=10$ is robust across datasets and generators, and we adopt it as the default. A privacy auditor, by contrast, has access to the true membership labels by construction and seeks the tightest possible bound on the generator’s vulnerability. For a given $x^{\star}$, the dominant cost is the $2k$ KDE evaluations against reference sets of size $m$ and $m+1$, giving a per-query cost of $\mathcal{O}(k\cdot m\cdot d)$. Computing scores for all $k^{\prime}\in\{1,\ldots,k\}$ requires no additional KDE evaluations: the per-neighbor log-density-ratio terms can be summed cumulatively in $\mathcal{O}(k)$ time. Sweeping $k$ exhaustively up to $k_{\max}$ is therefore asymptotically equivalent to a single evaluation at $k_{\max}$, at cost $\mathcal{O}(k_{\max}\cdot m\cdot d)$. An auditor can thus report the strongest configuration over $k$, while our adversary defaults to $k=10$.

Decision threshold

The membership prediction rule (1) requires a decision threshold $\gamma$. In principle $\gamma$ is tunable for a specific application, but $\hat{\mathcal{I}}(x^{\star};R,S)>0$ implies some degree of local overfitting to $x^{\star}$ and serves as a natural threshold.

5. Simulation Study

Before benchmarking Gen-LRA on real tabular generators (Section 6), we conduct a controlled simulation study to directly examine the structural properties of Theorems 4.2 and 4.4. The motivation is that real generators confound three questions: whether the attack works, under what conditions it works, and whether the parameters $(\epsilon,\rho)$ are meaningful in practice. A simulation lets us address all three under controlled conditions, and additionally study how the linearity, bandwidth-scale and localization properties from Section 4.5 manifest empirically.

5.1. Simulation Design

We directly instantiate the local overfitting model of Definition 4.3. Concretely, we fix a population distribution $P=\mathcal{N}(0,I_{d})$ and sample $T$, a held-out non-member set, and $R$ independently from $P$, each of size $n=500$. To generate the synthetic set $S$, each of $n$ synthetic points is drawn as

[TABLE]

This directly implements Equation 8 with $\phi_{\rho}(\cdot;x_{j})=\mathcal{N}(\cdot;x_{j},\rho^{2}I_{d})$, making $\epsilon$ and $\rho$ controllable configurations rather than latent properties of a trained model. We use $d=40$, consistent with moderate-dimensional tabular data.

Attacks.

We evaluate Gen-LRA against two common threat-model-compatible baselines: DOMIAS (van Breugel et al., 2023), which computes a pointwise density ratio $\frac{\hat{p}_{S}(x^{\star})}{\hat{p}_{R}(x^{\star})}$, and DCR (Chen et al., 2020), which scores test points by distance to the nearest synthetic neighbor. We additionally include two variants of Gen-LRA, one with $k=1$ (single-neighbor localization) and one with $k=10$ (the default used in our benchmark experiments). Including both variants lets us isolate the role of the localization parameter $k$ in shaping the attack’s global and tail behavior.

Experimental grid.

We compute $\epsilon\in\{0,0.25,0.5,0.75,1.0\}$ and $\rho\in\{0.1,0.2,\ldots,1.7\}$ over 10 random seeds per grid point. For each $(\epsilon,\rho,\text{seed})$ combination, we compute attack performance using Area Under the Curve (AUC) and True Positive Rate at the Fixed False Positive Rate of 0 (TPR@FPR=0) (Carlini et al., 2021) by scoring all members and held-out non-members. This yields a measurement of both the global attack performance and the calibration of the classifier under various privacy leakage settings.

5.2. Simulation Results

Figure 2 presents four primary results. The top row reports mean AUC; the bottom row reports mean TPR@FPR=0 with respective error bars. The left column varies $\epsilon$ at fixed memorization scale; the right column varies $\rho$ at fixed overfitting strength.

5.2.1. Linear Dependence on $\epsilon$

Panels (a) and (c) report AUC and TPR@FPR=0 as functions of $\epsilon$ at fixed memorization scale $\rho=1$. Both metrics scale linearly with $\epsilon$ for all four attacks, validating Theorem 4.4’s prediction that the mean score gap is linear in the overfitting strength. Gen-LRA achieves the largest slope on both metrics, with $k=10$ reaching AUC $\approx 0.81$ at $\epsilon=1$ on panel (a) and $k=1$ and $k=10$ both reaching TPR@FPR=0 $\approx 0.40$ on panel (c).

The advantage over baselines is substantially larger for TPR@FPR =0 than AUC: at $\epsilon=1$, Gen-LRA’s TPR@FPR=0 is roughly $2\times$ that of DCR and $4\times$ that of DOMIAS, whereas the corresponding AUC ratios are smaller. This reflects the structural advantage of Gen-LRA’s likelihood-ratio formulation. The theorem does not predict particular slope magnitudes, but the empirical ordering matches the structural argument in Section 4.4: aggregating evidence over $k$ synthetic neighbors yields a stronger linear response than the single-point attacks DOMIAS and DCR.

5.2.2. Localization and Three Regimes of Memorization

Panels (b) and (d) sweep $\rho$ at fixed $\epsilon=0.5$. Recall that $\rho$ is the scale of noise added to a training point when producing a memorized synthetic point: small $\rho$ means synthetic points are nearly exact copies of training points, large $\rho$ means they are heavily perturbed. The curves divide into three regimes that correspond to different types of empirical memorization.

Near-exact memorization ($\rho\lesssim 0.7$). Synthetic points memorized from training data are nearly identical to their source records. For a member $x^{\star}\in T$, this produces a single synthetic point $s_{1}$ that is a near-copy of $x^{\star}$, and adding $x^{\star}$ to the reference set creates a sharp, narrow peak in $\hat{p}_{R\cup\{x^{\star}\}}$ centered at $x^{\star}$. The likelihood ratio $\hat{p}_{R\cup\{x^{\star}\}}(s)/\hat{p}_{R}(s)$ carries useful signal only for synthetic points near $x^{*}$; elsewhere the two density estimates are nearly identical and the ratio is uninformative. DOMIAS, DCR, and Gen-LRA $k=1$ target this peak. All three capture the full membership signal and achieve comparable AUC ($\approx 0.69$) and TPR@FPR=0 ($\approx 0.40$). Gen-LRA $k=10$ underperforms on TPR@FPR=0 here as included additional synthetic neighbors are unrelated to $x^{\star}$ and their contributions effectively add noise to the score.

Approximate memorization ($\rho\in[0.7,1.5]$). Synthetic points retain a recognizable relationship to training points but are spread out enough that no single synthetic point is a reliable copy of any given training point. The peak in $\hat{p}_{R\cup\{x^{\star}\}}$ is still localized to $x^{\star}$, but probability mass in $S$ is no longer concentrated at $x^{*}$: instead, several synthetic points sit near $x^{\star}$, each carrying partial evidence. DOMIAS and DCR collapse in this regime. DOMIAS’s single-point density ratio loses signal because the synthetic data is no longer concentrated at $x^{\star}$, and DCR loses signal because the closest synthetic point to a member is no longer reliably closer than the closest synthetic point to a non-member. Gen-LRA $k=1$ also degrades but at a lower rate than DCR and DOMIAS. Gen-LRA $k=10$ however, aggregates evidence across all the relevant synthetic neighbors and dominates on both AUC and TPR@FPR=0.

Diffuse memorization ($\rho\gtrsim 1.5$). Synthetic points are perturbed heavily. No local feature of $S$ distinguishes regions near training points from regions near non-training points, and all four attacks collapse to random performance.

5.2.3. The Role of Bandwidth

Figure 3 repeats the $\rho$ sweep from panels (b) and (d) for Gen-LRA at three bandwidths: $0.1\times$, $1\times$, and $10\times$ Silverman’s rule. The narrow ($0.1\times$) and default ($1\times$) bandwidths produce nearly indistinguishable curves on both AUC and TPR@FPR=0, indicating that Gen-LRA is robust to under-smoothing in this regime. The wide bandwidth ($10\times$) collapses Gen-LRA’s performance to near-random AUC and near-zero TPR@FPR=0 across all $\rho$. This corresponds with item (3) of Section 4.5: when $h\gg\rho$, the KDE oversmooths the perturbation introduced by adding $x^{\star}$ to $R$, washing out the local density ratio that Gen-LRA relies on; when $h\ll\rho$, the estimate is noisier but empirically retains the structural signal that the score depends on.

6. Experiments

The simulation study in Section 5 shows that different attacks and hyperparameter settings perform differently across memorization regimes. This raises a methodological concern: a small benchmark risks producing results that are artifacts of which regimes happen to dominate the selection, biasing the comparison toward attacks suited to those particular conditions. We therefore construct a large and diverse benchmark to assess Gen-LRA’s effectiveness across a broad distribution of synthetic datasets: 35 tabular datasets sampled from OpenML (Bischl et al., 2019), 9 tabular generative architectures, 9 membership inference attacks, and 5 random seeds. This yields 1,525 unique synthetic datasets and over 10,000 unique attack runs, spanning a wide range of memorization conditions that approximate what practitioners could encounter in deployment.

6.1. Benchmark Setup

For each dataset, following standard tabular synthetic data evaluation practice (Qian et al., 2023), we randomly partition the data without replacement into a training set $T$, holdout set $H$, and reference set $R$ in an 80/10/10 ratio. The generator is trained on $T$ and produces a synthetic dataset $S$ of size $|T|$. MIAs are then evaluated on a balanced test set composed of $\min(1000,|H|)$ members sampled from $T$ and an equal number of non-members sampled from $H$. Full details on datasets, MIAs, and generators are provided in Appendix B.

Gen-LRA and DOMIAS rely on density estimation, which we implement using Gaussian Kernel Density Estimation (KDE) with Silverman’s rule for bandwidth selection. We find KDE outperforms deep learning based density estimators on this task for Gen-LRA (Section 7.3). Because KDE handles one hot encoded categorical data poorly, we use ordinal encoding for these attacks; an ablation across Principle Component Analysis and Variational Autoencoder based encodings appears in Appendix C.1 and shows that ordinal encoding performs best. For all other attacks, numeric features are scaled and categorical features one hot encoded.

6.2. Baselines

We compare Gen-LRA with a fixed $k=10$ against eight competing MIAs that operate under compatible threat models: LOGAN, MC, DCR, DCR-Diff, Classifier, Local Neighborhood, DOMIAS, and DPI (Hayes et al., 2017; Hilprecht et al., 2019; Chen et al., 2020; Houssiau et al., 2022; van Breugel et al., 2023; Ward et al., 2024).

We evaluate across nine tabular generators spanning the major architectural families: PATEGAN ($\epsilon=1$) for differentially private generation; Ads-GAN, CTGAN, and TVAE for adversarial training; Normalizing Flows for likelihood-based generation; ARF for tree-based generation; Tab-DDPM and TabSyn for diffusion-based generation; and RealTabFormer (RTF) for transformer-based generation (Ankan and Panda, 2015; Yoon et al., 2019, 2020b; Xu et al., 2019; Durkan et al., 2019; Watson et al., 2023; Kotelnikov et al., 2022; Zhang et al., 2024; Solatorio and Dupriez, 2023). For RealTabFormer and TabSyn we use the original implementations with default hyperparameters; for all other architectures we use the default Synthcity (Qian et al., 2023) implementations and settings.

All experiments were conducted on an AWS G5.2xlarge EC2 instance. The main benchmark required approximately 500 hours of compute, with an additional 80 hours used for the deep learning density estimation experiments described in Section 7.3.

7. Experiment Results

We summarize benchmark performance along two complementary axes: aggregate ranking across all 1,525 synthetic datasets, and performance restricted to the top 100 runs by each metric. The aggregate ranking measures how often each attack is the strongest available choice across diverse synthetic datasets a practitioner may encounter. The top-100 view measures attack performance in the synthetic datasets with the greatest amount of detected leakage.

7.1. Aggregate Ranking

An adversary attacking an unknown generator must commit to a single attack as they have no ground truth labels, so a relevant question is which attack most often delivers the strongest signal across a diverse sample of synthetic datasets. Table 1 reports, for each attack and metric, the fraction of runs in which the attack ranks first (Top-1), the fraction of runs in which it ranks in the top three (Top-3), and the mean rank across all 1,525 runs. The Top-1 rate answers this question directly, and Gen-LRA is the dominant choice. Gen-LRA achieves the highest Top-1 rate on every metric (38.6% on AUC, 42.1% on TPR@FPR=0, and 46.2% on TPR@FPR=0.1), more than $2\times$ the rate of any baseline.

The closest competitors are DCR-Diff at 18.4% on TPR@FPR=0 and Classifier at 16.0% on AUC; no baseline exceeds a 19% Top-1 rate on any metric. Gen-LRA’s margin widens at higher false-positive rates, reaching $2.8\times$ the next-best attack at TPR@FPR=0.1, indicating that the benefit of aggregating evidence over $k$ synthetic neighbors is most pronounced at operating points permitting a moderate false-positive budget. The supporting Top-3 rates (71.7%–78.5% for Gen-LRA, against $\leq 60.3\%$ for any baseline) and mean ranks (2.54–2.83 for Gen-LRA, against $\geq 3.78$ for any baseline) confirm that this dominance is consistent.

7.2. Performance on the Most Vulnerable Runs

Aggregate ranking measures an adversary’s preference for an attack; we now turn to attack performance for runs that exhibited the highest detected leakage. Table 2 reports each metric’s mean and standard deviation over the top 100 runs by that metric, the regime in which the adversary’s attack choice has the largest practical consequence. Gen-LRA achieves the highest mean AUC (0.59) and the highest mean TPR at every false-positive rate, with the largest gap at low FPR: Gen-LRA’s mean TPR@FPR=0 of 0.05 is approximately $1.7\times$ that of DCR-Diff and $2.5\times$ that of DOMIAS.

We additionally report medians and interquartile ranges in the Appendix Table 6, which shows that the closest baseline performance is outlier driven. DCR and DCR-Diff have median TPR@FPR =0 values (0.011 and 0.024) well below their means, with IQRs touching zero. Gen-LRA’s median TPR@FPR=0 of 0.055 exceeds its own mean, its IQR is bounded away from zero ($[0.034,0.067]$), and its median AUC of 0.594 exceeds the third quartile of every baseline.

These outliers are concentrated on datasets generated by RealTabFormer, which accounts for 42 of the top 100 runs by AUC. Recently, (Ward et al., 2025) documented that RealTabFormer often produces near-exact copies of training records. The simulation in Section 5.2 identifies this small $\rho$ regime as the one in which distance based baselines are most competitive with Gen-LRA at the fixed $k=10$ used in our benchmark, implying that the top 100 runs overrepresent the regime least favorable to Gen-LRA, yet Gen-LRA leads on every metric in both mean and median.

7.3. Deep Learning Density Estimation

Gen-LRA estimates the likelihood of high dimensional data, a setting in which Gaussian Kernel Density Estimation (KDE) is typically outperformed by modern density estimators on metrics such as average negative log likelihood and negative evidence lower bound (De Cao et al., 2019; Wen and Hang, 2022). It is therefore worth examining whether replacing KDE with a more expressive estimator improves attack performance. Following (van Breugel et al., 2023), we perform an ablation (details in Appendix C.2) with Block Neural Autoregressive Flows (BNAF) (De Cao et al., 2019) as the surrogate density estimator for Gen-LRA.

Figure 4 shows that KDE matches or outperforms BNAF on both AUC and TPR@FPR=0.01, with the largest gap on the generators that exhibit the most extreme privacy leakage. While BNAF produces a tighter distribution of scores across runs, this stability comes from a consistent failure to detect the strongest leakage signals: BNAF’s upper tail tops out near 0.56 AUC, whereas KDE reaches above 0.65 on the same generators. The bandwidth analysis in Section 4.5 explains this result: the Gen-LRA signal is governed by the kernel overlap term $\Psi(h,\rho,x^{\star})$, which requires the surrogate’s local smoothing scale to match the memorization scale $\rho$. KDE controls this scale explicitly through its bandwidth. BNAF however optimizes for global density accuracy and exposes no analogous parameter, so its implicit smoothing scale is set by global training dynamics rather than matched to $\rho$. The result is a surrogate that is more expressive globally but less faithful in the local neighborhood where Gen-LRA’s signal is found.

A second issue compounds the first: Gen-LRA compares two surrogates fit on $R$ and $R\cup\{x^{\star}\}$, and recovering the influence of a single point requires the difference between them to reflect that point rather than training stochasticity. KDE is deterministic given its bandwidth and data, so the two fits differ only in the contribution of $x^{\star}$. BNAF and other modern density estimators are trained by stochastic optimization, and the two fits differ in initialization, batch order, and optimizer state, which adds noise to the influence estimate of Gen-LRA. We therefore default to KDE for Gen-LRA, and recommend it to practitioners on the basis of empirical performance and substantially lower computational cost.

7.4. Privacy-Utility Trade-off

The preceding results compare attacks against each other; we now ask whether the privacy leakage detected by Gen-LRA tracks the quality of the synthetic data being audited. Figure 5 plots mean Gen-LRA AUC against three quality measures averaged across models for each benchmark run: XGBoost test AUC under a train-on-synthetic-test-on-real protocol (downstream utility), maximum mean discrepancy and Jensen-Shannon distance between real and synthetic samples (distributional fidelity). Lower MMD and JS values indicate higher fidelity, while higher XGBoost test AUC indicates higher utility.

PATE-GAN at $\epsilon=1$ occupies the worst-quality corner of all three panels, with the lowest utility (XGBoost AUC $\approx 0.59$) and the worst fidelity on both MMD and JS, and also the lowest Gen-LRA AUC ($\approx 0.518$). This is the expected behavior of differentially private synthesis: the privacy guarantee that limits Gen-LRA’s signal also limits the synthetic data’s resemblance to the population. At the opposite end, RealTabFormer, ARF, and TabSyn achieve the highest utility and the strongest fidelity scores while exhibiting the highest Gen-LRA AUC values in the benchmark. These generators produce synthetic data that supports downstream learning effectively, but the same fidelity that makes the synthetic data useful renders it more vulnerable to No-Box MIAs.

8. Discussion

8.1. Interpreting the Benchmark

Theorem 4.2 characterizes the Gen-LRA score as a sum of localized density ratios over the $k$ synthetic neighbors of $x^{\star}$, and Theorem 4.4 shows that the expected score gap between members and non-members is governed by the kernel-overlap term $\Psi(h,\rho,x^{\star})$ between the surrogate’s bandwidth and the generator’s memorization scale. The simulation in Section 5 instantiates this score in three regimes the theory makes precise. Under near-exact memorization, single-point statistics already capture the membership signal and aggregation over $k$ neighbors only adds noise. Under approximate memorization, signal is dispersed across multiple synthetic points and aggregation recovers what single-point statistics miss. Under diffuse memorization, no local statistic separates members from non-members, a regime in which all No-Box attacks must fail. Extending the model to capture non-uniform overfitting across training points, for instance, by allowing $\epsilon$ and $\rho$ to vary with the local density of the population, is a natural refinement.

The distribution of overfitting behavior across modern tabular generators is not known a priori, and prior MIA evaluations have rarely been broad enough to characterize it. Most published tabular generative model MIA works span a handful of datasets and a small set of generator architectures, leaving open the question of whether reported performance generalizes beyond the conditions of the benchmark itself. Our evaluation spans 1,525 synthetic datasets across 9 generator families and 35 datasets, covering the architectural and statistical diversity a practitioner could realistically encounter.

Across this distribution, Gen-LRA achieves the highest Top-1 rate on every metric at more than $2\times$ that of any baseline, indicating that the approximate-memorization regime predicted by Theorem 4.4 to favor Gen-LRA is well-represented in practice. RealTabFormer is the most prominent exception, with known near-exact memorization behavior that dominates the top-100 runs by AUC and accounts for the outlier-driven means observed for distance-based baselines. Even within this subset, Gen-LRA achieves the highest mean and median on every metric.

These numbers also understate Gen-LRA’s true power. We fix $k=10$ across all 1,525 runs rather than tuning it per dataset or per generator, but a privacy auditor has access to ground-truth membership labels by construction, and the cost analysis in Section 4.6 shows that sweeping $k$ up to $k_{\max}$ is asymptotically equivalent to a single evaluation at $k_{\max}$. An auditor using Gen-LRA can therefore report the strongest configuration over $k$ at no additional cost, yielding strictly tighter bounds than the fixed-$k$ numbers reported here. The benchmark results should be read as a conservative estimate of what Gen-LRA delivers in practice.

The flip side is also informative: at fixed $k=10$, Gen-LRA is not the top-ranked attack on roughly 60% of runs. No single attack, Gen-LRA included, is uniformly optimal across datasets and generator architectures in our benchmark. Whether a uniformly optimal No-Box MIA exists is an important open question with direct consequences for how synthetic data should be audited in practice.

8.2. No-Box Auditing Estimates Realistic Privacy Leakage

A common objection to No-Box MIAs is that they are weaker than shadow-box or query-based alternatives, and therefore underestimate privacy risk. This framing conflates two distinct questions. Privacy leakage is not a property of a model alone — it is a property of a model conditioned on a threat model. The relevant question for a practitioner releasing synthetic data is not ”what is the maximum leakage achievable by any conceivable adversary?” but ”what leakage is achievable by an adversary I should plausibly defend against?”

Shadow and white-box attacks assume the adversary knows the generator’s architecture or has direct access. A defender following standard release practice, withholding model details and releasing only the synthetic dataset, eliminates the information these attacks depend on. The leakage they measure is therefore not the leakage the release exposes, but the leakage that would be exposed under a counterfactual disclosure regime the defender did not adopt. The No-Box setting, by contrast, corresponds to the release itself: the adversary sees only the synthetic dataset and auxiliary population data they could plausibly obtain. Less conservative threat models remain essential for other auditing problems — verifying differential privacy guarantees, for instance, requires the strongest attacks available regardless of deployment realism (Annamalai et al., 2024). But sample-level auditing under realistic adversaries is equally important, and progress in the No-Box regime, both theoretical and empirical, is what gives practitioners the tools to assess the privacy of the artifacts they actually release.

8.3. Limitations

Gen-LRA’s theoretical guarantees and empirical performance come with three limitations, each suggesting a direction for future work.

Scope of the local overfitting model.

Definition 4.3 characterizes the generator’s output density as the population density plus additive excess mass concentrated within distance $\rho$ of each training point, and Theorem 4.4 provides guarantees only for generators whose output density admits this decomposition with $\epsilon>0$ and $\rho$ comparable to the surrogate’s bandwidth. The model further idealizes $\phi_{\rho}$ and $\epsilon$ as uniform across training points, whereas in practice memorization is non-uniform: rare records, outliers, and points in low-density regions are typically memorized more strongly than the rest. The model is nonetheless rich enough to capture the three regimes that distinguish Gen-LRA’s behavior from single-point baselines (Section 5), and its predictions are consistent with Gen-LRA’s empirical performance on real generators.

Per-target hyperparameter sub-optimality.

The non-uniformity of memorization carries through to Gen-LRA itself: a single choice of bandwidth $h$ and locality $k$ is unlikely to be simultaneously optimal for every $x^{\star}$, and per-target attack power varies with the local memorization geometry around each test point. Our work does not disentangle this per-target variation. That said, the cost analysis in Section 4.6 establishes that adaptive per-target configurations are computationally feasible, making this a natural and tractable direction for future work rather than an obstacle.

Surrogate scaling.

KDE is known to scale poorly with dimensionality, and the empirical observation that KDE outperforms BNAF in our benchmark relies on the moderate dimensions ($d\lesssim 100$ after encoding) of the OpenML datasets we evaluate. On substantially higher-dimensional tabular data, neither KDE nor BNAF may be adequate, and Gen-LRA’s surrogate component would require revisiting. The density-ratio-estimation literature (Sugiyama et al., 2012; Kanamori et al., 2009) establishes that estimating a ratio directly is more sample-efficient than estimating two densities and dividing them, suggesting an attractive alternative to the two surrogate fits in Algorithm 1. Adapting these methods to Gen-LRA is non-trivial: standard direct-ratio estimators assume two independent samples drawn from distinct distributions, whereas the two distributions whose ratio we require are induced by reference sets that differ by a single point. Closing this gap likely requires new estimators tailored to the influence-function structure of the problem, which we view as a promising direction for scaling Gen-LRA to higher-dimensional regimes.

9. Conclusion

Our results suggest that the dominant No-Box membership inference attacks for synthetic data release underestimate the privacy risk of the artifacts they evaluate. DOMIAS tests the wrong hypothesis, evaluating where synthetic density exceeds reference density rather than whether a specific point influenced the synthetic distribution; and distance-based methods such as DCR detect leakage only when memorization is near-exact. Both classes underestimate in the approximate-memorization regime, which our benchmark shows is where most modern tabular generators likely operate. A practitioner concluding that a release is private on the basis of these attacks is relying on measurements whose inductive biases are matched to a specific overfitting phenomena that the generator may not exhibit.

Gen-LRA addresses this gap with an attack that is both principled and practical. The likelihood-ratio influence-function formulation directly tests the membership hypothesis, the closed-form characterization (Theorem 4.2) makes explicit what the attack measures, and the mean-score-gap result (Theorem 4.4) provides testable predictions about when it succeeds, predictions we validate in a controlled simulation and that align with empirical performance across 1,525 synthetic datasets spanning nine generators. Gen-LRA achieves the highest Top-1 rate on every metric at more than $2\times$ that of any baseline, and corresponding highest mean values in the top 100 highest privacy leakage runs.

The framework we develop suggests several directions for future work. Adaptive bandwidth and locality selection on a per-target basis would likely increase attack performance, particularly for the long-tail records that are memorized most strongly. Replacing the two separate density estimates in Algorithm 1 with a direct density-ratio estimator tailored to the single-point-difference setting offers a route to scaling Gen-LRA beyond the moderate-dimensional regimes where KDE remains effective, and would likely improve performance on our benchmark as well. Extending the local-overfitting model and the influence-function framing to settings where synthetic tabular data release is becoming common such as longitudinal records is a natural next step. Finally, Gen-LRA’s broad but non-uniform dominance leaves open whether a uniformly optimal No-Box MIA exists—a question we view as central to the privacy auditing literature, since such an attack would let practitioners directly certify a synthetic dataset’s privacy under the threat model rather than relying on guarantees from defensive frameworks.

10. Ethical Considerations

This work proposes a membership inference attack against synthetic tabular data, which is itself a re-identification method. We address the resulting ethical considerations along three dimensions: the risks the attack poses, the benefits it provides, and the steps we have taken to ensure that the latter outweigh the former.

Risks. Adversaries who can infer whether an individual’s record was used to train a generative model pose direct privacy risks to that individual, particularly in domains such as healthcare, finance, and the social sciences where sensitive personal data is routinely used. A synthetic dataset that fails to obfuscate membership information can enable re-identification of training-set participants, undermining the privacy guarantees that motivate synthetic data release in the first place. Gen-LRA, by improving the state of the art in No-Box membership inference, raises the ceiling on what such an adversary can achieve.

Benefits. The same capability that creates risk also enables stronger privacy auditing. Practitioners releasing synthetic datasets currently rely on similarity-based heuristics that have been shown to be inadequate proxies for privacy, or on attacks whose threat-model assumptions do not match realistic release scenarios. Gen-LRA gives auditors a principled, computationally tractable tool to assess sample-level privacy leakage under the threat model that actually corresponds to public release. Without attacks of this kind, defenders cannot measure the privacy of the artifacts they release.

Steps taken to minimize harm. We evaluate Gen-LRA exclusively on publicly available datasets from OpenML, which contain no personally identifying information beyond what their original releasers have already chosen to publish. We do not target any deployed synthetic data release, nor do we attempt to re-identify individuals in any real-world dataset. The generative models we attack are trained by us on these public datasets specifically for the purpose of evaluating Gen-LRA, so no third party’s privacy claims are challenged by our experiments. We release our code openly to support reproducibility and to enable defenders to audit their own releases, which we view as the primary use case. Responsible disclosure does not apply here, as no specific deployed system is implicated; the vulnerability we characterize is structural to overfitting in tabular generative models and has been documented in prior work.

We believe adversarial work of this kind is essential to the development of trustworthy synthetic data release. Deferring the publication of stronger attacks would leave practitioners auditing with tools that systematically underestimate privacy risk, which we view as a net riskier outcome.

Appendix A Proofs and Supporting Results

This appendix provides proofs for the three theoretical results of Section 4.

A.1. Proof of Theorem 4.1 (Invariance)

Proof.

Let $g:\mathcal{X}\to\mathcal{X}$ be a continuously differentiable invertible function with Jacobian $J(x)=\frac{dg}{dx}(x)$, where $|J(x)|\neq 0$ for all $x\in\mathcal{X}$. For sets $A\subseteq\mathcal{X}$, write $\tilde{A}=g(A)$, and for a density $p$ let $\tilde{p}$ denote the density induced on the transformed space.

By the change-of-variables formula for probability densities,

[TABLE]

where $|J(A)|$ denotes the product of Jacobian determinants evaluated on $A$ (treating $J$ as acting pointwise on a set of samples).

Applying this to the conditional density in the numerator of the log-likelihood ratio:

[TABLE]

Similarly for the denominator:

[TABLE]

Taking the ratio of these two conditional densities:

[TABLE]

The Jacobian factors cancel because the joint samples $(S,R\cup\{x^{\star}\})$ and $(S,R)$ differ by a single point whose Jacobian appears identically in the numerator of one ratio and the denominator of the other. This leaves

[TABLE]

Taking logarithms yields $\mathcal{I}(g(x^{\star}),g(R),g(S))=\mathcal{I}(x^{\star},R,S)$. ∎

A.2. Proof of Theorem 4.2 (Closed-Form Approximation)

Proof.

Let $\hat{p}_{R}$ denote a Gaussian KDE fit on $R$ with bandwidth $h$, where $|R|=m$. The proof proceeds in four steps.

Step 1: Exact decomposition of the augmented KDE. By the definition of the Gaussian KDE,

[TABLE]

This identity is exact; no approximation is used.

Step 2: Ratio form. Dividing both sides by $\hat{p}_{R}(s)$,

[TABLE]

Step 3: Taylor expansion of the logarithm. Let

[TABLE]

The regularity assumption $\hat{p}_{R}(s)\geq c>0$ on $S_{k}(x^{\star})$, together with the boundedness of the Gaussian kernel $K_{h}$, implies that $|u_{s}|\leq C/(m+1)$ for a constant $C$ depending on $c$, $h$, and $d$. For $|u|\to 0$, the Taylor expansion

[TABLE]

gives

[TABLE]

Step 4: Summation over the local neighborhood. Summing over $s\in S_{k}(x^{\star})$:

[TABLE]

For fixed $k$ or $k=o(m)$, the remainder is $O_{p}(m^{-2})$, which completes the proof. ∎

A.3. Proof of Theorem 4.4 (Mean Score Gap)

Proof sketch.

We work throughout with the leading-order expression from Theorem 4.2:

[TABLE]

The proof proceeds in four steps.

Step 1: Expected score. Let $g(s,x^{\star}):=K_{h}(s-x^{\star})/\hat{p}_{R}(s)-1$. Taking expectations over the randomness in $S$:

[TABLE]

The $k$-NN localization concentrates $s$ near $x^{\star}$, where the kernel $K_{h}(s-x^{\star})$ is appreciable; outside this region $g(s,x^{\star})\approx-1$ and contributes only to the constant baseline. For the purpose of comparing $\mu_{1}$ and $\mu_{0}$, we can treat the expectation as being taken with respect to $p_{G}$ itself; localization affects $\mu_{0}$ and $\mu_{1}$ symmetrically and does not contribute to their difference.

Step 2: Substitution of the overfitting decomposition. Plugging Equation 8 into $\mathbb{E}_{s\sim p_{G}}[K_{h}(s-x^{\star})/\hat{p}_{R}(s)]$ yields three additive pieces:

[TABLE]

Step 3: Differencing member vs. non-member expectations. We examine how each piece changes between the hypotheses $H_{0}$ ($x^{\star}\sim P$, independent of $T$) and $H_{1}$ ($x^{\star}\in T$).

Piece A is a functional of $x^{\star}$ and the population density $p$ only; it does not reference $T$. Under both hypotheses, $x^{\star}$ has marginal distribution $P$, so Piece A contributes identically to $\mu_{0}$ and $\mu_{1}$ and cancels in the difference. The same argument applies to Piece C: the residual $r$ is a property of the generator’s output density, not of $x^{\star}$’s membership status.

For Piece B, define the kernel-overlap integral

[TABLE]

which measures the overlap between the kernel centered at $x^{\star}$ and the memorization bump centered at $x_{i}$. This integral is largest when $x_{i}=x^{\star}$; we denote this self-overlap by $\Psi(h,\rho,x^{\star}):=f(x^{\star},x^{\star})$. Piece B then reads $\frac{\epsilon}{|T|}\sum_{i=1}^{|T|}f(x^{\star},x_{i})$, and we compare this sum under the two hypotheses.

Let $B(x^{\star}):=\sum_{i:x_{i}\neq x^{\star}}f(x^{\star},x_{i})$ denote the background sum: the total overlap between the kernel at $x^{\star}$ and memorization bumps centered at training points other than $x^{\star}$. Then:

• •

Under $H_{0}$ ($x^{\star}\sim P$, independent of $T$): none of the $x_{i}$ equals $x^{\star}$, so $\sum_{i=1}^{|T|}f(x^{\star},x_{i})=B(x^{\star})$, a sum of $|T|$ terms.

• •

Under $H_{1}$ ($x^{\star}=x_{j}$ for some $j$): the sum decomposes as

[TABLE]

where the first term is the self-overlap guaranteed by the memorization bump $\phi_{\rho}(\cdot;x_{j})$ sitting on top of the kernel at $x^{\star}=x_{j}$, and the second term is a background sum of $|T|-1$ terms.

The two versions of $B(x^{\star})$ differ by a single term out of $|T|$, which is $O(1/|T|)$ relative to the full background; to leading order they are equal. The dominant contribution to $\mu_{1}-\mu_{0}$ is therefore the self-overlap term, which is present only under $H_{1}$:

[TABLE]

Step 4: Positivity of the signal term. The integrand of $\Psi(h,\rho,x^{\star})$ is a product of non-negative quantities. It is strictly positive whenever the supports of $\phi_{\rho}(\cdot;x^{\star})$ and $K_{h}(\cdot-x^{\star})$ intersect—i.e., whenever the attacker’s bandwidth reaches the memorization region. ∎

A member $x^{\star}\in T$ exhibits extra local synthetic density compared to a non-member because the memorization bump $\phi_{\rho}(\cdot;x^{\star})$ is placed at $x^{\star}$’s own location. All other contributions to local synthetic density, the population term, the residual, and memorization bumps placed at other training points, occur symmetrically for members and non-members (both of which are distributed according to $P$) and cancel in the difference. The attack’s power thus derives entirely from the self-overlap signal.

Appendix B Experiments/ Replication Details

B.1. MIAs for Generative Models Descriptions

The Membership Inference Attacks referenced in this paper is are described as follows:

Distance to Closest Record (DCR / DCR-Diff). A common class of membership inference attacks relies on the hypothesis that synthetic generators may reproduce training records or place generated points unusually close to them in feature space (Chen et al., 2020). Under this intuition, query points corresponding to members should exhibit smaller distances to the synthetic dataset than non-members. The Distance to Closest Record (DCR) attack quantifies this effect using the score

[TABLE]

where $d(\cdot,\cdot)$ denotes a chosen distance function. A natural extension, DCR-Diff, incorporates a reference dataset $R$ to normalize this proximity signal by comparing closeness to both datasets:

[TABLE]

DOMIAS / Density Estimation. DOMIAS (van Breugel et al., 2023) approaches membership inference from a distributional perspective by comparing how likely a query record is under the synthetic distribution relative to a reference distribution. Specifically, it evaluates the density ratio

[TABLE]

where $p_{S}$ and $p_{R}$ are density estimates fit to the synthetic and reference datasets, respectively. In practice, these densities may be estimated using approaches such as kernel density estimation or neural density models. A simpler related baseline proposed in (Houssiau et al., 2022) uses only the synthetic density estimate,

[TABLE]

without explicitly contrasting against a reference distribution.

Data Plagiarism Index (DPI) / Local Neighborhood. The Data Plagiarism Index (DPI) (Ward et al., 2024) focuses on localized memorization by examining the composition of the neighborhood surrounding a query point. Given $x^{*}$, the method constructs a $k$-nearest-neighbor set $D(x^{*})$ drawn jointly from synthetic and reference data, and computes the ratio of synthetic to reference neighbors:

[TABLE]

Higher values indicate stronger local resemblance to the synthetic data distribution. A closely related local attack in (Houssiau et al., 2022) replaces the fixed-$k$ neighborhood with all points falling within a specified radius of $x^{*}$.

LOGAN / Classifier-based. LOGAN was first introduced as a white-box membership inference attack (Hayes et al., 2017) and later adapted to the black-box setting in (van Breugel et al., 2023). The core idea is to train a classifier to distinguish samples from the target synthetic dataset $S$ and a reference dataset $R$. In the original formulation, this classifier is implemented as the discriminator of a GAN trained on synthetic data, and the resulting score

[TABLE]

is used as the membership signal. Records receiving higher discriminator confidence are considered more likely to be members. Subsequent work (Houssiau et al., 2022) generalized this approach by replacing the GAN discriminator with standard supervised classifiers such as random forests.

B.2. Generative Model Architecture Descriptions

In all experiments, we use the implementations of these models from the Python package Synthcity (Qian et al., 2023). For benchmarking purposes, we use the default hyperparameters for each model. A brief description of each model is as follows:

• •

CTGAN (Xu et al., 2019): Conditional Tabular Generative Adversarial Network uses a GAN framework with conditional generator and discriminator to capture multi-modal distributions. It employs mode normalization to better learn mixed-type distributions.

• •

TVAE (Xu et al., 2019): Tabular Variational Auto-Encoder is similar to CTGAN in its use of mode normalizing techniques, but instead of a GAN architecture, it employs a Variational Autoencoder.

• •

Normalizing Flows (NFlows) (Durkan et al., 2019): Normalizing flows transform a simple base distribution (e.g., Gaussian) into a more complex one matching the data by applying a sequence of invertible, differentiable mappings.

• •

Bayesian Network (BN) (Ankan and Panda, 2015): Bayesian Networks use a Directed Acyclic Graph to represent the joint probability distribution over variables as a product of marginal and conditional distributions. It then samples the empirical distributions estimated from the training dataset.

• •

Adversarial Random Forests (ARF) (Watson et al., 2023): ARFs extend the random forest model by adding an adversarial stage. Random forests generate synthetic samples which are scored against the real data by a discriminator network. This score is used to re-train the forests iteratively.

• •

Tab-DDPM (Kotelnikov et al., 2022): Tabular Denoising Diffusion Probabilistic Model adapts the DDPM framework for image synthesis. It iteratively refines random noise into synthetic data by learning the data distribution through gradients of a classifier on partially corrupted samples with Gaussian noise.

• •

PATEGAN (Yoon et al., 2019): The PATEGAN model uses a neural encoder to map discrete tabular data into a continuous latent representation which is sampled from during generation by the GAN discriminator and generator pair.

• •

Ads-GAN (Yoon et al., 2020b): Ads-GAN uses a GAN architecture for tabular synthesis but also adds an identifiability metric to increase its ability to not mimic training data.

• •

TabSyn (Zhang et al., 2024): TabSyn uses a Variational Auto-Encoder to learn a latent space in which it builds a diffusion model from. TabSyn usually achieves state of the art data quality metrics relative to other methods compared.

B.3. Benchmarking Datasets References

Appendix C Ablations

The ablations in for the deep learning and encoding experiments were conducted on a smaller-scale version of the experimental protocol from Section 6. Across the 15 datasets listed below, each generator was trained on $\min(0.8\cdot|D|,N)$ records, with synthetic, holdout, and reference sets each drawn at the same size. We use $N=1000$ for the encoding ablation (Appendix C.1) and $N=250$ for the deep learning ablation (Appendix C.2) to induce greater overfitting in the latter setting. All other aspects of the protocol, including generator hyperparameters and attack configurations, match the main benchmark. The reduced scale lets us sweep design choices that would be prohibitively expensive at full benchmark size while preserving the qualitative conditions under which Gen-LRA is intended to be deployed. Datasets:

1. (1)

Abalone (OpenML) 2. (2)

Adult (Becker,Barry and Kohavi,Ronny, 1996) 3. (3)

Bean (UCI) 4. (4)

Churn-Modeling (Kaggle) 5. (5)

Faults (UCI) 6. (6)

HTRU (UCI) 7. (7)

Indian Liver Patient (Kaggle) 8. (8)

Insurance (Kaggle) 9. (9)

Magic (Kaggle) 10. (10)

News (UCI) 11. (11)

Nursery (Kaggle) 12. (12)

Obesity (Kaggle) 13. (13)

Shoppers (Kaggle) 14. (14)

Titanic (Kaggle) 15. (15)

Wilt (OpenML)

C.1. Gen-LRA Encoding

As our main experiment uses Kernel Density Estimation (KDE) over (usually) heterogeneous datasets, we present an ablation for encoding tabular data to be numeric such that KDE can converge. We experiment with 3 common strategies used in the density estimation literature: ordinal encoding for categorical variables, one-hot encoding categorical variables and then performing Principle Component Analysis (PCA), and using a Variational Auto-Encoder to learn continuous latent representations of the data.

We repeat our ablation subset experiment with these three encoding schemes. For PCA we use the number of eigenvectors that explain up to 95 %variance and for the VAE encoding we use TabSyn’s original auto-encoder with default settings. Overall, we evaluate the top 100 runs for each metric and find that Ordinal encodings yield the best results (see Table 4).

C.2. Deep Learning Estimation

We replace the KDE surrogate in Gen-LRA with Block Neural Autoregressive Flows (BNAF) (De Cao et al., 2019), following the implementation and hyperparameters of (van Breugel et al., 2023). Each BNAF model is trained with default implementation parameters with early stopping on a held-out validation split of the reference set, using the Adam optimizer with default learning rate. We fit one BNAF on $R$ and a second on $R\cup\{x^{\star}\}$ for each test point and compute the Gen-LRA score using these two surrogates in place of the corresponding KDEs. To induce greater overfitting in the trained generators and thereby produce stronger membership signal for both surrogates to recover, we reduce the training, holdout, reference, and synthetic set sizes to 250 records each. We then evaluate the top 100 highest-scored runs for each metric from the ordinal-encoded KDE and BNAF versions of Gen-LRA. All other aspects of the small-scale benchmark protocol, including datasets, generators, and attack baselines, match Section C. Results and analysis appear in Section 7.3.

C.3. Ablation: Different $k$ sizes

Gen-LRA targets local overfitting by utilizing the $k$-nearest neighbors in $S$ to $x^{*}$. Consequently, $k$ serves as a hyperparameter in the attack. To assess the impact of $k$ on attack efficacy, we replicate the benchmarking experiments from Section 6 across varying values of $k$. The average AUC-ROC and corresponding standard deviations for the top 300 runs for each evaluation metric are reported in Table 5. Empirically, we observe that smaller values of $k$ generally enhance attack performance, though this effect varies by model.

Appendix D Additional Figures