Evaluating Differential Privacy Against Membership Inference in Federated Learning: Insights from the NIST Genomics Red Team Challenge

TL;DR

This paper empirically evaluates how differential privacy defends against membership inference attacks in federated learning, using a stacking attack strategy across different privacy levels, revealing measurable leakage even at high privacy settings.

Contribution

It introduces a stacking attack method that improves inference accuracy and provides an empirical analysis of differential privacy's effectiveness at various privacy levels in federated learning.

Findings

Stacking attack outperforms baselines in unprotected and low privacy settings.

Measurable membership leakage persists at $\epsilon=200$ despite differential privacy.

Results characterize how inference degrades across privacy tiers in federated learning.

Abstract

While Federated Learning (FL) mitigates direct data exposure, the resulting trained models remain susceptible to membership inference attacks (MIAs). This paper presents an empirical evaluation of Differential Privacy (DP) as a defense mechanism against MIAs in FL, leveraging the environment of the 2025 NIST Genomics Privacy-Preserving Federated Learning (PPFL) Red Teaming Event. To improve inference accuracy, we propose a stacking attack strategy that ensembles seven black-box estimators to train a meta-classifier on prediction probabilities and cross-entropy losses. We evaluate this methodology against target models under three privacy configurations: an unprotected convolutional neural network (CNN, $\epsilon=\infty$), a low-privacy DP model ($\epsilon=200$), and a high-privacy DP model ($\epsilon=10$). The attack outperforms all baselines in the No DP and Low Privacy settings and, critically, maintains measurable membership leakage at $\epsilon=200$ where a single-signal LiRA baseline collapses. Evaluated on an independent third-party benchmark, these results provide an empirical characterisation of how stacking-based inference degrades across calibrated DP tiers in FL.