Spurious Privacy Leakage in Neural Networks

TL;DR

This paper explores how spurious correlations in neural networks lead to privacy vulnerabilities, revealing that models often memorize spurious features and that robust methods do not mitigate privacy disparities across groups.

Contribution

It introduces the concept of spurious privacy leakage, analyzes why robust methods fail to reduce privacy disparity, and compares how different architectures influence privacy in the presence of spurious data.

Findings

Spurious groups are more vulnerable to privacy attacks than non-spurious groups.

Privacy disparity increases in tasks with fewer classes due to spurious features.

Robust methods do not prevent memorization of spurious features during training.

Abstract

Neural networks trained on real-world data often exhibit biases while simultaneously being vulnerable to privacy attacks aimed at extracting sensitive information. Despite extensive research on each problem individually, their intersection remains poorly understood. In this work, we investigate the privacy impact of spurious correlation bias. We introduce \emph{spurious privacy leakage}, a phenomenon in which spurious groups are significantly more vulnerable to privacy attacks than non-spurious groups. We observe that privacy disparity between groups increases in tasks with simpler objectives (e.g. fewer classes) due to spurious features. Counterintuitively, we demonstrate that spurious robust methods, designed to reduce spurious bias, fail to mitigate privacy disparity. Our analysis reveals that this occurs because robust methods can reduce reliance on spurious features for prediction, but do not prevent their memorization during training. Finally, we systematically compare the privacy of different model architectures trained with spurious data, demonstrating that, contrary to previous work, architectural choice can affect privacy evaluation.