Defending Membership Inference Attacks via Privacy-aware Sparsity Tuning

TL;DR

This paper introduces PAST, a privacy-aware sparsity tuning method that adaptively regularizes model parameters based on their privacy sensitivity, significantly reducing membership inference risks while maintaining utility.

Contribution

The paper proposes PAST, a novel adaptive regularization technique that selectively sparsifies parameters impacting privacy, improving the privacy-utility tradeoff in over-parameterized models.

Findings

PAST effectively reduces membership inference attack success rates.

PAST achieves a better privacy-utility balance compared to traditional regularization methods.

Extensive experiments validate the superiority of PAST in privacy preservation.

Abstract

Over-parameterized models are typically vulnerable to membership inference attacks, which aim to determine whether a specific sample is included in the training of a given model. Previous Weight regularizations (e.g., L1 regularization) typically impose uniform penalties on all parameters, leading to a suboptimal tradeoff between model utility and privacy. In this work, we first show that only a small fraction of parameters substantially impact the privacy risk. In light of this, we propose Privacy-aware Sparsity Tuning (PAST), a simple fix to the L1 Regularization, by employing adaptive penalties to different parameters. Our key idea behind PAST is to promote sparsity in parameters that significantly contribute to privacy leakage. In particular, we construct the adaptive weight for each parameter based on its privacy sensitivity, i.e., the gradient of the loss gap with respect to the parameter. Using PAST, the network shrinks the loss gap between members and non-members, leading to strong resistance to privacy attacks. Extensive experiments demonstrate the superiority of PAST, achieving a state-of-the-art balance in the privacy-utility trade-off.