Silver Linings in the Shadows: Harnessing Membership Inference for Machine Unlearning

TL;DR

This paper introduces a novel machine unlearning method that effectively removes specific data influence from neural networks, enhancing privacy while maintaining model performance, validated through empirical tests and theoretical analysis.

Contribution

The paper presents a new unlearning mechanism using a tailored loss function that combines classification and membership inference losses, adaptable to various privacy mechanisms.

Findings

Effective removal of data influence demonstrated across multiple datasets and architectures.

Superior unlearning performance in terms of efficacy and latency.

Maintains primary task accuracy after unlearning.

Abstract

With the continued advancement and widespread adoption of machine learning (ML) models across various domains, ensuring user privacy and data security has become a paramount concern. In compliance with data privacy regulations, such as GDPR, a secure machine learning framework should not only grant users the right to request the removal of their contributed data used for model training but also facilitates the elimination of sensitive data fingerprints within machine learning models to mitigate potential attack - a process referred to as machine unlearning. In this study, we present a novel unlearning mechanism designed to effectively remove the impact of specific data samples from a neural network while considering the performance of the unlearned model on the primary task. In achieving this goal, we crafted a novel loss function tailored to eliminate privacy-sensitive information from weights and activation values of the target model by combining target classification loss and membership inference loss. Our adaptable framework can easily incorporate various privacy leakage approximation mechanisms to guide the unlearning process. We provide empirical evidence of the effectiveness of our unlearning approach with a theoretical upper-bound analysis through a membership inference mechanism as a proof of concept. Our results showcase the superior performance of our approach in terms of unlearning efficacy and latency as well as the fidelity of the primary task, across four datasets and four deep learning architectures.