Privacy Adhering Machine Un-learning in NLP

TL;DR

This paper introduces efficient machine unlearning methods for NLP tasks, enabling rapid data removal from models with minimal performance impact, addressing privacy regulations like GDPR and CCPA.

Contribution

It proposes novel, computationally efficient unlearning approaches (SISA-FC and SISA-A) for NLP models that significantly reduce resource usage while maintaining accuracy.

Findings

Achieved 90-95% memory reduction

Reduced unlearning time by 100x

Maintained model performance after unlearning

Abstract

Regulations introduced by General Data Protection Regulation (GDPR) in the EU or California Consumer Privacy Act (CCPA) in the US have included provisions on the \textit{right to be forgotten} that mandates industry applications to remove data related to an individual from their systems. In several real world industry applications that use Machine Learning to build models on user data, such mandates require significant effort both in terms of data cleansing as well as model retraining while ensuring the models do not deteriorate in prediction quality due to removal of data. As a result, continuous removal of data and model retraining steps do not scale if these applications receive such requests at a very high frequency. Recently, a few researchers proposed the idea of \textit{Machine Unlearning} to tackle this challenge. Despite the significant importance of this task, the area of Machine Unlearning is under-explored in Natural Language Processing (NLP) tasks. In this paper, we explore the Unlearning framework on various GLUE tasks \cite{Wang:18}, such as, QQP, SST and MNLI. We propose computationally efficient approaches (SISA-FC and SISA-A) to perform \textit{guaranteed} Unlearning that provides significant reduction in terms of both memory (90-95\%), time (100x) and space consumption (99\%) in comparison to the baselines while keeping model performance constant.