Random Relabeling for Efficient Machine Unlearning

TL;DR

This paper introduces random relabeling, an efficient method for machine unlearning that supports data removal requests in supervised learning, reducing computational costs compared to retraining from scratch.

Contribution

The paper proposes a novel unlearning scheme called random relabeling applicable to generic supervised learning algorithms for efficient data removal.

Findings

Supports sequential data removal requests in online settings.

Provides a removal certification method based on probability distribution similarity.

Applicable to logit-based classifiers.

Abstract

Learning algorithms and data are the driving forces for machine learning to bring about tremendous transformation of industrial intelligence. However, individuals' right to retract their personal data and relevant data privacy regulations pose great challenges to machine learning: how to design an efficient mechanism to support certified data removals. Removal of previously seen data known as machine unlearning is challenging as these data points were implicitly memorized in training process of learning algorithms. Retraining remaining data from scratch straightforwardly serves such deletion requests, however, this naive method is not often computationally feasible. We propose the unlearning scheme random relabeling, which is applicable to generic supervised learning algorithms, to efficiently deal with sequential data removal requests in the online setting. A less constraining removal certification method based on probability distribution similarity with naive unlearning is further developed for logit-based classifiers.