Beyond Uniform Deletion: A Data Value-Weighted Framework for Certified Machine Unlearning

TL;DR

This paper introduces a data value-weighted framework for machine unlearning that improves model performance and robustness by accounting for the heterogeneous utility of data points during the unlearning process.

Contribution

The paper proposes a novel Data Value-Weighted Unlearning (DVWU) framework that incorporates data utility heterogeneity into unlearning, enhancing existing methods with a flexible weighting strategy.

Findings

DVWU improves predictive accuracy after data deletion.

The framework enhances robustness against unlearning attacks.

It is adaptable to various gradient-based unlearning methods.

Abstract

As the right to be forgotten becomes legislated worldwide, machine unlearning mechanisms have emerged to efficiently update models for data deletion and enhance user privacy protection. However, existing machine unlearning algorithms frequently neglect the fact that different data points may contribute unequally to model performance (i.e., heterogeneous data values). Treat them equally in machine unlearning procedure can potentially degrading the performance of updated models. To address this limitation, we propose Data Value-Weighted Unlearning (DVWU), a general unlearning framework that accounts for data value heterogeneity into the unlearning process. Specifically, we design a weighting strategy based on data values, which are then integrated into the unlearning procedure to enable differentiated unlearning for data points with varying utility to the model. The DVWU framework can be broadly adapted to various existing machine unlearning methods. We use the one-step Newton update as an example for implementation, developing both output and objective perturbation algorithms to achieve certified unlearning. Experiments on both synthetic and real-world datasets demonstrate that our methods achieve superior predictive performance and robustness compared to conventional unlearning approaches. We further show the extensibility of our framework on gradient ascent method by incorporating the proposed weighting strategy into the gradient terms, highlighting the adaptability of DVWU for broader gradient-based deep unlearning methods.