Controllable Machine Unlearning via Gradient Pivoting

TL;DR

This paper introduces CUP, a novel gradient-based algorithm for controllable machine unlearning that effectively balances unlearning efficacy and model fidelity by navigating the Pareto frontier with a single hyperparameter.

Contribution

We reformulate machine unlearning as a multi-objective optimization problem and propose CUP, a gradient pivoting method that allows precise control over the unlearning trade-off using a hyperparameter.

Findings

CUP outperforms existing methods in generating Pareto-optimal solutions.

CUP provides fine-grained control over unlearning and fidelity trade-offs.

Experimental results across vision tasks validate CUP's effectiveness.

Abstract

Machine unlearning (MU) aims to remove the influence of specific data from a trained model. However, approximate unlearning methods, often formulated as a single-objective optimization (SOO) problem, face a critical trade-off between unlearning efficacy and model fidelity. This leads to three primary challenges: the risk of over-forgetting, a lack of fine-grained control over the unlearning process, and the absence of metrics to holistically evaluate the trade-off. To address these issues, we reframe MU as a multi-objective optimization (MOO) problem. We then introduce a novel algorithm, Controllable Unlearning by Pivoting Gradient (CUP), which features a unique pivoting mechanism. Unlike traditional MOO methods that converge to a single solution, CUP's mechanism is designed to controllably navigate the entire Pareto frontier. This navigation is governed by a single intuitive hyperparameter, the `unlearning intensity', which allows for precise selection of a desired trade-off. To evaluate this capability, we adopt the hypervolume indicator, a metric that captures both the quality and diversity of the entire set of solutions an algorithm can generate. Our experimental results demonstrate that CUP produces a superior set of Pareto-optimal solutions, consistently outperforming existing methods across various vision tasks.