Soft Weighted Machine Unlearning

TL;DR

This paper introduces a soft-weighted framework for machine unlearning that mitigates over-unlearning issues, improves fairness and robustness, and maintains utility better than traditional binary removal methods.

Contribution

It proposes a novel soft-weighted approach with influence functions and convex optimization, enhancing unlearning algorithms for fairness and robustness without utility loss.

Findings

Soft-weighted scheme outperforms hard-weighted schemes in fairness and robustness.

The approach alleviates utility decline in unlearning tasks.

Versatile integration into existing unlearning algorithms.

Abstract

Machine unlearning, as a post-hoc processing technique, has gained widespread adoption in addressing challenges like bias mitigation and robustness enhancement, colloquially, machine unlearning for fairness and robustness. However, existing non-privacy unlearning-based solutions persist in using binary data removal framework designed for privacy-driven motivation, leading to significant information loss, a phenomenon known as over-unlearning. While over-unlearning has been largely described in many studies as primarily causing utility degradation, we investigate its fundamental causes and provide deeper insights in this work through counterfactual leave-one-out analysis. In this paper, we introduce a weighted influence function that assigns tailored weights to each sample by solving a convex quadratic programming problem analytically. Building on this, we propose a soft-weighted framework enabling fine-grained model adjustments to address the over-unlearning challenge. We demonstrate that the proposed soft-weighted scheme is versatile and can be seamlessly integrated into most existing unlearning algorithms. Extensive experiments show that in fairness- and robustness-driven tasks, the soft-weighted scheme significantly outperforms hard-weighted schemes in fairness/robustness metrics and alleviates the decline in utility metric, thereby enhancing machine unlearning algorithm as an effective correction solution.