Certified Per-Instance Unlearning Using Individual Sensitivity Bounds

TL;DR

This paper introduces a method for certified machine unlearning that uses adaptive, per-instance noise calibration based on individual data point contributions, reducing noise and improving performance.

Contribution

It develops a framework for per-instance sensitivity bounds in noisy gradient methods, enabling more efficient certified unlearning, especially in ridge regression and deep learning.

Findings

Per-instance sensitivity bounds reduce noise injection in unlearning.

The approach achieves certified unlearning with less performance degradation.

Empirical results validate the theoretical bounds in linear and deep learning models.

Abstract

Certified machine unlearning can be achieved via noise injection leading to differential privacy guarantees, where noise is calibrated to worst-case sensitivity. Such conservative calibration often results in performance degradation, limiting practical applicability. In this work, we investigate an alternative approach based on adaptive per-instance noise calibration tailored to the individual contribution of each data point to the learned solution. This raises the following challenge: how can one establish formal unlearning guarantees when the mechanism depends on the specific point to be removed? To define individual data point sensitivities in noisy gradient dynamics, we consider the use of per-instance differential privacy. For ridge regression trained via Langevin dynamics, we derive high-probability per-instance sensitivity bounds, yielding certified unlearning with substantially less noise injection. We corroborate our theoretical findings through experiments in linear settings and provide further empirical evidence on the relevance of the approach in deep learning settings.