Statistical Roughness-Informed Machine Unlearning

TL;DR

This paper introduces SRAGU, a novel layer-wise spectral reweighting method for machine unlearning that enhances stability and effectiveness in removing data influence from deep neural networks, especially under large or adversarial deletions.

Contribution

The paper proposes SRAGU, a new unlearning algorithm that adaptively reweights layer updates based on spectral diagnostics, improving stability and accuracy in unlearning tasks.

Findings

SRAGU improves stability in unlearning large or adversarial data deletions.

Spectral reweighting concentrates updates in stable layers, reducing catastrophic forgetting.

Empirical results show better alignment with retrained models and reduced leakage signals.

Abstract

Machine unlearning aims to remove the influence of a designated forget set from a trained model while preserving utility on the retained data. In modern deep networks, approximate unlearning frequently fails under large or adversarial deletions due to pronounced layer-wise heterogeneity: some layers exhibit stable, well-regularized representations while others are brittle, undertrained, or overfit, so naive update allocation can trigger catastrophic forgetting or unstable dynamics. We propose Statistical-Roughness Adaptive Gradient Unlearning (SRAGU), a mechanism-first unlearning algorithm that reallocates unlearning updates using layer-wise statistical roughness operationalized via heavy-tailed spectral diagnostics of layer weight matrices. Starting from an Adaptive Gradient Unlearning (AGU) sensitivity signal computed on the forget set, SRAGU estimates a WeightWatcher-style heavy-tailed exponent for each layer, maps it to a bounded spectral stability weight, and uses this stability signal to spectrally reweight the AGU sensitivities before applying the same minibatch update form. This concentrates unlearning motion in spectrally stable layers while damping updates in unstable or overfit layers, improving stability under hard deletions. We evaluate unlearning via behavioral alignment to a gold retrained reference model trained from scratch on the retained data, using empirical prediction-divergence and KL-to-gold proxies on a forget-focused query set; we additionally report membership inference auditing as a complementary leakage signal, treating forget-set points as should-be-forgotten members during evaluation.