Fully Decentralized Certified Unlearning

TL;DR

This paper introduces RR-DU, a decentralized unlearning algorithm with theoretical guarantees and empirical results showing improved test accuracy and effective data removal on image benchmarks.

Contribution

It proposes a novel decentralized unlearning method with convergence, privacy, and utility guarantees, addressing the underexplored decentralized setting.

Findings

RR-DU achieves $( ext{ε,δ})$ unlearning certificates.

It outperforms decentralized DP baselines in test accuracy.

It effectively reduces forget accuracy to near random guessing.

Abstract

Machine unlearning (MU) seeks to remove the influence of specified data from a trained model in response to privacy requests or data poisoning. While certified unlearning has been analyzed in centralized and server-orchestrated federated settings (via guarantees analogous to differential privacy, DP), the decentralized setting -- where peers communicate without a coordinator remains underexplored. We study certified unlearning in decentralized networks with fixed topologies and propose RR-DU, a random-walk procedure that performs one projected gradient ascent step on the forget set at the unlearning client and a geometrically distributed number of projected descent steps on the retained data elsewhere, combined with subsampled Gaussian noise and projection onto a trust region around the original model. We provide (i) convergence guarantees in the convex case and stationarity guarantees in the nonconvex case, (ii) $(\varepsilon,\delta)$ network-unlearning certificates on client views via subsampled Gaussian R\'enyi DP (RDP) with segment-level subsampling, and (iii) deletion-capacity bounds that scale with the forget-to-local data ratio and quantify the effect of decentralization (network mixing and randomized subsampling) on the privacy-utility trade-off. Empirically, on image benchmarks (MNIST, CIFAR-10), RR-DU matches a given $(\varepsilon,\delta)$ while achieving higher test accuracy than decentralized DP baselines and reducing forget accuracy to random guessing ($\approx 10\%$).