Tackling Federated Unlearning as a Parameter Estimation Problem

TL;DR

This paper presents a novel federated unlearning method using information theory and second-order Hessian information to efficiently erase specific data from models while maintaining high accuracy and privacy.

Contribution

It introduces a parameter estimation-based framework for federated unlearning that selectively resets sensitive parameters without full retraining.

Findings

Achieves near-random membership inference success rates

Maintains high accuracy (~0.9 normalized) after unlearning

Effectively neutralizes backdoor attacks

Abstract

Privacy regulations require the erasure of data from deep learning models. This is a significant challenge that is amplified in Federated Learning, where data remains on clients, making full retraining or coordinated updates often infeasible. This work introduces an efficient Federated Unlearning framework based on information theory, modeling leakage as a parameter estimation problem. Our method uses second-order Hessian information to identify and selectively reset only the parameters most sensitive to the data being forgotten, followed by minimal federated retraining. This model-agnostic approach supports categorical and client unlearning without requiring server access to raw client data after initial information aggregation. Evaluations on benchmark datasets demonstrate strong privacy (MIA success near random, categorical knowledge erased) and high performance (Normalized Accuracy against re-trained benchmarks of $\approx$ 0.9), while aiming for increased efficiency over complete retraining. Furthermore, in a targeted backdoor attack scenario, our framework effectively neutralizes the malicious trigger, restoring model integrity. This offers a practical solution for data forgetting in FL.