Orthogonal Soft Pruning for Efficient Class Unlearning

TL;DR

FedOrtho introduces an orthogonalized kernel approach with soft pruning for efficient, controllable class unlearning in federated learning, achieving high forgetting quality, low costs, and strong retention.

Contribution

The paper proposes FedOrtho, a novel federated unlearning framework combining orthogonalized kernels and one-shot soft pruning for effective class unlearning.

Findings

Achieves over 98% forgetting quality on multiple datasets.

Reduces computational and communication costs by 2-3 orders of magnitude.

Maintains over 97% retention accuracy and mitigates membership inference risks.

Abstract

Efficient and controllable data unlearning in federated learning remains challenging, due to the trade-off between forgetting and retention performance. Especially under non-independent and identically distributed (non-IID) settings, where deep feature entanglement exacerbates this dilemma. To address this challenge, we propose FedOrtho, a federated unlearning framework that combines orthogonalized deep convolutional kernels with an activation-driven controllable one-shot soft pruning (OSP) mechanism. FedOrtho enforces kernel orthogonality and local-global alignment to decouple feature representations and mitigate client drift. This structural independence enables precise one-shot pruning of forgetting-related kernels while preserving retained knowledge. FedOrtho achieves SOTA performance on CIFAR-10, CIFAR100 and TinyImageNet with ResNet and VGG frameworks, verifying that FedOrtho supports class-, client-, and sample-level unlearning with over 98% forgetting quality. It reduces computational and communication costs by 2-3 orders of magnitude in federated settings and achieves subsecond-level erasure in centralized scenarios while maintaining over 97% retention accuracy and mitigating membership inference risks.