Lethe:Adapter-Augmented Dual-Stream Update for Persistent Knowledge Erasure in Federated Unlearning

TL;DR

Lethe is a federated unlearning method that prevents reactivation of erased knowledge during continued training by de-correlating unlearned information from retained knowledge, ensuring persistent erasure in federated models.

Contribution

It introduces a novel adapter-augmented dual-stream update mechanism that maintains knowledge erasure during ongoing federated training, addressing the issue of knowledge resurfacing.

Findings

Supports unlearning at all levels in federated systems.

Maintains low resurfacing rate (<1%) after multiple training rounds.

Outperforms existing methods in persistent knowledge erasure.

Abstract

Federated unlearning (FU) aims to erase designated client-level, class-level, or sample-level knowledge from a global model. Existing studies commonly assume that the collaboration ends up with the unlearning operation, overlooking the follow-up situation where the federated training continues over the remaining data.We identify a critical failure mode, termed Knowledge resurfacing, by revealing that continued training can re-activate unlearned knowledge and cause the removed influence to resurface in the global model. To address this, we propose Lethe, a novel federated unlearning method that de-correlates knowledge to be unlearned from knowledge to be retained, ensuring persistent erasure during continued training.Lethe follows a Reshape--Rectify--Restore pipeline: a temporary adapter is first trained with gradient ascent on the unlearning data to obtain magnified updates, which is then used as corrective signals to diverge layer-wise rectification on the remaining updates in two streams. Finally, the adapter is removed and a short recovery stage is performed on the retained data. Our experiments show that Lethe supports unlearning in the federated system at all levels in a unified manner and maintains superior persistence (Resurfacing Rate <1% in most cases) even after numerous rounds of follow-up training.