Heterogeneous Decentralized Machine Unlearning with Seed Model Distillation

TL;DR

This paper introduces HDUS, a decentralized machine unlearning framework using seed model distillation, enabling efficient and scalable unlearning in heterogeneous IoT environments.

Contribution

The paper proposes a novel decentralized unlearning framework that employs seed model distillation for erasable ensembles, supporting heterogeneous device models.

Findings

HDUS achieves state-of-the-art unlearning performance.

The framework is scalable to heterogeneous device models.

Extensive experiments validate its effectiveness on real-world datasets.

Abstract

As some recent information security legislation endowed users with unconditional rights to be forgotten by any trained machine learning model, personalized IoT service providers have to put unlearning functionality into their consideration. The most straightforward method to unlearn users' contribution is to retrain the model from the initial state, which is not realistic in high throughput applications with frequent unlearning requests. Though some machine unlearning frameworks have been proposed to speed up the retraining process, they fail to match decentralized learning scenarios. In this paper, we design a decentralized unlearning framework called HDUS, which uses distilled seed models to construct erasable ensembles for all clients. Moreover, the framework is compatible with heterogeneous on-device models, representing stronger scalability in real-world applications. Extensive experiments on three real-world datasets show that our HDUS achieves state-of-the-art performance.