Toward Efficient Data-Free Unlearning

TL;DR

This paper introduces ISPF, a novel data-free unlearning method that improves the retention of useful knowledge while removing unwanted information, outperforming previous approaches.

Contribution

The paper proposes the Inhibited Synthetic PostFilter (ISPF), a new technique that reduces over-filtering and enhances knowledge retention during data-free unlearning.

Findings

ISPF effectively reduces forgetting information.

ISPF outperforms existing data-free unlearning methods.

Experimental results validate the efficiency of ISPF.

Abstract

Machine unlearning without access to real data distribution is challenging. The existing method based on data-free distillation achieved unlearning by filtering out synthetic samples containing forgetting information but struggled to distill the retaining-related knowledge efficiently. In this work, we analyze that such a problem is due to over-filtering, which reduces the synthesized retaining-related information. We propose a novel method, Inhibited Synthetic PostFilter (ISPF), to tackle this challenge from two perspectives: First, the Inhibited Synthetic, by reducing the synthesized forgetting information; Second, the PostFilter, by fully utilizing the retaining-related information in synthesized samples. Experimental results demonstrate that the proposed ISPF effectively tackles the challenge and outperforms existing methods.