LoReUn: Data Itself Implicitly Provides Cues to Improve Machine Unlearning

TL;DR

This paper introduces LoReUn, a novel data reweighting method for machine unlearning that leverages data loss to better forget undesired data, improving safety and effectiveness in generative models.

Contribution

The paper proposes Loss-based Reweighting Unlearning (LoReUn), a simple, effective strategy that dynamically adjusts data importance during unlearning based on loss, with minimal computational cost.

Findings

LoReUn improves unlearning accuracy in image classification and generation.

It reduces harmful content in text-to-image diffusion models.

The method is plug-and-play and computationally efficient.

Abstract

Recent generative models face significant risks of producing harmful content, which has underscored the importance of machine unlearning (MU) as a critical technique for eliminating the influence of undesired data. However, existing MU methods typically assign the same weight to all data to be forgotten, which makes it difficult to effectively forget certain data that is harder to unlearn than others. In this paper, we empirically demonstrate that the loss of data itself can implicitly reflect its varying difficulty. Building on this insight, we introduce Loss-based Reweighting Unlearning (LoReUn), a simple yet effective plug-and-play strategy that dynamically reweights data during the unlearning process with minimal additional computational overhead. Our approach significantly reduces the gap between existing MU methods and exact unlearning in both image classification and generation tasks, effectively enhancing the prevention of harmful content generation in text-to-image diffusion models.