Invisible Watermarks, Visible Gains: Steering Machine Unlearning with Bi-Level Watermarking Design

TL;DR

This paper introduces Water4MU, a novel watermarking-based approach for machine unlearning that strategically modifies data content to enable precise removal of sensitive data, improving unlearning efficiency and model utility.

Contribution

The paper proposes a bi-level optimization framework, Water4MU, integrating digital watermarking with machine unlearning to enhance data removal effectiveness and model performance.

Findings

Water4MU effectively improves unlearning in image classification and generation.

Watermarking facilitates precise data removal without degrading unrelated model tasks.

Outperforms existing unlearning methods in challenging forget scenarios.

Abstract

With the increasing demand for the right to be forgotten, machine unlearning (MU) has emerged as a vital tool for enhancing trust and regulatory compliance by enabling the removal of sensitive data influences from machine learning (ML) models. However, most MU algorithms primarily rely on in-training methods to adjust model weights, with limited exploration of the benefits that data-level adjustments could bring to the unlearning process. To address this gap, we propose a novel approach that leverages digital watermarking to facilitate MU by strategically modifying data content. By integrating watermarking, we establish a controlled unlearning mechanism that enables precise removal of specified data while maintaining model utility for unrelated tasks. We first examine the impact of watermarked data on MU, finding that MU effectively generalizes to watermarked data. Building on this, we introduce an unlearning-friendly watermarking framework, termed Water4MU, to enhance unlearning effectiveness. The core of Water4MU is a bi-level optimization (BLO) framework: at the upper level, the watermarking network is optimized to minimize unlearning difficulty, while at the lower level, the model itself is trained independently of watermarking. Experimental results demonstrate that Water4MU is effective in MU across both image classification and image generation tasks. Notably, it outperforms existing methods in challenging MU scenarios, known as "challenging forgets".