Forget by Uncertainty: Orthogonal Entropy Unlearning for Quantized Neural Networks

TL;DR

This paper introduces OEU, a novel unlearning framework for quantized neural networks that maximizes uncertainty on forgotten data and orthogonally projects gradients to preserve model utility, addressing key challenges in privacy-aware model unlearning.

Contribution

The paper proposes a new unlearning method combining entropy maximization and gradient orthogonal projection, offering theoretical guarantees and improved performance over existing approaches.

Findings

OEU achieves superior forgetting effectiveness.

OEU maintains higher retain accuracy.

Extensive experiments validate the method's advantages.

Abstract

The deployment of quantized neural networks on edge devices, combined with privacy regulations like GDPR, creates an urgent need for machine unlearning in quantized models. However, existing methods face critical challenges: they induce forgetting by training models to memorize incorrect labels, conflating forgetting with misremembering, and employ scalar gradient reweighting that cannot resolve directional conflicts between gradients. We propose OEU, a novel Orthogonal Entropy Unlearning framework with two key innovations: 1) Entropy-guided unlearning maximizes prediction uncertainty on forgotten data, achieving genuine forgetting rather than confident misprediction, and 2) Gradient orthogonal projection eliminates interference by projecting forgetting gradients onto the orthogonal complement of retain gradients, providing theoretical guarantees for utility preservation under first-order approximation. Extensive experiments demonstrate that OEU outperforms existing methods in both forgetting effectiveness and retain accuracy.