DualOptim+: Bridging Shared and Decoupled Optimizer States for Better Machine Unlearning in Large Language Models

TL;DR

DualOptim+ is an innovative optimization framework that enhances machine unlearning in large language models by balancing shared and decoupled optimizer states, with a quantized variant reducing memory use.

Contribution

It introduces a dual-state optimizer architecture with a quantized version, improving unlearning efficiency and performance in large language models.

Findings

Consistently achieves better trade-offs in unlearning tasks.

Reduces memory overhead with DualOptim+ 8bit.

Effective across multiple tasks including safety alignment and multi-task learning.

Abstract

We propose DualOptim+, a novel optimization framework for improving machine unlearning in large language models. It introduces a base state to capture common representations shared by forgetting and retaining objectives and delta states to preserve objective-specific residuals. This architecture allows the optimizer to adaptively bridge shared and decoupled states based on the directional conflict between forgetting and retaining gradients. We further introduce DualOptim+ 8bit, a quantized variant that reduces memory overhead without compromising performance. Extensive experiments across fictitious and real-world unlearning, safety alignment, and multi-task learning tasks demonstrate that DualOptim+ consistently achieves a superior trade-off between different objectives. Codes are available at https://github.com/CityU-MLO/DualOptimPlus.