CATNIP: LLM Unlearning via Calibrated and Tokenized Negative Preference Alignment

TL;DR

CATNIP introduces a novel unlearning method for large language models that calibrates gradient updates based on model confidence, enabling effective forgetting of undesirable knowledge without needing retention data.

Contribution

The paper proposes CATNIP, a new unlearning approach that improves control over forgetting by using token-level confidence, reducing data requirements and enhancing performance.

Findings

Effective unlearning without retention data.

Stronger forgetting and preservation tradeoffs.

Robust to data scarcity and length variation.

Abstract

Pretrained knowledge memorized in LLMs raises critical concerns over safety and privacy, which has motivated LLM Unlearning as a technique for selectively removing the influences of undesirable knowledge. Existing approaches, rooted in Gradient Ascent (GA), often degrade general domain knowledge while relying on retention data or curated contrastive pairs, which can be either impractical or data and computationally prohibitive. Negative Preference Alignment has been explored for unlearning to tackle the limitations of GA, which, however, remains confined by its choice of reference model and shows undermined performance in realistic data settings. These limitations raise two key questions: i) Can we achieve effective unlearning that quantifies model confidence in undesirable knowledge and uses it to calibrate gradient updates more precisely, thus reducing catastrophic forgetting? ii) Can we make unlearning robust to data scarcity and length variation? We answer both questions affirmatively with CATNIP (Calibrated and Tokenized Negative Preference Alignment), a principled method that rescales unlearning effects in proportion to the model's token-level confidence, thus ensuring fine-grained control over forgetting. Extensive evaluations on MUSE and WMDP benchmarks demonstrated that our work enables effective unlearning without requiring retention data or contrastive unlearning response pairs, with stronger knowledge forgetting and preservation tradeoffs than state-of-the-art methods.