Beyond Forgetting: Machine Unlearning Elicits Controllable Side Behaviors and Capabilities

TL;DR

This paper explores how machine unlearning in large language models can induce controllable behaviors and capabilities, revealing both risks and opportunities through the lens of the Linear Representation Hypothesis.

Contribution

It introduces a novel perspective on representation misdirection, demonstrating that unlearning can elicit controllable side behaviors and enhance capabilities in LLMs.

Findings

Unlearning can control model behaviors like truthfulness and sentiment.

Unlearning can improve in-context learning capabilities.

Side behaviors and capabilities are linked to high-level concept vectors.

Abstract

We consider Representation Misdirection (RM), a class of large language model (LLM) unlearning methods that achieve forgetting by redirecting the forget-representations, that is, latent representations of forget-samples, toward a target vector. Despite being important, the roles of the target vector used in RM, however, remain underexplored. Here, we approach and revisit RM through the lens of the Linear Representation Hypothesis. Specifically, if one can identify a one-dimensional representation corresponding to a high-level concept, the Linear Representation Hypothesis enables linear operations on this concept vector within the forget-representation space. Under this view, we hypothesize that, beyond forgetting, machine unlearning via RM elicits controllable emergent side behaviors and stronger side capabilities corresponding to the high-level concept. Our hypothesis is empirically validated across a wide range of tasks, including behavioral control (e.g., controlling unlearned models' truthfulness, sentiment, refusal, and language) and capability enhancement (e.g., improving unlearned models' in-context learning (ICL) capability). Our findings reveal that this phenomenon could be either a hidden risk if misused or a mechanism that can be harnessed for developing unlearned models that require stronger capabilities and controllable behaviors.