Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skills

TL;DR

This paper introduces R2MU, a novel unlearning method for large reasoning models that effectively erases sensitive reasoning traces and answers while maintaining reasoning capabilities, addressing safety concerns in chain-of-thought models.

Contribution

The paper presents R2MU, a new approach for unlearning in LRMs that targets reasoning traces, not just final answers, improving safety without sacrificing reasoning skills.

Findings

R2MU significantly reduces sensitive information leakage in reasoning traces.

R2MU maintains reasoning performance while erasing sensitive data.

Experiments on state-of-the-art models validate the effectiveness of R2MU.

Abstract

Recent advances in large reasoning models (LRMs) have enabled strong chain-of-thought (CoT) generation through test-time computation. While these multi-step reasoning capabilities represent a major milestone in language model performance, they also introduce new safety risks. In this work, we present the first systematic study to revisit the problem of machine unlearning in the context of LRMs. Machine unlearning refers to the process of removing the influence of sensitive, harmful, or undesired data or knowledge from a trained model without full retraining. We show that conventional unlearning algorithms, originally designed for non-reasoning models, are inadequate for LRMs. In particular, even when final answers are successfully erased, sensitive information often persists within the intermediate reasoning steps, i.e., CoT trajectories. To address this challenge, we extend conventional unlearning and propose Reasoning-aware Representation Misdirection for Unlearning ($R^2MU$), a novel method that effectively suppresses sensitive reasoning traces and prevents the generation of associated final answers, while preserving the model's reasoning ability. Our experiments demonstrate that $R^2MU$ significantly reduces sensitive information leakage within reasoning traces and achieves strong performance across both safety and reasoning benchmarks, evaluated on state-of-the-art models such as DeepSeek-R1-Distill-LLaMA-8B and DeepSeek-R1-Distill-Qwen-14B.