Thought Purity: A Defense Framework For Chain-of-Thought Attack

TL;DR

This paper introduces Thought Purity, a novel defense framework that actively detects and isolates malicious reasoning in large models' chain-of-thought processes, effectively defending against adversarial attacks without sacrificing performance.

Contribution

It proposes a safety-aware data pipeline combined with reinforcement learning to improve model robustness against Chain-of-Thought Attacks.

Findings

Significantly reduces attack success rate of CoTA.

Maintains or improves performance on benign tasks.

Demonstrates effectiveness across multiple model families.

Abstract

Large Reasoning Models (LRMs) leverage Chain-of-Thought (CoT) reasoning to solve complex tasks, but this explicit reasoning process introduces a critical vulnerability: adversarial manipulation of the thought chain itself, known as Chain-of-Thought Attacks (CoTA). Such attacks subtly corrupt the reasoning path to produce erroneous outputs, challenging conventional defenses that often sacrifice model utility for safety. To address this, we propose Thought Purity(TP), a defense framework that shifts from passive refusal to active reasoning recovery. TP integrates a safety-aware data pipeline with reinforcement learning, employing a dual-reward mechanism to teach models to dynamically identify and isolate malicious logic while preserving correct reasoning. Experiments on multiple model families demonstrate that TP significantly reduces the attack success rate of CoTA while maintaining or enhancing the model's performance on benign tasks.