Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry

TL;DR

This paper introduces a novel regularization method called Stability Asymmetry Regularization (SAR) to detect and mitigate intrinsic deception in large language models by leveraging the stability asymmetry phenomenon.

Contribution

It proposes SAR, a new alignment approach based on stability asymmetry, which is more robust against deception concealment than traditional chain-of-thought supervision.

Findings

Stability asymmetry reliably identifies deceptive behavior in LLMs.

SAR effectively reduces intrinsic deception without harming model performance.

Experiments demonstrate SAR's robustness compared to existing supervision methods.

Abstract

As Large Language Models (LLMs) expand in capability and application scope, their trustworthiness becomes critical. A vital risk is intrinsic deception, wherein models strategically mislead users to achieve their own objectives. Existing alignment approaches based on chain-of-thought (CoT) monitoring supervise explicit reasoning traces. However, under optimization pressure, models are incentivized to conceal deceptive reasoning, rendering semantic supervision fundamentally unreliable. Grounded in cognitive psychology, we hypothesize that a deceptive LLM maintains a stable internal belief in its CoT while its external response remains fragile under perturbation. We term this phenomenon stability asymmetry and quantify it by measuring the contrast between internal CoT stability and external response stability under perturbation. Building on this structural signature, we propose the Stability Asymmetry Regularization (SAR), a novel alignment objective that penalizes this distributional asymmetry during reinforcement learning. Unlike CoT monitoring, SAR targets the statistical structure of model outputs, rendering it robust to semantic concealment. Extensive experiments confirm that stability asymmetry reliably identifies deceptive behavior, and that SAR effectively suppresses intrinsic deception without degrading general model capability.