The Point of No Return: Counterfactual Localization of Deceptive Commitment in Language-Model Reasoning

TL;DR

This paper introduces counterfactual localization to identify when language models become committed to deception during reasoning, using a large-scale, environment-agnostic dataset and attention-based features.

Contribution

It presents a novel method for pinpointing deceptive commitment points in reasoning traces and provides a large, diverse dataset for studying deception in language models.

Findings

Attention-based transition features generalize across environments.

Compact attention-head sets can suppress deceptive commitment.

Detected commitment points align with interpretable decision shifts.

Abstract

Existing deception datasets label completed outputs as honest or deceptive, treating deception as a property of the final response rather than a function of the model's reasoning trace. This obscures a more fundamental question: when does a language model become committed to deception? We introduce counterfactual localization: for each sentence prefix in a reasoning trace, we fix the prefix, resample continuations, and estimate the probability of a deceptive outcome. To scale this, we construct five environments (spanning strategic bluffing, maze guidance, financial advice, used-car sales, and offer negotiation) in which deception is never prompted but emerges from strategic incentives and labels follow mechanically from environment state rather than subjective human judgment. The resulting corpus localizes $\sim$1.46M sentences across four reasoning models, drawn from over 94.1M sampled continuations, 91.5B generated tokens, and over 100K scenarios. Sentence-level human evaluation confirms that detected commitment points correspond to interpretable shifts in decision state. Using this resource, we show that lexical cues for commitment prediction transfer poorly across environments, whereas attention-based transition features generalize out of distribution, suggesting that deceptive commitment is reflected in reusable changes in reasoning dynamics rather than surface form. We further identify compact attention-head sets (under 10% of heads) that, selected on one environment, causally suppress deceptive commitment across held-out environments. We release the corpus as a substrate for studying deception, and more broadly commitment, in language-model reasoning.