Diagnosing Pathological Chain-of-Thought in Reasoning Models

TL;DR

This paper introduces a practical toolkit with metrics to diagnose and differentiate between three types of reasoning failures in large language models, enhancing AI safety and model monitoring.

Contribution

It defines and measures three distinct CoT reasoning pathologies and validates the approach using models deliberately trained to exhibit these issues.

Findings

Developed simple, task-agnostic metrics for CoT pathologies

Created model organisms with specific reasoning failures

Provided a practical toolkit for training-time monitoring

Abstract

Chain-of-thought (CoT) reasoning is fundamental to modern LLM architectures and represents a critical intervention point for AI safety. However, CoT reasoning may exhibit failure modes that we note as pathologies, which prevent it from being useful for monitoring. Prior work has identified three distinct pathologies: post-hoc rationalization, where models generate plausible explanations backwards from predetermined answers; encoded reasoning, where intermediate steps conceal information within seemingly interpretable text; and internalized reasoning, where models replace explicit reasoning with meaningless filler tokens while computing internally. To better understand and discriminate between these pathologies, we create a set of concrete metrics that are simple to implement, computationally inexpensive, and task-agnostic. To validate our approach, we develop model organisms deliberately trained to exhibit specific CoT pathologies. Our work provides a practical toolkit for assessing CoT pathologies, with direct implications for training-time monitoring.