The Kinetics of Reasoning: How Chain-of-Thought Shapes Learning in Transformers?

TL;DR

This paper explores how chain-of-thought supervision influences transformer learning dynamics, revealing that it accelerates generalization but is limited by task complexity, and introduces a kinetic model to understand these effects.

Contribution

It introduces a kinetic modeling framework for transformer learning, characterizes the dynamic emergence of trace faithfulness, and analyzes how CoT mechanistically alters internal computations.

Findings

CoT accelerates generalization but struggles with complex tasks

A kinetic model quantifies learning speed and shape variations

Trace faithfulness is a dynamic property that develops during training

Abstract

Chain-of-thought (CoT) supervision can substantially improve transformer performance, yet the mechanisms by which models learn to follow and benefit from CoT remain poorly understood. We investigate these learning dynamics through the lens of grokking by pretraining transformers on symbolic reasoning tasks with tunable algorithmic complexity and controllable data composition to study their generalization. Models were trained under two settings: (i) producing only final answers, and (ii) emitting explicit CoT traces before answering. Our results show that while CoT generally improves task performance, its benefits depend on task complexity. To quantify these effects, we model the accuracy of the logarithmic training steps with a three-parameter logistic curve, revealing how the learning speed and shape vary with task complexity, data distribution, and the presence of CoT supervision. We also uncover a transient trace unfaithfulness phase: early in training, models often produce correct answers while skipping or contradicting CoT steps, before later aligning their reasoning traces with answers. Empirically, we (1) demonstrate that CoT accelerates generalization but does not overcome tasks with higher algorithmic complexity, such as finding list intersections; (2) introduce a kinetic modeling framework for understanding transformer learning; (3) characterize trace faithfulness as a dynamic property that emerges over training; and (4) show CoT alters internal transformer computation mechanistically.