Scaling Competence, Shrinking Reasoning: Cognitive Signatures in Language Model Learning

TL;DR

This paper explores how language models develop reasoning skills during training, revealing a progression similar to human learning stages, and shows that reasoning tokens can diagnose training progress and optimize learning.

Contribution

It introduces a cognitive-inspired framework to analyze reasoning in language models, linking training dynamics with the Four Stages of Competence and proposing metrics for training diagnostics.

Findings

Reasoning token length peaks at conscious competence stage

Models retain performance without reasoning after training

Reasoning tokens can diagnose training stages and convergence

Abstract

We analyze reasoning in language models during task-specific fine-tuning and draws parallel between reasoning tokens--intermediate steps generated while solving problem and the human working memory. Drawing from cognitive science, we align training dynamics with the Four Stages of Competence: models initially produce incorrect outputs without reasoning, then begin reasoning (but still fail), eventually reason effectively, and finally solve tasks without explicit reasoning. We find that reasoning token length expands as performance improves, peaks at the stage of conscious competence, then declines as the model internalizes the task. Notably, after training, models retain performance even when reasoning is removed--suggesting it scaffolded learning but is no longer needed. This progression offers actionable insights: reasoning token dynamics can serve as a signal for diagnosing training stage, identifying convergence, and guiding early stopping. We propose metrics to track this trajectory and argue that reasoning behavior is valuable for understanding and optimizing reasoning model training.