Is my model "mind blurting"? Interpreting the dynamics of reasoning tokens with Recurrence Quantification Analysis (RQA)

TL;DR

This paper introduces Recurrence Quantification Analysis (RQA) as a novel method to analyze the reasoning dynamics of large models during inference, capturing patterns beyond simple response length.

Contribution

The paper presents RQA as a new, non-textual approach to analyze reasoning chains, providing insights into model behavior and task complexity that are not accessible through traditional metrics.

Findings

RQA captures signals not reflected by response length.

RQA improves prediction of task complexity by 8%.

RQA reveals patterns of repetition and stalling in latent representations.

Abstract

Test-time compute is central to large reasoning models, yet analysing their reasoning behaviour through generated text is increasingly impractical and unreliable. Response length is often used as a brute proxy for reasoning effort, but this metric fails to capture the dynamics and effectiveness of the Chain of Thoughts (CoT) or the generated tokens. We propose Recurrence Quantification Analysis (RQA) as a non-textual alternative for analysing model's reasoning chains at test time. By treating token generation as a dynamical system, we extract hidden embeddings at each generation step and apply RQA to the resulting trajectories. RQA metrics, including Determinism and Laminarity, quantify patterns of repetition and stalling in the model's latent representations. Analysing 3,600 generation traces from DeepSeek-R1-Distill, we show that RQA captures signals not reflected by response length, but also substantially improves prediction of task complexity by 8\%. These results help establish RQA as a principled tool for studying the latent token generation dynamics of test-time scaling in reasoning models.