Real-Time Progress Prediction in Reasoning Language Models

TL;DR

This paper explores the feasibility of real-time progress prediction in reasoning language models, introducing a method to generate and monitor progress estimates during inference with promising accuracy.

Contribution

It proposes a two-stage fine-tuning approach enabling reasoning models to produce real-time progress estimates, improving transparency and oversight.

Findings

Average error of 10% for sequences under 16,000 tokens

Progress prediction is feasible with linear probes and fine-tuning

Enhances interpretability of long reasoning processes

Abstract

Recent advances in reasoning language models -- particularly those that use long, latent chains of thought -- have demonstrated remarkable capabilities in complex, agentic tasks. However, as these models operate over increasingly extended time horizons, their internal progress becomes opaque to users, complicating expectation management and real-time oversight. In this work, we investigate whether real-time progress prediction is feasible. We discretize progress and train a linear probe to classify reasoning states. We then introduce a two-stage fine-tuning approach that enables reasoning models to generate progress estimates (0$\rightarrow$100\%) during inference. Our best fine-tuned model achieves an average error of 10\% for sequences less than 16,000 tokens, offering a practical mechanism for monitoring and interpreting model reasoning in real time.