Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

TL;DR

This paper introduces a novel recurrent latent reasoning architecture that scales test-time computation by iteratively reasoning in latent space, enabling improved reasoning performance without extensive training data or large context windows.

Contribution

The paper presents a new recurrent latent reasoning model that scales test-time compute through iterative unrolling, achieving significant reasoning improvements without specialized training data.

Findings

Model scales to 3.5 billion parameters and 800 billion tokens.

Performance on reasoning benchmarks improves significantly.

Achieves reasoning capabilities comparable to much larger models.

Abstract

We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space. Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time. This stands in contrast to mainstream reasoning models that scale up compute by producing more tokens. Unlike approaches based on chain-of-thought, our approach does not require any specialized training data, can work with small context windows, and can capture types of reasoning that are not easily represented in words. We scale a proof-of-concept model to 3.5 billion parameters and 800 billion tokens. We show that the resulting model can improve its performance on reasoning benchmarks, sometimes dramatically, up to a computation load equivalent to 50 billion parameters.