Scaling Latent Reasoning via Looped Language Models

TL;DR

Ouro introduces a family of pre-trained LoopLM models that embed reasoning into the pre-training process through iterative latent computation, achieving superior reasoning and manipulation capabilities compared to traditional chain-of-thought methods.

Contribution

This work presents Ouro, a novel pre-trained Looped Language Model that incorporates reasoning during training via iterative latent computation and entropy regularization, scaling to 7.7T tokens.

Findings

Ouro models outperform comparable SOTA LLMs on various benchmarks.

Superior reasoning capabilities are due to knowledge manipulation, not increased capacity.

LoopLM produces reasoning traces more aligned with final outputs than explicit CoT.

Abstract

Modern LLMs are trained to "think" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training data. We present and open-source Ouro, named after the recursive Ouroboros, a family of pre-trained Looped Language Models (LoopLM) that instead build reasoning into the pre-training phase through (i) iterative computation in latent space, (ii) an entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and 2.6B models enjoy superior performance that match the results of up to 12B SOTA LLMs across a wide range of benchmarks. Through controlled experiments, we show this advantage stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities. We also show that LoopLM yields reasoning traces more aligned with final outputs than explicit CoT. We hope our results show the potential of LoopLM as a novel scaling direction in the reasoning era. Our model is available here: http://ouro-llm.github.io.