SpiralThinker: Latent Reasoning through an Iterative Process with Text-Latent Interleaving

TL;DR

SpiralThinker introduces a stabilized iterative latent reasoning framework that interleaves latent and textual reasoning, achieving state-of-the-art results across various reasoning tasks.

Contribution

It presents a novel method combining progressive alignment and structured annotations to stabilize and improve latent reasoning dynamics.

Findings

Achieves state-of-the-art performance on reasoning tasks.

Both iteration and alignment are essential for effective reasoning.

Proper alignment and iteration numbers vary by dataset.

Abstract

Recent advances in large reasoning models have been driven by reinforcement learning and test-time scaling, accompanied by growing interest in latent rather than purely textual reasoning. However, existing latent reasoning methods lack mechanisms to ensure stable reasoning dynamics in latent space and a systematic way to interleave implicit and explicit reasoning. We introduce SpiralThinker, a stabilized iterative latent reasoning framework that performs iterative updates over latent representations while interleaving latent and textual reasoning steps. At its core, it combines a progressive alignment objective that explicitly regulates latent representations across iterations with structured annotations for text-latent interleaving, thereby stabilizing latent updates and maintaining coherence with textual reasoning. Across mathematical, logical, and commonsense reasoning tasks, SpiralThinker achieves state-of-the-art performance among latent reasoning baselines. Further analysis shows that both iteration and alignment are essential, that the optimal numbers of latent tokens and iterations vary by dataset, and that proper alignment is crucial for effective iterative latent reasoning. Overall, SpiralThinker bridges iterative computation and latent reasoning, demonstrating that aligned iterative updates can reliably steer reasoning in the latent space.