One Step Forward and K Steps Back: Better Reasoning with Denoising Recursion Models

TL;DR

This paper introduces Denoising Recursion Models, which improve iterative refinement in reasoning tasks by training models to reverse noise over multiple steps, leading to better alignment and performance.

Contribution

The paper proposes a novel training method for recursive models that enhances reasoning by reversing noise over multiple steps, outperforming previous models on complex tasks.

Findings

Outperforms Tiny Recursion Model on ARC-AGI

Better alignment of training and testing behaviors

Encourages non-greedy, forward-looking generation

Abstract

Looped transformers scale computational depth without increasing parameter count by repeatedly applying a shared transformer block and can be used for iterative refinement, where each loop rewrites a full fixed-size prediction in parallel. On difficult problems, such as those that require search-like computation, reaching a highly structured solution starting from noise can require long refinement trajectories. Learning such trajectories is challenging when training specifies only the target solution and provides no supervision over the intermediate refinement path. Diffusion models tackle this issue by corrupting data with varying magnitudes of noise and training the model to reverse it in a \textit{single step}. However, this process misaligns training and testing behaviour. We introduce Denoising Recursion Models, a method that similarly corrupts data with noise but trains the model to reverse the corruption over \textit{multiple} recursive steps. This strategy provides a tractable curriculum of intermediate states, while better aligning training with testing and incentivizing non-greedy, forward-looking generation. Through extensive experiments, we show this approach outperforms the Tiny Recursion Model (TRM) on ARC-AGI, where it recently achieved breakthrough performance.