Spatial Reasoning with Denoising Models

TL;DR

This paper introduces Spatial Reasoning Models (SRMs), a novel framework using denoising generative models for reasoning over continuous spatial variables, addressing hallucination issues and improving reasoning accuracy significantly.

Contribution

The paper presents SRMs as a new approach for spatial reasoning with generative models, including a benchmark for evaluating reasoning quality and insights into generation order and sampling strategies.

Findings

Order of generation can be predicted by the denoising network

Reasoning accuracy improved from less than 1% to over 50%

Benchmark datasets and tools are provided for future research

Abstract

We introduce Spatial Reasoning Models (SRMs), a framework to perform reasoning over sets of continuous variables via denoising generative models. SRMs infer continuous representations on a set of unobserved variables, given observations on observed variables. Current generative models on spatial domains, such as diffusion and flow matching models, often collapse to hallucination in case of complex distributions. To measure this, we introduce a set of benchmark tasks that test the quality of complex reasoning in generative models and can quantify hallucination. The SRM framework allows to report key findings about importance of sequentialization in generation, the associated order, as well as the sampling strategies during training. It demonstrates, for the first time, that order of generation can successfully be predicted by the denoising network itself. Using these findings, we can increase the accuracy of specific reasoning tasks from <1% to >50%. Our project website provides additional videos, code, and the benchmark datasets: https://geometric-rl.mpi-inf.mpg.de/srm