3D Space as a Scratchpad for Editable Text-to-Image Generation

TL;DR

This paper introduces a 3D reasoning workspace for visual language models, enabling spatial reasoning and precise, editable image generation from text prompts, significantly improving alignment and control.

Contribution

It proposes a novel 3D scratchpad framework that bridges language and image synthesis, supporting intuitive 3D edits that enhance spatial accuracy and controllability.

Findings

Achieves 32% improvement in text alignment on GenAI-Bench.

Supports intuitive 3D edits that reliably propagate into images.

Demonstrates the benefit of explicit 3D reasoning for image generation.

Abstract

Recent progress in large language models (LLMs) has shown that reasoning improves when intermediate thoughts are externalized into explicit workspaces, such as chain-of-thought traces or tool-augmented reasoning. Yet, visual language models (VLMs) lack an analogous mechanism for spatial reasoning, limiting their ability to generate images that accurately reflect geometric relations, object identities, and compositional intent. We introduce the concept of a spatial scratchpad -- a 3D reasoning substrate that bridges linguistic intent and image synthesis. Given a text prompt, our framework parses subjects and background elements, instantiates them as editable 3D meshes, and employs agentic scene planning for placement, orientation, and viewpoint selection. The resulting 3D arrangement is rendered back into the image domain with identity-preserving cues, enabling the VLM to generate spatially consistent and visually coherent outputs. Unlike prior 2D layout-based methods, our approach supports intuitive 3D edits that propagate reliably into final images. Empirically, it achieves a 32% improvement in text alignment on GenAI-Bench, demonstrating the benefit of explicit 3D reasoning for precise, controllable image generation. Our results highlight a new paradigm for vision-language models that deliberate not only in language, but also in space. Code and visualizations at https://oindrilasaha.github.io/3DScratchpad/