Visual Foresight for Robotic Stow: A Diffusion-Based World Model from Sparse Snapshots

TL;DR

FOREST is a diffusion-based world model that predicts post-stow bin configurations from sparse observations, improving warehouse planning and execution accuracy.

Contribution

It introduces a novel diffusion transformer model conditioned on stow intent, representing bin states with item-aligned masks for better foresight.

Findings

Substantially improves geometric accuracy of post-stow layouts

Maintains performance in downstream load-quality and reasoning tasks

Provides useful foresight signals for warehouse planning

Abstract

Automated warehouses execute millions of stow operations, where robots place objects into storage bins. For these systems it is valuable to anticipate how a bin will look from the current observations and the planned stow behavior before real execution. We propose FOREST, a stow-intent-conditioned world model that represents bin states as item-aligned instance masks and uses a latent diffusion transformer to predict the post-stow configuration from the observed context. Our evaluation shows that FOREST substantially improves the geometric agreement between predicted and true post-stow layouts compared with heuristic baselines. We further evaluate the predicted post-stow layouts in two downstream tasks, in which replacing the real post-stow masks with FOREST predictions causes only modest performance loss in load-quality assessment and multi-stow reasoning, indicating that our model can provide useful foresight signals for warehouse planning.