PerpetualWonder: Long-Horizon Action-Conditioned 4D Scene Generation

TL;DR

PerpetualWonder is a hybrid generative simulator that enables long-horizon, action-conditioned 4D scene generation from a single image by establishing a closed-loop system linking physical state and visual representation.

Contribution

It introduces the first true closed-loop system with a unified representation and a robust update mechanism for long-horizon, action-conditioned 4D scene generation.

Findings

Successfully simulates complex multi-step interactions from a single image.

Maintains physical plausibility and visual consistency over long horizons.

Outperforms previous methods in long-horizon scene generation.

Abstract

We introduce PerpetualWonder, a hybrid generative simulator that enables long-horizon, action-conditioned 4D scene generation from a single image. Current works fail at this task because their physical state is decoupled from their visual representation, which prevents generative refinements to update the underlying physics for subsequent interactions. PerpetualWonder solves this by introducing the first true closed-loop system. It features a novel unified representation that creates a bidirectional link between the physical state and visual primitives, allowing generative refinements to correct both the dynamics and appearance. It also introduces a robust update mechanism that gathers supervision from multiple viewpoints to resolve optimization ambiguity. Experiments demonstrate that from a single image, PerpetualWonder can successfully simulate complex, multi-step interactions from long-horizon actions, maintaining physical plausibility and visual consistency.