Dreaming the Unseen: World Model-regularized Diffusion Policy for Out-of-Distribution Robustness

TL;DR

This paper introduces Dream Diffusion Policy (DDP), a novel framework that enhances visuomotor control robustness against out-of-distribution disturbances by integrating a diffusion world model for predictive imagination and adaptive visual stream abandonment.

Contribution

The paper presents a new diffusion-based world model integrated into policies, enabling real-time detection of anomalies and reliance on internal imagination to improve OOD robustness.

Findings

Achieves 73.8% OOD success rate on MetaWorld, significantly higher than baseline.

Maintains 83.3% success under severe spatial shifts, outperforming previous methods.

Retains 76.7% success rate using only open-loop imagination after initialization.

Abstract

Diffusion policies excel at visuomotor control but often fail catastrophically under severe out-of-distribution (OOD) disturbances, such as unexpected object displacements or visual corruptions. To address this vulnerability, we introduce the Dream Diffusion Policy (DDP), a framework that deeply integrates a diffusion world model into the policy's training objective via a shared 3D visual encoder. This co-optimization endows the policy with robust state-prediction capabilities. When encountering sudden OOD anomalies during inference, DDP detects the real-imagination discrepancy and actively abandons the corrupted visual stream. Instead, it relies on its internal "imagination" (autoregressively forecasted latent dynamics) to safely bypass the disruption, generating imagined trajectories before smoothly realigning with physical reality. Extensive evaluations demonstrate DDP's exceptional resilience. Notably, DDP achieves a 73.8% OOD success rate on MetaWorld (vs. 23.9% without predictive imagination) and an 83.3% success rate under severe real-world spatial shifts (vs. 3.3% without predictive imagination). Furthermore, as a stress test, DDP maintains a 76.7% real-world success rate even when relying entirely on open-loop imagination post-initialization.