World-VLA-Loop: Closed-Loop Learning of Video World Model and VLA Policy

TL;DR

World-VLA-Loop introduces a closed-loop learning framework that jointly refines video world models and VLA policies, significantly improving robotic task performance with minimal real-world interaction.

Contribution

It presents a state-aware video world model and a co-evolving cycle of policy and world model refinement, advancing the integration of simulation and reinforcement learning for robotics.

Findings

Enhanced VLA policy performance in simulation and real-world tasks.

Effective refinement of world models through failure rollouts.

Minimal physical interaction needed for training.

Abstract

Recent progress in robotic world models has leveraged video diffusion transformers to predict future observations conditioned on historical states and actions. While these models can simulate realistic visual outcomes, they often exhibit poor action-following precision, hindering their utility for downstream robotic learning. In this work, we introduce World-VLA-Loop, a closed-loop framework for the joint refinement of world models and Vision-Language-Action (VLA) policies. We propose a state-aware video world model that functions as a high-fidelity interactive simulator by jointly predicting future observations and reward signals. To enhance reliability, we introduce the SANS dataset, which incorporates near-success trajectories to improve action-outcome alignment within the world model. This framework enables a closed-loop for reinforcement learning (RL) post-training of VLA policies entirely within a virtual environment. Crucially, our approach facilitates a co-evolving cycle: failure rollouts generated by the VLA policy are iteratively fed back to refine the world model precision, which in turn enhances subsequent RL optimization. Evaluations across simulation and real-world tasks demonstrate that our framework significantly boosts VLA performance with minimal physical interaction, establishing a mutually beneficial relationship between world modeling and policy learning for general-purpose robotics. Project page: https://showlab.github.io/World-VLA-Loop/.