Sword: Style-Robust World Models as Simulators via Dynamic Latent Bootstrapping for VLA Policy Post-Training

TL;DR

Sword introduces a robust World Model framework that enhances generalization and fidelity in vision-language-action tasks by disentangling visual textures and maintaining consistency through dynamic latent bootstrapping.

Contribution

The paper proposes Structure-Guided Style Augmentation and Dynamic Latent Bootstrapping to improve World Models' robustness and generalization in VLA policy training.

Findings

Outperforms baseline WoVR on LIBERO benchmark in generalization and robustness.

Significantly improves generation quality and fidelity of simulated environments.

Increases success rate of reinforcement-learning post-training for VLA models.

Abstract

The integration of Vision-Language-Action (VLA) models with World Models has gained increasing attention. One representative approach treats learned World Models as generative simulators, enabling policy optimization entirely within "imagination." However, when deployed as simulators for specific environments such as the LIBERO benchmark, existing World Models often suffer from poor generalization and long-horizon error accumulation. During closed-loop rollouts, these models are highly sensitive to initial-state perturbations; minor changes in color, illumination, and other visual factors can trigger cascading hallucinations, leading to severe blurriness or overexposure. Moreover, long-horizon error accumulation further degrades the quality and fidelity of predicted future states. These issues limit the reliability of World Models as simulators. To mitigate these problems, we propose Sword, a robust World Model framework. Our method introduces Structure-Guided Style Augmentation to disentangle the visual textures of interactive environments from task-relevant dynamics, thereby improving generalization. We further propose Dynamic Latent Bootstrapping, which maintains consistency between training and inference while keeping memory consumption low. Extensive experiments on the LIBERO benchmark show that our method significantly outperforms the baseline WoVR in terms of generalization, generation quality, robustness, fidelity, and the success rate of reinforcement-learning post-training for VLA models.