Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

TL;DR

Simulation Distillation (SimDist) leverages physics simulators for pretraining world models, enabling rapid real-world adaptation in contact-rich robotic tasks by transferring learned components and updating only the dynamics model.

Contribution

SimDist introduces a scalable pretraining framework that distills simulator priors into world models and efficiently adapts them to real-world tasks with minimal updates.

Findings

SimDist enables rapid improvement in contact-rich manipulation tasks.

Prior methods struggle to adapt or degrade during online finetuning.

SimDist outperforms existing adaptation methods in various robotic tasks.

Abstract

Robot learning requires adaptation methods that improve reliably from limited, mixed-quality interaction data. This is especially challenging in long-horizon, contact-rich tasks, where end-to-end policy finetuning remains inefficient and brittle. World models offer a compelling alternative: by predicting the outcomes of candidate action sequences, they enable online planning through counterfactual reasoning. However, training action-conditioned robotic world models directly in the real world requires diverse data at impractical scale. We introduce Simulation Distillation (SimDist), a framework that uses physics simulators as a scalable source of action-conditioned robot experience. During pretraining, SimDist distills structural priors from the simulator into a world model that enables planning from raw real-world observations. During real-world adaptation, SimDist transfers the encoder, reward model, and value function learned in simulation, and updates only the latent dynamics model using real-world prediction losses. This reduces adaptation to supervised system identification while preserving dense, long-horizon planning signals for online improvement. Across contact-rich manipulation and quadruped locomotion tasks, SimDist rapidly improves with experience, while prior adaptation methods struggle to make progress or degrade during online finetuning. Project website and code: https://sim-dist.github.io