Learning Massively Multitask World Models for Continuous Control

TL;DR

This paper introduces Newt, a multitask world model trained on hundreds of tasks with online interaction, demonstrating improved performance, data efficiency, and adaptability in continuous control tasks.

Contribution

It presents a novel approach combining large-scale pretraining and online learning for multitask control, along with a new benchmark of 200 diverse tasks.

Findings

Newt outperforms strong baselines in multitask settings.

It exhibits strong open-loop control capabilities.

Enables rapid adaptation to unseen tasks.

Abstract

General-purpose control demands agents that act across many tasks and embodiments, yet research on reinforcement learning (RL) for continuous control remains dominated by single-task or offline regimes, reinforcing a view that online RL does not scale. Inspired by the foundation model recipe (large-scale pretraining followed by light RL) we ask whether a single agent can be trained on hundreds of tasks with online interaction. To accelerate research in this direction, we introduce a new benchmark with 200 diverse tasks spanning many domains and embodiments, each with language instructions, demonstrations, and optionally image observations. We then present \emph{Newt}, a language-conditioned multitask world model that is first pretrained on demonstrations to acquire task-aware representations and action priors, and then jointly optimized with online interaction across all tasks. Experiments show that Newt yields better multitask performance and data-efficiency than a set of strong baselines, exhibits strong open-loop control, and enables rapid adaptation to unseen tasks. We release our environments, demonstrations, code for training and evaluation, as well as 200+ checkpoints.