Probing the Impact of Scale on Data-Efficient, Generalist Transformer World Models for Atari

TL;DR

This paper investigates how scaling transformer-based world models affects data efficiency and performance across Atari games, revealing environment-specific regimes and the stabilizing effect of joint training.

Contribution

It provides a detailed analysis of scaling behaviors in transformer world models on Atari, highlighting the importance of scaling strategies and joint training for generalist agents.

Findings

Environments fall into distinct scaling regimes with different fidelity trends.

Joint training stabilizes scaling dynamics across multiple environments.

Policies learned in simulation achieve high scores, close to expert performance.

Abstract

Developing generalist systems that retain human-like data efficiency is a central challenge. While world models (WMs) offer a promising path, existing research often conflates architectural mechanisms with the independent impact of model \emph{scale}. In this work, we use a minimalist transformer world model to analyze scaling behaviors on the Atari 100k benchmark, using fixed offline datasets derived from a presupposed expert policy. Our results reveal that environments fundamentally fall into distinct scaling regimes, even when constrained by identical offline data budgets and model capacities. For individual tasks, some environments naturally allow models to pass the interpolation threshold, yielding monotonic improvements in the overparameterized regime, while others remain trapped in the classical regime, where larger world models degrade fidelity. In the unified setting, i.e., a single transformer trained on a suite of 26 Atari environments, we uncover that joint training stabilizes scaling dynamics, ensuring monotonic gains across all environments, regardless of their distinct inherent scaling regimes. Finally, we demonstrate that improved fidelity translates directly to downstream control, with policies learned entirely within the simulated dynamics achieving a median expert-random-normalized score of 0.770. Our findings suggest that future progress lies as much in precise scaling strategies as in architectural innovation.