Solaris: Building a Multiplayer Video World Model in Minecraft

TL;DR

Solaris is a novel multiplayer video world model for Minecraft that captures multi-agent interactions, enabling consistent multi-view observations and advancing multi-agent environment modeling.

Contribution

We developed a multiplayer data collection system and a staged training pipeline for Solaris, enabling multi-agent modeling in video game environments.

Findings

Collected 12.64 million multiplayer frames for training.

Our architecture outperforms existing baselines.

Introduced Checkpointed Self Forcing for longer-horizon modeling.

Abstract

Existing action-conditioned video generation models (video world models) are limited to single-agent perspectives, failing to capture the multi-agent interactions of real-world environments. We introduce Solaris, a multiplayer video world model that simulates consistent multi-view observations. To enable this, we develop a multiplayer data system designed for robust, continuous, and automated data collection on video games such as Minecraft. Unlike prior platforms built for single-player settings, our system supports coordinated multi-agent interaction and synchronized videos + actions capture. Using this system, we collect 12.64 million multiplayer frames and propose an evaluation framework for multiplayer movement, memory, grounding, building, and view consistency. We train Solaris using a staged pipeline that progressively transitions from single-player to multiplayer modeling, combining bidirectional, causal, and Self Forcing training. In the final stage, we introduce Checkpointed Self Forcing, a memory-efficient Self Forcing variant that enables a longer-horizon teacher. Results show our architecture and training design outperform existing baselines. Through open-sourcing our system and models, we hope to lay the groundwork for a new generation of multi-agent world models.