Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse

TL;DR

This paper explores the development of generalist game-playing AI agents capable of mastering diverse games and creating new game worlds, aiming to advance towards Artificial General Intelligence.

Contribution

It introduces a comprehensive framework and roadmap for building generalist game agents through four pillars: Dataset, Model, Harness, and Benchmark.

Findings

Progression from single-game mastery to creating and evolving within a game multiverse.

Identification of five fundamental trade-offs in developing generalist game agents.

Proposal of a five-level roadmap towards omnipotent generalist agents.

Abstract

The real world unfolds along a single set of physics laws, yet human intelligence demonstrates a remarkable capacity to generalize experiences from this singular physical existence into a multiverse of games, each governed by entirely different rules, aesthetics, physics, and objectives. This omni-reality adaptability is a hallmark of general intelligence. As Artificial Intelligence progresses towards Artificial General Intelligence, the multiverse of games has evolved from mere entertainment into the ultimate ground for training and evaluating AGI. The pursuit of this generality has unfolded across four eras: from environment-specific symbolic and reinforcement learning agents, to current large foundation models acting as generalist players, and toward a future creator stage where agent both creates new game worlds and continually evolves within them. We trace the full lifecycle of a generalist game player along four interdependent pillars: Dataset, Model, Harness, and Benchmark. Every advance across these pillars can be read as an attempt to break one of five fundamental trade-offs that currently bound the whole system. Building on this end-to-end view, we chart a five-level roadmap, progressing from single-game mastery to the ultimate creator stage in which the agent simultaneously creates and evolves within theoretical game multiverse. Taken together, our work offers a unified lens onto a rapidly shifting field,and a principled path toward the omnipotent generalist agent capable of seamlessly mastering any challenge within the multiverse of games, thereby paving the way for AGI.