SIMA 2: A Generalist Embodied Agent for Virtual Worlds

TL;DR

SIMA 2 is a versatile embodied agent capable of understanding and acting in diverse 3D virtual worlds, demonstrating advanced reasoning, interaction, and autonomous learning capabilities, marking progress toward adaptable agents in virtual and real environments.

Contribution

Introduces SIMA 2, a generalist embodied agent built on Gemini, capable of complex interactions, reasoning, and autonomous skill acquisition in virtual worlds, surpassing prior simple command-based models.

Findings

SIMA 2 approaches human performance in diverse games.

The agent generalizes well to unseen environments.

It can autonomously learn new skills through self-generated tasks.

Abstract

We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a significant step toward active, goal-directed interaction within an embodied environment. Unlike prior work (e.g., SIMA 1) limited to simple language commands, SIMA 2 acts as an interactive partner, capable of reasoning about high-level goals, conversing with the user, and handling complex instructions given through language and images. Across a diverse portfolio of games, SIMA 2 substantially closes the gap with human performance and demonstrates robust generalization to previously unseen environments, all while retaining the base model's core reasoning capabilities. Furthermore, we demonstrate a capacity for open-ended self-improvement: by leveraging Gemini to generate tasks and provide rewards, SIMA 2 can autonomously learn new skills from scratch in a new environment. This work validates a path toward creating versatile and continuously learning agents for both virtual and, eventually, physical worlds.