CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter

TL;DR

CrafterDojo introduces a suite of foundation models and tools that transform the Crafter environment into a lightweight, versatile platform for developing and testing general-purpose embodied AI agents, inspired by Minecraft.

Contribution

The paper presents CrafterDojo, a comprehensive set of foundation models, datasets, and tools that enable rapid prototyping and research in embodied AI within the Crafter environment.

Findings

Introduction of CrafterVPT, CrafterCLIP, and CrafterSteve-1 models.

Provision of datasets CrafterPlay and CrafterCaption.

Open-source codebase and benchmark evaluations available.

Abstract

Developing general-purpose embodied agents is a core challenge in AI. Minecraft provides rich complexity and internet-scale data, but its slow speed and engineering overhead make it unsuitable for rapid prototyping. Crafter offers a lightweight alternative that retains key challenges from Minecraft, yet its use has remained limited to narrow tasks due to the absence of foundation models that have driven progress in the Minecraft setting. In this paper, we present CrafterDojo, a suite of foundation models and tools that unlock the Crafter environment as a lightweight, prototyping-friendly, and Minecraft-like testbed for general-purpose embodied agent research. CrafterDojo addresses this by introducing CrafterVPT, CrafterCLIP, and CrafterSteve-1 for behavior priors, vision-language grounding, and instruction following, respectively. In addition, we provide toolkits for generating behavior and caption datasets (CrafterPlay and CrafterCaption), reference agent implementations, benchmark evaluations, and a complete open-source codebase.