Learning Controllable 3D Level Generators

TL;DR

This paper applies PCGRL to 3D content generation in Minecraft, creating complex, controllable levels by training RL agents on functional constraints, demonstrating diversity, generalization, and utility for analyzing generator performance.

Contribution

It introduces new PCGRL tasks for 3D environments, showcasing the first application of RL-based generators with controllability and complex level creation in Minecraft.

Findings

RL agents generate diverse, complex 3D levels

Agents generalize to new initial states and control targets

Controllability tests help analyze generator success and failure

Abstract

Procedural Content Generation via Reinforcement Learning (PCGRL) foregoes the need for large human-authored data-sets and allows agents to train explicitly on functional constraints, using computable, user-defined measures of quality instead of target output. We explore the application of PCGRL to 3D domains, in which content-generation tasks naturally have greater complexity and potential pertinence to real-world applications. Here, we introduce several PCGRL tasks for the 3D domain, Minecraft (Mojang Studios, 2009). These tasks will challenge RL-based generators using affordances often found in 3D environments, such as jumping, multiple dimensional movement, and gravity. We train an agent to optimize each of these tasks to explore the capabilities of previous research in PCGRL. This agent is able to generate relatively complex and diverse levels, and generalize to random initial states and control targets. Controllability tests in the presented tasks demonstrate their utility to analyze success and failure for 3D generators.