Human-Aligned Procedural Level Generation Reinforcement Learning via Text-Level-Sketch Shared Representation

TL;DR

This paper introduces VIPCGRL, a deep reinforcement learning framework that integrates text, level, and sketches to improve human-aligned procedural content generation, demonstrating enhanced human-likeness and controllability.

Contribution

It proposes a shared embedding space trained via quadruple contrastive learning across modalities, enabling better alignment with human styles and goals in PCGRL.

Findings

VIPCGRL outperforms baselines in human-likeness metrics

Experimental results validate improved controllability and alignment

Human evaluations confirm enhanced human-centered behavior

Abstract

Human-aligned AI is a critical component of co-creativity, as it enables models to accurately interpret human intent and generate controllable outputs that align with design goals in collaborative content creation. This direction is especially relevant in procedural content generation via reinforcement learning (PCGRL), which is intended to serve as a tool for human designers. However, existing systems often fall short of exhibiting human-centered behavior, limiting the practical utility of AI-driven generation tools in real-world design workflows. In this paper, we propose VIPCGRL (Vision-Instruction PCGRL), a novel deep reinforcement learning framework that incorporates three modalities-text, level, and sketches-to extend control modality and enhance human-likeness. We introduce a shared embedding space trained via quadruple contrastive learning across modalities and human-AI styles, and align the policy using an auxiliary reward based on embedding similarity. Experimental results show that VIPCGRL outperforms existing baselines in human-likeness, as validated by both quantitative metrics and human evaluations. The code and dataset will be available upon publication.