Toward Human-AI Alignment in Large-Scale Multi-Player Games

TL;DR

This paper introduces an interpretable framework for evaluating human-AI behavioral alignment in complex multiplayer games, analyzing extensive gameplay data and comparing AI and human behaviors in a high-dimensional behavioral space.

Contribution

It proposes a novel behavior manifold framework to interpret and compare human and AI gameplay behaviors at a high level, advancing human-AI alignment research in multiplayer gaming.

Findings

Humans show variability in fight-flight and explore-exploit behaviors.

AI agents tend to be more uniform and engage mainly in solo play.

Significant behavioral differences highlight the need for interpretable AI evaluation.

Abstract

Achieving human-AI alignment in complex multi-agent games is crucial for creating trustworthy AI agents that enhance gameplay. We propose a method to evaluate this alignment using an interpretable task-sets framework, focusing on high-level behavioral tasks instead of low-level policies. Our approach has three components. First, we analyze extensive human gameplay data from Xbox's Bleeding Edge (100K+ games), uncovering behavioral patterns in a complex task space. This task space serves as a basis set for a behavior manifold capturing interpretable axes: fight-flight, explore-exploit, and solo-multi-agent. Second, we train an AI agent to play Bleeding Edge using a Generative Pretrained Causal Transformer and measure its behavior. Third, we project human and AI gameplay to the proposed behavior manifold to compare and contrast. This allows us to interpret differences in policy as higher-level behavioral concepts, e.g., we find that while human players exhibit variability in fight-flight and explore-exploit behavior, AI players tend towards uniformity. Furthermore, AI agents predominantly engage in solo play, while humans often engage in cooperative and competitive multi-agent patterns. These stark differences underscore the need for interpretable evaluation, design, and integration of AI in human-aligned applications. Our study advances the alignment discussion in AI and especially generative AI research, offering a measurable framework for interpretable human-agent alignment in multiplayer gaming.