Human-Like Navigation Behavior: A Statistical Evaluation Framework

TL;DR

This paper introduces a statistical framework using a non-parametric two-sample hypothesis test to evaluate how human-like artificial agents' navigation behaviors are, moving beyond mere task proficiency.

Contribution

It presents a novel hypothesis testing method that quantitatively compares artificial agent behavior to human behavior, providing a more realistic measure of believability.

Findings

The $p$-value correlates with human judgment of human-likeness.

The method effectively measures behavioral similarity between humans and AI agents.

Proficiency alone does not indicate human-like behavior.

Abstract

Recent advancements in deep reinforcement learning have brought forth an impressive display of highly skilled artificial agents capable of complex intelligent behavior. In video games, these artificial agents are increasingly deployed as non-playable characters (NPCs) designed to enhance the experience of human players. However, while it has been shown that the convincing human-like behavior of NPCs leads to increased engagement in video games, the believability of an artificial agent's behavior is most often measured solely by its proficiency at a given task. Recent work has hinted that proficiency alone is not sufficient to discern human-like behavior. Motivated by this, we build a non-parametric two-sample hypothesis test designed to compare the behaviors of artificial agents to those of human players. We show that the resulting $p$-value not only aligns with anonymous human judgment of human-like behavior, but also that it can be used as a measure of similarity.