Automatable Evaluation Method Oriented toward Behaviour Believability for Video Games

TL;DR

The paper introduces an automated evaluation method for assessing the believability of agents in video games by comparing behavior vectors to human data, enabling efficient and scalable validation.

Contribution

It proposes a novel, vector-based evaluation approach that reduces reliance on human judgment during agent development and testing.

Findings

The method can distinguish between believable agents and humans.

Even simple evaluations reveal behavioral differences.

Results show potential despite analysis challenges.

Abstract

Classic evaluation methods of believable agents are time-consuming because they involve many human to judge agents. They are well suited to validate work on new believable behaviours models. However, during the implementation, numerous experiments can help to improve agents' believability. We propose a method which aim at assessing how much an agent's behaviour looks like humans' behaviours. By representing behaviours with vectors, we can store data computed for humans and then evaluate as many agents as needed without further need of humans. We present a test experiment which shows that even a simple evaluation following our method can reveal differences between quite believable agents and humans. This method seems promising although, as shown in our experiment, results' analysis can be difficult.