Evaluating and Comparing Crowd Simulations: Perspectives from a Crowd Authoring Tool

TL;DR

This paper extends a crowd simulation authoring tool to include scenario evaluation metrics, validating a new quantitative measure for selecting optimal crowd configurations, with expert agreement supporting its effectiveness.

Contribution

It introduces a new quantitative metric for evaluating crowd simulation scenarios within an authoring tool, validated through experiments and comparison with existing metrics.

Findings

Experts agree with the proposed metric

The metric effectively differentiates crowd configurations

Validation across multiple scenarios confirms robustness

Abstract

Crowd simulation is a research area widely used in diverse fields, including gaming and security, assessing virtual agent movements through metrics like time to reach their goals, speed, trajectories, and densities. This is relevant for security applications, for instance, as different crowd configurations can determine the time people spend in environments trying to evacuate them. In this work, we extend WebCrowds, an authoring tool for crowd simulation, to allow users to build scenarios and evaluate them through a set of metrics. The aim is to provide a quantitative metric that can, based on simulation data, select the best crowd configuration in a certain environment. We conduct experiments to validate our proposed metric in multiple crowd simulation scenarios and perform a comparison with another metric found in the literature. The results show that experts in the domain of crowd scenarios agree with our proposed quantitative metric.