GUARD: Constructing Realistic Two-Player Matrix and Security Games for Benchmarking Game-Theoretic Algorithms

TL;DR

This paper introduces GUARD, a flexible framework for generating realistic security and matrix game benchmarks using open datasets, enabling better evaluation of game-theoretic algorithms beyond traditional random or classical games.

Contribution

The paper presents a novel framework that creates realistic game instances from real-world data, addressing the lack of accessible datasets for benchmarking security games.

Findings

Generated games are more realistic and diverse than random baselines.

Benchmarking shows significant differences in algorithm performance on realistic vs. random games.

Theoretical analysis reveals limitations of random game benchmarks.

Abstract

Game-theoretic algorithms are commonly benchmarked on recreational games, classical constructs from economic theory such as congestion and dispersion games, or entirely random game instances. While the past two decades have seen the rise of security games -- grounded in real-world scenarios like patrolling and infrastructure protection -- their practical evaluation has been hindered by limited access to the datasets used to generate them. In particular, although the structural components of these games (e.g., patrol paths derived from maps) can be replicated, the critical data defining target values -- central to utility modeling -- remain inaccessible. In this paper, we introduce a flexible framework that leverages open-access datasets to generate realistic matrix and security game instances. These include animal movement data for modeling anti-poaching scenarios and demographic and infrastructure data for infrastructure protection. Our framework allows users to customize utility functions and game parameters, while also offering a suite of preconfigured instances. We provide theoretical results highlighting the degeneracy and limitations of benchmarking on random games, and empirically compare our generated games against random baselines across a variety of standard algorithms for computing Nash and Stackelberg equilibria, including linear programming, incremental strategy generation, and self-play with no-regret learners.