Generating Novelty in Open-World Multi-Agent Strategic Board Games

TL;DR

GNOME is an experimental platform designed to evaluate multi-agent AI systems' ability to handle unanticipated novelty in open-world strategic board games, exemplified through Monopoly, fostering research on AI robustness.

Contribution

The paper introduces GNOME, a novel platform that separates AI development from simulation, enabling testing of unanticipated novelty in multi-agent environments.

Findings

Demonstrated GNOME at NeurIPS 2020 using Monopoly.

Outlined experimental design for DARPA SAIL-ON program.

Facilitated open discussion on AI robustness and novelty.

Abstract

We describe GNOME (Generating Novelty in Open-world Multi-agent Environments), an experimental platform that is designed to test the effectiveness of multi-agent AI systems when faced with \emph{novelty}. GNOME separates the development of AI gameplaying agents with the simulator, allowing \emph{unanticipated} novelty (in essence, novelty that is not subject to model-selection bias). Using a Web GUI, GNOME was recently demonstrated at NeurIPS 2020 using the game of Monopoly to foster an open discussion on AI robustness and the nature of novelty in real-world environments. In this article, we further detail the key elements of the demonstration, and also provide an overview of the experimental design that is being currently used in the DARPA Science of Artificial Intelligence and Learning for Open-World Novelty (SAIL-ON) program to evaluate external teams developing novelty-adaptive gameplaying agents.