AMaze: An intuitive benchmark generator for fast prototyping of generalizable agents

TL;DR

AMaze is a new, human-interactive benchmark generator for training and evaluating generalizable agents in maze navigation tasks with visual signs, promoting rapid prototyping and improved generalization over static environments.

Contribution

The paper introduces AMaze, a flexible benchmark generator enabling easy maze creation and human interaction, enhancing agent generalization and rapid prototyping capabilities.

Findings

Interactive training outperforms direct training in generalization.

Median performance gains ranged from 50% to 100%.

Maximal performance achieved with human-in-the-loop training.

Abstract

Traditional approaches to training agents have generally involved a single, deterministic environment of minimal complexity to solve various tasks such as robot locomotion or computer vision. However, agents trained in static environments lack generalization capabilities, limiting their potential in broader scenarios. Thus, recent benchmarks frequently rely on multiple environments, for instance, by providing stochastic noise, simple permutations, or altogether different settings. In practice, such collections result mainly from costly human-designed processes or the liberal use of random number generators. In this work, we introduce AMaze, a novel benchmark generator in which embodied agents must navigate a maze by interpreting visual signs of arbitrary complexities and deceptiveness. This generator promotes human interaction through the easy generation of feature-specific mazes and an intuitive understanding of the resulting agents' strategies. As a proof-of-concept, we demonstrate the capabilities of the generator in a simple, fully discrete case with limited deceptiveness. Agents were trained under three different regimes (one-shot, scaffolding, interactive), and the results showed that the latter two cases outperform direct training in terms of generalization capabilities. Indeed, depending on the combination of generalization metric, training regime, and algorithm, the median gain ranged from 50% to 100% and maximal performance was achieved through interactive training, thereby demonstrating the benefits of a controllable human-in-the-loop benchmark generator.