Towards QD-suite: developing a set of benchmarks for Quality-Diversity algorithms

TL;DR

This paper advocates for developing standardized, challenging, and scalable benchmarks for Quality-Diversity algorithms, addressing current limitations and identifying key challenges through three proposed benchmark problems.

Contribution

It introduces three novel benchmark problems targeting key challenges in QD algorithms, inspired by the need for standardized and representative evaluation tools.

Findings

Identified behavior metric bias as a challenge in QD.

Proposed benchmarks for behavioral plateaus and evolvability traps.

Environments satisfy criteria for challenge, scalability, and affordability.

Abstract

While the field of Quality-Diversity (QD) has grown into a distinct branch of stochastic optimization, a few problems, in particular locomotion and navigation tasks, have become de facto standards. Are such benchmarks sufficient? Are they representative of the key challenges faced by QD algorithms? Do they provide the ability to focus on one particular challenge by properly disentangling it from others? Do they have much predictive power in terms of scalability and generalization? Existing benchmarks are not standardized, and there is currently no MNIST equivalent for QD. Inspired by recent works on Reinforcement Learning benchmarks, we argue that the identification of challenges faced by QD methods and the development of targeted, challenging, scalable but affordable benchmarks is an important step. As an initial effort, we identify three problems that are challenging in sparse reward settings, and propose associated benchmarks: (1) Behavior metric bias, which can result from the use of metrics that do not match the structure of the behavior space. (2) Behavioral Plateaus, with varying characteristics, such that escaping them would require adaptive QD algorithms and (3) Evolvability Traps, where small variations in genotype result in large behavioral changes. The environments that we propose satisfy the properties listed above.