Extract-QD Framework: A Generic Approach for Quality-Diversity in Noisy, Stochastic or Uncertain Domains

TL;DR

The paper introduces the Extract-QD Framework, a modular approach that unifies and enhances quality-diversity algorithms for noisy and uncertain environments, providing a reliable new method and a tool for developing tailored solutions.

Contribution

It presents the EQD Framework for unifying and developing UQD methods, and introduces EME, a new method that outperforms existing approaches in uncertain domains.

Findings

EME consistently outperforms or matches existing methods on benchmarks.

The EQD Framework enables augmentation of existing QD algorithms with improved performance.

Using the framework, new task-specific approaches can be developed efficiently.

Abstract

Quality-Diversity (QD) has demonstrated potential in discovering collections of diverse solutions to optimisation problems. Originally designed for deterministic environments, QD has been extended to noisy, stochastic, or uncertain domains through various Uncertain-QD (UQD) methods. However, the large number of UQD methods, each with unique constraints, makes selecting the most suitable one challenging. To remedy this situation, we present two contributions: first, the Extract-QD Framework (EQD Framework), and second, Extract-ME (EME), a new method derived from it. The EQD Framework unifies existing approaches within a modular view, and facilitates developing novel methods by interchanging modules. We use it to derive EME, a novel method that consistently outperforms or matches the best existing methods on standard benchmarks, while previous methods show varying performance. In a second experiment, we show how our EQD Framework can be used to augment existing QD algorithms and in particular the well-established Policy-Gradient-Assisted-MAP-Elites method, and demonstrate improved performance in uncertain domains at no additional evaluation cost. For any new uncertain task, our contributions now provide EME as a reliable "first guess" method, and the EQD Framework as a tool for developing task-specific approaches. Together, these contributions aim to lower the cost of adopting UQD insights in QD applications.