Discount Model Search for Quality Diversity Optimization in High-Dimensional Measure Spaces

TL;DR

This paper introduces Discount Model Search (DMS), a novel approach for quality diversity optimization in high-dimensional measure spaces, overcoming limitations of existing algorithms like CMA-MAE.

Contribution

DMS uses a model-based approach to guide exploration in high-dimensional measure spaces, enabling effective diversity search where previous methods stagnate.

Findings

DMS outperforms CMA-MAE and other black-box QD algorithms in high-dimensional benchmarks.

DMS enables measure specification via datasets of images, bypassing manual measure design.

DMS achieves better exploration in high-dimensional measure spaces, including image domains.

Abstract

Quality diversity (QD) optimization searches for a collection of solutions that optimize an objective while attaining diverse outputs of a user-specified, vector-valued measure function. Contemporary QD algorithms are typically limited to low-dimensional measures because high-dimensional measures are prone to distortion, where many solutions found by the QD algorithm map to similar measures. For example, the state-of-the-art CMA-MAE algorithm guides measure space exploration with a histogram in measure space that records so-called discount values. However, CMA-MAE stagnates in domains with high-dimensional measure spaces because solutions with similar measures fall into the same histogram cell and hence receive the same discount value. To address these limitations, we propose Discount Model Search (DMS), which guides exploration with a model that provides a smooth, continuous representation of discount values. In high-dimensional measure spaces, this model enables DMS to distinguish between solutions with similar measures and thus continue exploration. We show that DMS facilitates new capabilities for QD algorithms by introducing two new domains where the measure space is the high-dimensional space of images, which enables users to specify their desired measures by providing a dataset of images rather than hand-designing the measure function. Results in these domains and on high-dimensional benchmarks show that DMS outperforms CMA-MAE and other existing black-box QD algorithms.