Multi-Objective Covariance Matrix Adaptation MAP-Annealing

TL;DR

This paper introduces MO-CMA-MAE, a novel multi-objective quality-diversity algorithm that uses covariance matrix adaptation to improve solution diversity and quality, demonstrating significant performance gains in game map generation.

Contribution

The paper presents MO-CMA-MAE, integrating CMA-ES into MOQD to adaptively optimize the Pareto archive, advancing the state-of-the-art in multi-objective quality-diversity optimization.

Findings

Significant performance improvements in three MOQD domains.

Enhanced map diversity and quality in game map generation.

Effective use of covariance adaptation for multi-objective optimization.

Abstract

Quality-Diversity (QD) optimization is an emerging field that focuses on finding a set of behaviorally diverse and high-quality solutions. While the quality is typically defined w.r.t. a single objective function, recent work on Multi-Objective Quality-Diversity (MOQD) extends QD optimization to simultaneously optimize multiple objective functions. This opens up multi-objective applications for QD, such as generating a diverse set of game maps that maximize difficulty, realism, or other properties. Existing MOQD algorithms use non-adaptive methods such as mutation and crossover to search for non-dominated solutions and construct an archive of Pareto Sets (PS). However, recent work in QD has demonstrated enhanced performance through the use of covariance-based evolution strategies for adaptive solution search. We propose bringing this insight into the MOQD problem, and introduce MO-CMA-MAE, a new MOQD algorithm that leverages Covariance Matrix Adaptation-Evolution Strategies (CMA-ES) to optimize the hypervolume associated with every PS within the archive. We test MO-CMA-MAE on three MOQD domains, and for generating maps of a co-operative video game, showing significant improvements in performance.