On the use of feature-maps and parameter control for improved quality-diversity meta-evolution

TL;DR

This paper enhances quality-diversity meta-evolution by introducing feature-maps and dynamic parameter control, significantly improving solution quality and robustness in robotic arm tasks.

Contribution

It proposes a novel meta-evolution system with feature-maps and adaptive parameter control, improving archive quality and robot resilience compared to traditional methods.

Findings

Non-linear and feature-selection feature-maps improve meta-fitness 15-fold and 3-fold.

Reinforcement learning is among the top parameter control strategies.

The approach enables the robot arm to recover over 80% reachability after damages.

Abstract

In Quality-Diversity (QD) algorithms, which evolve a behaviourally diverse archive of high-performing solutions, the behaviour space is a difficult design choice that should be tailored to the target application. In QD meta-evolution, one evolves a population of QD algorithms to optimise the behaviour space based on an archive-level objective, the meta-fitness. This paper proposes an improved meta-evolution system such that (i) the database used to rapidly populate new archives is reformulated to prevent loss of quality-diversity; (ii) the linear transformation of base-features is generalised to a feature-map, a function of the base-features parametrised by the meta-genotype; and (iii) the mutation rate of the QD algorithm and the number of generations per meta-generation are controlled dynamically. Experiments on an 8-joint planar robot arm compare feature-maps (linear, non-linear, and feature-selection), parameter control strategies (static, endogenous, reinforcement learning, and annealing), and traditional MAP-Elites variants, for a total of 49 experimental conditions. Results reveal that non-linear and feature-selection feature-maps yield a 15-fold and 3-fold improvement in meta-fitness, respectively, over linear feature-maps. Reinforcement learning ranks among top parameter control methods. Finally, our approach allows the robot arm to recover a reach of over 80% for most damages and at least 60% for severe damages.