Quality Diversity Through Surprise

TL;DR

This paper explores the use of surprise as a diversity measure in quality diversity algorithms, demonstrating that combining surprise with novelty enhances search efficiency, speed, and robustness in complex robot navigation tasks.

Contribution

Introduces three new quality diversity algorithms utilizing surprise, alone or with novelty, and empirically shows their superior performance over existing methods.

Findings

Surprise-based algorithms outperform traditional methods in maze navigation.

Synergistic use of surprise and novelty improves search robustness.

Surprise enhances the exploration capabilities of quality diversity algorithms.

Abstract

Quality diversity is a recent family of evolutionary search algorithms which focus on finding several well-performing (quality) yet different (diversity) solutions with the aim to maintain an appropriate balance between divergence and convergence during search. While quality diversity has already delivered promising results in complex problems, the capacity of divergent search variants for quality diversity remains largely unexplored. Inspired by the notion of surprise as an effective driver of divergent search and its orthogonal nature to novelty this paper investigates the impact of the former to quality diversity performance. For that purpose we introduce three new quality diversity algorithms which employ surprise as a diversity measure, either on its own or combined with novelty, and compare their performance against novelty search with local competition, the state of the art quality diversity algorithm. The algorithms are tested in a robot navigation task across 60 highly deceptive mazes. Our findings suggest that allowing surprise and novelty to operate synergistically for divergence and in combination with local competition leads to quality diversity algorithms of significantly higher efficiency, speed and robustness.