Exploring the Performance-Reproducibility Trade-off in Quality-Diversity

TL;DR

This paper formalizes the performance-reproducibility trade-off in uncertain Quality-Diversity algorithms and introduces new algorithms to optimize solutions based on user-defined preferences, improving over existing methods.

Contribution

It introduces four new a-priori QD algorithms and one a-posteriori algorithm that explicitly handle the performance-reproducibility trade-off in uncertain environments.

Findings

Our methods effectively find solutions matching specified trade-off preferences.

Considering the trade-off improves the quality of solutions over existing methods.

The approaches are applicable in complex real-world uncertain scenarios.

Abstract

Quality-Diversity (QD) algorithms have exhibited promising results across many domains and applications. However, uncertainty in fitness and behaviour estimations of solutions remains a major challenge when QD is used in complex real-world applications. While several approaches have been proposed to improve the performance in uncertain applications, many fail to address a key challenge: determining how to prioritise solutions that perform consistently under uncertainty, in other words, solutions that are reproducible. Most prior methods improve fitness and reproducibility jointly, ignoring the possibility that they could be contradictory objectives. For example, in robotics, solutions may reliably walk at 90% of the maximum velocity in uncertain environments, while solutions that walk faster are also more prone to falling over. As this is a trade-off, neither one of these two solutions is "better" than the other. Thus, algorithms cannot intrinsically select one solution over the other, but can only enforce given preferences over these two contradictory objectives. In this paper, we formalise this problem as the performance-reproducibility trade-off for uncertain QD. We propose four new a-priori QD algorithms that find optimal solutions for given preferences over the trade-offs. We also propose an a-posteriori QD algorithm for when these preferences cannot be defined in advance. Our results show that our approaches successfully find solutions that satisfy given preferences. Importantly, by simply accounting for this trade-off, our approaches perform better than existing uncertain QD methods. This suggests that considering the performance-reproducibility trade-off unlocks important stepping stones that are usually missed when only performance is optimised.