Multi-Objective Algorithms for Learning Open-Ended Robotic Problems

TL;DR

This paper introduces Multi-Objective Learning (MOL), a novel evolutionary algorithm-based framework that improves quadrupedal robot training by optimizing for performance and diversity, leading to more stable and adaptable locomotion.

Contribution

It presents a new multi-objective evolutionary approach for open-ended robotic learning, outperforming traditional reinforcement learning methods in stability and adaptability.

Findings

Achieved 19% fewer errors in difficult scenarios.

Demonstrated superior stability and adaptability over baseline methods.

Enhanced training efficiency in the MuJoCo simulator.

Abstract

Quadrupedal locomotion is a complex, open-ended problem vital to expanding autonomous vehicle reach. Traditional reinforcement learning approaches often fall short due to training instability and sample inefficiency. We propose a novel method leveraging multi-objective evolutionary algorithms as an automatic curriculum learning mechanism, which we named Multi-Objective Learning (MOL). Our approach significantly enhances the learning process by projecting velocity commands into an objective space and optimizing for both performance and diversity. Tested within the MuJoCo physics simulator, our method demonstrates superior stability and adaptability compared to baseline approaches. As such, it achieved 19\% and 44\% fewer errors against our best baseline algorithm in difficult scenarios based on a uniform and tailored evaluation respectively. This work introduces a robust framework for training quadrupedal robots, promising significant advancements in robotic locomotion and open-ended robotic problems.