Evolutionary Warm-Starts for Reinforcement Learning in Industrial Continuous Control

TL;DR

This paper explores how evolution strategies, specifically CMA-ES, can enhance reinforcement learning in industrial continuous control by providing warm-start demonstrations that improve stability and performance.

Contribution

It introduces a hybrid evolutionary-RL approach using CMA-ES for demonstration generation to support RL in industrial control tasks.

Findings

CMA-ES-guided initialization improves RL stability and performance.

Demonstration trajectories serve as a strong oracle reference.

Hybrid approach shows promise for complex industrial applications.

Abstract

Reinforcement learning (RL) is still rarely applied in industrial control, partly due to the difficulty of training reliable agents for real-world conditions. This work investigates how evolution strategies can support RL in such settings by introducing a continuous-control adaptation of an industrial sorting benchmark. The CMA-ES algorithm is used to generate high-quality demonstrations that warm-start RL agents. Results show that CMA-ES-guided initialization significantly improves stability and performance. Furthermore, the demonstration trajectories generated with the CMA-ES provide a strong oracle reference performance level, which is of interest in its own right. The study delivers a focused proof of concept for hybrid evolutionary-RL approaches and a basis for future, more complex industrial applications.