Improving Intelligence of Evolutionary Algorithms Using Experience Share and Replay

TL;DR

This paper introduces PESA, a hybrid optimization algorithm combining PSO, ES, and SA with experience sharing and replay mechanisms, leading to improved exploration and convergence in high-dimensional benchmarks.

Contribution

PESA is the first hybrid algorithm integrating PSO, ES, and SA with experience replay, enhancing exploration and convergence in continuous optimization tasks.

Findings

PESA outperforms standalone ES, PSO, and SA on benchmark functions.

PESA achieves faster convergence and better global optima discovery.

PESA demonstrates improved exploration behavior.

Abstract

We propose PESA, a novel approach combining Particle Swarm Optimisation (PSO), Evolution Strategy (ES), and Simulated Annealing (SA) in a hybrid Algorithm, inspired from reinforcement learning. PESA hybridizes the three algorithms by storing their solutions in a shared replay memory. Next, PESA applies prioritized replay to redistribute data between the three algorithms in frequent form based on their fitness and priority values, which significantly enhances sample diversity and algorithm exploration. Additionally, greedy replay is used implicitly within SA to improve PESA exploitation close to the end of evolution. The validation against 12 high-dimensional continuous benchmark functions shows superior performance by PESA against standalone ES, PSO, and SA, under similar initial starting points, hyperparameters, and number of generations. PESA shows much better exploration behaviour, faster convergence, and ability to find the global optima compared to its standalone counterparts. Given the promising performance, PESA can offer an efficient optimisation option, especially after it goes through additional multiprocessing improvements to handle complex and expensive fitness functions.