Reinforcement learning Based Automated Design of Differential Evolution Algorithm for Black-box Optimization

TL;DR

This paper presents a reinforcement learning framework that automatically designs customized differential evolution algorithms for black-box optimization, improving adaptability and performance across diverse problems.

Contribution

It introduces a novel RL-based meta-learning approach to automatically generate tailored DE configurations for specific black-box problems.

Findings

Framework outperforms standard DE variants on benchmarks

RL effectively learns problem-specific DE strategies

Demonstrates potential for adaptive algorithm design

Abstract

Differential evolution (DE) algorithm is recognized as one of the most effective evolutionary algorithms, demonstrating remarkable efficacy in black-box optimization due to its derivative-free nature. Numerous enhancements to the fundamental DE have been proposed, incorporating innovative mutation strategies and sophisticated parameter tuning techniques to improve performance. However, no single variant has proven universally superior across all problems. To address this challenge, we introduce a novel framework that employs reinforcement learning (RL) to automatically design DE for black-box optimization through meta-learning. RL acts as an advanced meta-optimizer, generating a customized DE configuration that includes an optimal initialization strategy, update rule, and hyperparameters tailored to a specific black-box optimization problem. This process is informed by a detailed analysis of the problem characteristics. In this proof-of-concept study, we utilize a double deep Q-network for implementation, considering a subset of 40 possible strategy combinations and parameter optimizations simultaneously. The framework's performance is evaluated against black-box optimization benchmarks and compared with state-of-the-art algorithms. The experimental results highlight the promising potential of our proposed framework.