Meta Learning Black-Box Population-Based Optimizers

TL;DR

This paper introduces a meta-learning framework for designing population-based black-box optimizers that adapt to specific problem classes, improving performance and sample efficiency over traditional methods.

Contribution

It proposes a novel meta-learning approach using a POMDP formulation and deep recurrent networks to create adaptable, data-driven optimizers for black-box problems.

Findings

Learned optimizers outperform CMA-ES in various tasks

Meta-loss encourages adaptable search behavior

Optimizers generalize well to new problem contexts

Abstract

The no free lunch theorem states that no model is better suited to every problem. A question that arises from this is how to design methods that propose optimizers tailored to specific problems achieving state-of-the-art performance. This paper addresses this issue by proposing the use of meta-learning to infer population-based black-box optimizers that can automatically adapt to specific classes of problems. We suggest a general modeling of population-based algorithms that result in Learning-to-Optimize POMDP (LTO-POMDP), a meta-learning framework based on a specific partially observable Markov decision process (POMDP). From that framework's formulation, we propose to parameterize the algorithm using deep recurrent neural networks and use a meta-loss function based on stochastic algorithms' performance to train efficient data-driven optimizers over several related optimization tasks. The learned optimizers' performance based on this implementation is assessed on various black-box optimization tasks and hyperparameter tuning of machine learning models. Our results revealed that the meta-loss function encourages a learned algorithm to alter its search behavior so that it can easily fit into a new context. Thus, it allows better generalization and higher sample efficiency than state-of-the-art generic optimization algorithms, such as the Covariance matrix adaptation evolution strategy (CMA-ES).