MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning

TL;DR

MetaBox is a comprehensive, open-source benchmarking platform designed for evaluating Meta-Black-Box Optimization with Reinforcement Learning methods across diverse problem instances and baseline algorithms.

Contribution

It introduces the first unified benchmark platform for MetaBBO-RL, including a flexible template, extensive problem set, baseline library, and standardized metrics.

Findings

Benchmarking study on existing MetaBBO-RL methods

Demonstrates MetaBox's utility for evaluation and analysis

Provides insights into method performance across scenarios

Abstract

Recently, Meta-Black-Box Optimization with Reinforcement Learning (MetaBBO-RL) has showcased the power of leveraging RL at the meta-level to mitigate manual fine-tuning of low-level black-box optimizers. However, this field is hindered by the lack of a unified benchmark. To fill this gap, we introduce MetaBox, the first benchmark platform expressly tailored for developing and evaluating MetaBBO-RL methods. MetaBox offers a flexible algorithmic template that allows users to effortlessly implement their unique designs within the platform. Moreover, it provides a broad spectrum of over 300 problem instances, collected from synthetic to realistic scenarios, and an extensive library of 19 baseline methods, including both traditional black-box optimizers and recent MetaBBO-RL methods. Besides, MetaBox introduces three standardized performance metrics, enabling a more thorough assessment of the methods. In a bid to illustrate the utility of MetaBox for facilitating rigorous evaluation and in-depth analysis, we carry out a wide-ranging benchmarking study on existing MetaBBO-RL methods. Our MetaBox is open-source and accessible at: https://github.com/GMC-DRL/MetaBox.