MetaBox-v2: A Unified Benchmark Platform for Meta-Black-Box Optimization

TL;DR

MetaBox-v2 is a comprehensive, flexible, and efficient benchmarking platform for Meta-Black-Box Optimization, supporting multiple algorithms, extensive tasks, and providing valuable insights for researchers and practitioners.

Contribution

It introduces MetaBox-v2 with a unified architecture, faster training/testing, a broad benchmark suite, and extensible interfaces, advancing the state-of-the-art in MetaBBO benchmarking.

Findings

Reproduced 23 up-to-date baselines across different approaches.

Reduced training/testing time by 10-40 times with parallelization.

Provided detailed analysis of baseline performance and generalization.

Abstract

Meta-Black-Box Optimization (MetaBBO) streamlines the automation of optimization algorithm design through meta-learning. It typically employs a bi-level structure: the meta-level policy undergoes meta-training to reduce the manual effort required in developing algorithms for low-level optimization tasks. The original MetaBox (2023) provided the first open-source framework for reinforcement learning-based single-objective MetaBBO. However, its relatively narrow scope no longer keep pace with the swift advancement in this field. In this paper, we introduce MetaBox-v2 (https://github.com/MetaEvo/MetaBox) as a milestone upgrade with four novel features: 1) a unified architecture supporting RL, evolutionary, and gradient-based approaches, by which we reproduce $23$ up-to-date baselines; 2) efficient parallelization schemes, which reduce the training/testing time by $10-40$x; 3) a comprehensive benchmark suite of $18$ synthetic/realistic tasks ($1900$+ instances) spanning single-objective, multi-objective, multi-model, and multi-task optimization scenarios; 4) plentiful and extensible interfaces for custom analysis/visualization and integrating to external optimization tools/benchmarks. To show the utility of MetaBox-v2, we carry out a systematic case study that evaluates the built-in baselines in terms of the optimization performance, generalization ability and learning efficiency. Valuable insights are concluded from thorough and detailed analysis for practitioners and those new to the field.