Geometric Learning in Black-Box Optimization: A GNN Framework for Algorithm Performance Prediction

TL;DR

This paper introduces a graph neural network framework that models complex relationships in black-box optimization, improving algorithm performance prediction by capturing problem characteristics and algorithm configurations.

Contribution

It proposes a novel GNN-based approach using heterogeneous graph structures to better predict optimization algorithm performance, addressing limitations of tabular methods.

Findings

Up to 36.6% improvement in MSE over traditional methods

Effective modeling of complex dependencies between problems and algorithms

Demonstrated on multiple algorithm variants and benchmark problems

Abstract

Automated algorithm performance prediction in numerical blackbox optimization often relies on problem characterizations, such as exploratory landscape analysis features. These features are typically used as inputs to machine learning models and are represented in a tabular format. However, such approaches often overlook algorithm configurations, a key factor influencing performance. The relationships between algorithm operators, parameters, problem characteristics, and performance outcomes form a complex structure best represented as a graph. This work explores the use of heterogeneous graph data structures and graph neural networks to predict the performance of optimization algorithms by capturing the complex dependencies between problems, algorithm configurations, and performance outcomes. We focus on two modular frameworks, modCMA-ES and modDE, which decompose two widely used derivative-free optimization algorithms: the covariance matrix adaptation evolution strategy (CMA-ES) and differential evolution (DE). We evaluate 324 modCMA-ES and 576 modDE variants on 24 BBOB problems across six runtime budgets and two problem dimensions. Achieving up to 36.6% improvement in MSE over traditional tabular-based methods, this work highlights the potential of geometric learning in black-box optimization.