Polynomial-Model-Based Optimization for Blackbox Objectives

TL;DR

This paper introduces Polynomial-Model-Based Optimization (PMBO), a novel black-box optimization method that uses polynomial surrogates and an acquisition function to efficiently find minima in complex, unknown systems.

Contribution

PMBO is a new black-box optimizer that fits polynomial surrogates and iteratively updates them using Expected Improvement, outperforming existing algorithms in benchmark tests.

Findings

PMBO successfully competes with state-of-the-art algorithms.

In some cases, PMBO outperforms all compared algorithms.

PMBO is effective for diverse black-box optimization tasks.

Abstract

For a wide range of applications the structure of systems like Neural Networks or complex simulations, is unknown and approximation is costly or even impossible. Black-box optimization seeks to find optimal (hyper-) parameters for these systems such that a pre-defined objective function is minimized. Polynomial-Model-Based Optimization (PMBO) is a novel blackbox optimizer that finds the minimum by fitting a polynomial surrogate to the objective function. Motivated by Bayesian optimization the model is iteratively updated according to the acquisition function Expected Improvement, thus balancing the exploitation and exploration rate and providing an uncertainty estimate of the model. PMBO is benchmarked against other state-of-the-art algorithms for a given set of artificial, analytical functions. PMBO competes successfully with those algorithms and even outperforms all of them in some cases. As the results suggest, we believe PMBO is the pivotal choice for solving blackbox optimization tasks occurring in a wide range of disciplines.