Global Optimization Networks

TL;DR

This paper introduces global optimization networks (GONs), a novel approach for efficiently estimating global maximizers of black-box functions from noisy data, outperforming existing methods.

Contribution

It proposes a new class of functions called GONs, constructed via invertible and unimodal functions, enabling efficient global maximization inference and extending to conditional GONs.

Findings

GON maximizers outperform convex fits, GPR, and DNNs in predictions.

GONs provide more reasonable predictions for real-world problems.

Global maximizers can be inferred in O(D) time.

Abstract

We consider the problem of estimating a good maximizer of a black-box function given noisy examples. To solve such problems, we propose to fit a new type of function which we call a global optimization network (GON), defined as any composition of an invertible function and a unimodal function, whose unique global maximizer can be inferred in $\mathcal{O}(D)$ time. In this paper, we show how to construct invertible and unimodal functions by using linear inequality constraints on lattice models. We also extend to \emph{conditional} GONs that find a global maximizer conditioned on specified inputs of other dimensions. Experiments show the GON maximizers are statistically significantly better predictions than those produced by convex fits, GPR, or DNNs, and are more reasonable predictions for real-world problems.