Black-box optimization with a politician

TL;DR

This paper introduces a novel black-box convex optimization framework that efficiently handles expensive gradient computations, combining classical optimization concepts with empirical validation against leading algorithms.

Contribution

It develops a new optimization method integrating convex optimization principles, suitable for scenarios with costly gradient evaluations, and demonstrates its effectiveness empirically.

Findings

Outperforms state-of-the-art algorithms like BFGS in experiments

Provides a new framework combining first-order methods and analytical centers

Addresses optimization problems with expensive gradient computations

Abstract

We propose a new framework for black-box convex optimization which is well-suited for situations where gradient computations are expensive. We derive a new method for this framework which leverages several concepts from convex optimization, from standard first-order methods (e.g. gradient descent or quasi-Newton methods) to analytical centers (i.e. minimizers of self-concordant barriers). We demonstrate empirically that our new technique compares favorably with state of the art algorithms (such as BFGS).