Gradient-free optimization via integration

TL;DR

This paper introduces a gradient-free optimization method that fits functions to distributions using Bayesian updates and Monte Carlo, enabling convergence analysis and demonstrating effectiveness on complex machine learning tasks.

Contribution

The paper presents a novel model-based search algorithm that optimizes non-convex, non-differentiable functions through distribution fitting and implicit gradient descent, with convergence guarantees.

Findings

Algorithm effectively optimizes complex functions.

Convergence analysis for inhomogeneous gradient descent.

Successful application to a challenging classification task.

Abstract

We develop and analyse an approach to optimize functions $l\colon \mathbb{R}^d \rightarrow \mathbb{R}$ not assumed to be convex, differentiable or even continuous. The algorithm belongs to the class of model-based search methods. The idea is to fit recursively $l$ to a parametric family of distributions, using a Bayesian update followed by a reprojection back onto the chosen family. Remarkably, reprojection in our scenario boils down to computing expectations, which can be simply approximated through Monte Carlo. We show that when the family of distributions is appropriately chosen this approach can be interpreted as an implicit time-inhomogeneous gradient descent algorithm on a sequence of smoothed approximations of $l$, providing a route to establishing convergence. We establish new results for generic inhomogeneous gradient descent algorithms, which we specialise to the model-based search algorithm in the Gaussian scenario. We illustrate the performance of the algorithm on a challenging classification task in machine learning.