Non-local Optimization: Imposing Structure on Optimization Problems by Relaxation

TL;DR

This paper explores the structure of relaxations in stochastic optimization using measure theory and Fourier analysis, revealing properties like Lipschitzness and convexity that facilitate efficient optimization, especially when the original function is non-differentiable or non-convex.

Contribution

It provides a theoretical framework for understanding the structure of relaxations in stochastic optimization, highlighting conditions for effective optimization.

Findings

Optimal relaxation values are consistent with the original function.

Gradients of relaxations are Lipschitz continuous.

Convexity properties enable reliable optimization even with non-differentiable functions.

Abstract

In stochastic optimization, particularly in evolutionary computation and reinforcement learning, the optimization of a function $f: \Omega \to \mathbb{R}$ is often addressed through optimizing a so-called relaxation $\theta \in \Theta \mapsto \mathbb{E}_\theta(f)$ of $f$, where $\Theta$ resembles the parameters of a family of probability measures on $\Omega$. We investigate the structure of such relaxations by means of measure theory and Fourier analysis, enabling us to shed light on the success of many associated stochastic optimization methods. The main structural traits we derive and that allow fast and reliable optimization of relaxations are the consistency of optimal values of $f$, Lipschitzness of gradients, and convexity. We emphasize settings where $f$ itself is not differentiable or convex, e.g., in the presence of (stochastic) disturbance.