Path Integral Optimiser: Global Optimisation via Neural Schr\"odinger-F\"ollmer Diffusion

TL;DR

This paper introduces a neural diffusion-based optimizer that formulates global optimization as a Schr"odinger bridge problem, demonstrating promising results on moderate dimensions but facing challenges in high-dimensional spaces.

Contribution

It proposes a novel neural diffusion approach for global optimization based on Schr"odinger bridge sampling, integrating stochastic control and neural approximation techniques.

Findings

Promising per-step performance on 2 to 1247-dimensional tasks.

Struggles with high-dimensional spaces like 15.9k parameters.

Provides theoretical bounds and stochastic analysis for the optimizer.

Abstract

We present an early investigation into the use of neural diffusion processes for global optimisation, focusing on Zhang et al.'s Path Integral Sampler. One can use the Boltzmann distribution to formulate optimization as solving a Schr\"odinger bridge sampling problem, then apply Girsanov's theorem with a simple (single-point) prior to frame it in stochastic control terms, and compute the solution's integral terms via a neural approximation (a Fourier MLP). We provide theoretical bounds for this optimiser, results on toy optimisation tasks, and a summary of the stochastic theory motivating the model. Ultimately, we found the optimiser to display promising per-step performance at optimisation tasks between 2 and 1,247 dimensions, but struggle to explore higher-dimensional spaces when faced with a 15.9k parameter model, indicating a need for work on adaptation in such environments.