Efficient Stochastic Optimisation via Sequential Monte Carlo

TL;DR

This paper introduces a novel optimization method using sequential Monte Carlo samplers to efficiently handle functions with intractable gradients, reducing computational costs in machine learning tasks.

Contribution

It presents a new approach that replaces costly sampling in stochastic optimization with efficient SMC approximations, with proven convergence guarantees.

Findings

Significant computational savings demonstrated in energy-based model tuning.

Effective optimization achieved without inner sampling loops.

Convergence results established for the proposed SMC-based recursions.

Abstract

The problem of optimising functions with intractable gradients frequently arise in machine learning and statistics, ranging from maximum marginal likelihood estimation procedures to fine-tuning of generative models. Stochastic approximation methods for this class of problems typically require inner sampling loops to obtain (biased) stochastic gradient estimates, which rapidly becomes computationally expensive. In this work, we develop sequential Monte Carlo (SMC) samplers for optimisation of functions with intractable gradients. Our approach replaces expensive inner sampling methods with efficient SMC approximations, which can result in significant computational gains. We establish convergence results for the basic recursions defined by our methodology which SMC samplers approximate. We demonstrate the effectiveness of our approach on the reward-tuning of energy-based models within various settings.