State Space Emulation and Annealed Sequential Monte Carlo for High Dimensional Optimization

TL;DR

This paper introduces a novel approach combining state space emulation with annealed Sequential Monte Carlo to efficiently solve high-dimensional optimization problems by transforming them into state path estimation tasks.

Contribution

It presents a general emulation strategy for reformulating optimization problems as state space models and develops an annealed SMC method that improves sampling efficiency without burn-in.

Findings

Effective high-dimensional optimization via state space emulation

Annealed SMC outperforms vanilla simulated annealing in convergence

Applications demonstrate the method's versatility and efficiency

Abstract

Many high dimensional optimization problems can be reformulated into a problem of finding theoptimal state path under an equivalent state space model setting. In this article, we present a general emulation strategy for developing a state space model whose likelihood function (or posterior distribution) shares the same general landscape as the objective function of the original optimization problem. Then the solution of the optimization problem is the same as the optimal state path that maximizes the likelihood function or the posterior distribution under the emulated system. To find such an optimal path, we adapt a simulated annealing approach by inserting a temperature control into the emulated dynamic system and propose a novel annealed Sequential Monte Carlo (SMC) method that effectively generating Monte Carlo sample paths utilizing samples obtained on the high temperature scale. Compared to the vanilla simulated annealing implementation, annealed SMC is an iterative algorithm for state space model optimization that directly generates state paths from the equilibrium distributions with a decreasing sequence of temperatures through sequential importance sampling which does not require burn-in or mixing iterations to ensure quasi-equilibrium condition. Several applications of state space model emulation and the corresponding annealed SMC results are demonstrated.