Probabilistic Gaussian Homotopy: A Probability-Space Continuation Framework for Nonconvex Optimization

TL;DR

This paper introduces Probabilistic Gaussian Homotopy (PGH), a novel continuation framework that biases optimization towards low-energy regions using a probabilistic approach, improving performance on challenging nonconvex problems.

Contribution

The paper presents PGH, a new probabilistic homotopy method that deforms the Boltzmann distribution to better navigate nonconvex landscapes, along with a practical stochastic algorithm PGHO.

Findings

PGH effectively biases descent towards low-energy regions.

PGHO outperforms classical methods on high-dimensional benchmarks.

The framework links Gaussian continuation, Bayesian denoising, and diffusion smoothing.

Abstract

We introduce Probabilistic Gaussian Homotopy (PGH), a probability-space continuation framework for nonconvex optimization. Unlike classical Gaussian homotopy, which smooths the objective and uniformly averages gradients, PGH deforms the associated Boltzmann distribution and induces Boltzmann-weighted aggregation of perturbed gradients, which exponentially biases descent directions toward low-energy regions. We show that PGH corresponds to a log-sum-exp (soft-min) homotopy that smooths a nonconvex objective at scale $\lambda>0$ and recovers the original objective as $\lambda\to 0$, yielding a posterior-mean generalization of the Moreau envelope, and we derive a dynamical system governing minimizer evolution along an annealed homotopy path. This establishes a principled connection between Gaussian continuation, Bayesian denoising, and diffusion-style smoothing. We further propose Probabilistic Gaussian Homotopy Optimization (PGHO), a practical stochastic algorithm based on Monte Carlo gradient estimation, and demonstrate strong performance on high-dimensional nonconvex benchmarks and sparse recovery problems where classical gradient methods and objective-space smoothing frequently fail.