Mean-Field Langevin Diffusions with Density-dependent Temperature

TL;DR

This paper introduces a novel mean-field Langevin diffusion with a density-dependent temperature, enabling adaptive exploration of non-convex landscapes, and proves its well-posedness and convergence properties.

Contribution

It develops a new self-regulating Langevin dynamics model with density-dependent temperature, establishing existence, uniqueness, and convergence to equilibrium.

Findings

Density-dependent temperature improves convergence to global minimizers.

The model's invariant distribution can be explicitly characterized.

Numerical results show enhanced accuracy and convergence rate.

Abstract

In the context of non-convex optimization, we let the temperature of a Langevin diffusion to depend on the diffusion's own density function. The rationale is that the induced density captures to some extent the landscape imposed by the non-convex function to be minimized, such that a density-dependent temperature provides location-wise random perturbation that may better react to, for instance, the location and depth of local minimizers. As the Langevin dynamics is now self-regulated by its own density, it forms a mean-field stochastic differential equation (SDE) of the Nemytskii type, distinct from the standard McKean-Vlasov equations. Relying on Wasserstein subdifferential calculus, we first show that the corresponding (nonlinear) Fokker-Planck equation has a unique solution. Next, a weak solution to the SDE is constructed from the solution to the Fokker-Planck equation, by Trevisan's superposition principle. As time goes to infinity, we further show that the induced density converges to an invariant distribution, which admits an explicit formula in terms of the Lambert $W$ function. A numerical example suggests that the density-dependent temperature can simultaneously improve the accuracy of and rate of convergence to the estimate of global minimizers.