When is the Convergence Time of Langevin Algorithms Dimension Independent? A Composite Optimization Viewpoint

TL;DR

This paper demonstrates that Langevin algorithms can achieve dimension-independent convergence rates for certain convex problems by viewing them through the lens of composite optimization, addressing a long-standing open problem.

Contribution

It introduces a novel analysis technique that proves dimension-independent convergence rates for Langevin algorithms in convex optimization with normal priors.

Findings

Langevin algorithms can have dimension-free convergence rates for specific convex problems.

The new analysis technique is based on viewing Langevin as composite optimization.

Results apply to large classes of Lipschitz or smooth convex problems with normal priors.

Abstract

There has been a surge of works bridging MCMC sampling and optimization, with a specific focus on translating non-asymptotic convergence guarantees for optimization problems into the analysis of Langevin algorithms in MCMC sampling. A conspicuous distinction between the convergence analysis of Langevin sampling and that of optimization is that all known convergence rates for Langevin algorithms depend on the dimensionality of the problem, whereas the convergence rates for optimization are dimension-free for convex problems. Whether a dimension independent convergence rate can be achieved by Langevin algorithm is thus a long-standing open problem. This paper provides an affirmative answer to this problem for large classes of either Lipschitz or smooth convex problems with normal priors. By viewing Langevin algorithm as composite optimization, we develop a new analysis technique that leads to dimension independent convergence rates for such problems.