Interacting Contour Stochastic Gradient Langevin Dynamics

TL;DR

This paper introduces ICSGLD, a parallelized sampling method that improves efficiency and adaptability for large-scale Bayesian inference, demonstrating promising results over existing benchmarks.

Contribution

The paper presents ICSGLD, a novel parallel contour stochastic gradient Langevin dynamics method with a new random-field function for adaptive parameter estimation.

Findings

ICSGLD is theoretically more efficient than single-chain CSGLD.

ICSGLD effectively estimates self-adapting parameters in big data.

Numerical results show ICSGLD outperforms benchmark methods in large-scale uncertainty estimation.

Abstract

We propose an interacting contour stochastic gradient Langevin dynamics (ICSGLD) sampler, an embarrassingly parallel multiple-chain contour stochastic gradient Langevin dynamics (CSGLD) sampler with efficient interactions. We show that ICSGLD can be theoretically more efficient than a single-chain CSGLD with an equivalent computational budget. We also present a novel random-field function, which facilitates the estimation of self-adapting parameters in big data and obtains free mode explorations. Empirically, we compare the proposed algorithm with popular benchmark methods for posterior sampling. The numerical results show a great potential of ICSGLD for large-scale uncertainty estimation tasks.