Adaptive Scan Gibbs Sampler for Large Scale Inference Problems

TL;DR

This paper introduces an adaptive scan Gibbs sampler that dynamically optimizes update frequency and mini-batch size, significantly improving large-scale online inference performance across various Bayesian models.

Contribution

It proposes a novel adaptive batch-size Gibbs sampling method that enhances efficiency in large-scale inference tasks, outperforming traditional methods.

Findings

Outperforms collapsed Gibbs sampler in Bayesian Lasso.

Demonstrates improved convergence in DPMM and LDA models.

Adaptive sampling reduces computational cost for large datasets.

Abstract

For large scale on-line inference problems the update strategy is critical for performance. We derive an adaptive scan Gibbs sampler that optimizes the update frequency by selecting an optimum mini-batch size. We demonstrate performance of our adaptive batch-size Gibbs sampler by comparing it against the collapsed Gibbs sampler for Bayesian Lasso, Dirichlet Process Mixture Models (DPMM) and Latent Dirichlet Allocation (LDA) graphical models.