Bayesian Coresets: Revisiting the Nonconvex Optimization Perspective

TL;DR

This paper introduces a new optimization-based algorithm for Bayesian coreset selection, improving the speed and accuracy of scalable Bayesian inference through sparsity constrained optimization and accelerated methods.

Contribution

It revisits Bayesian coresets from a nonconvex optimization perspective, proposing a novel algorithm with explicit convergence guarantees and empirical validation.

Findings

Superior speed over existing methods

Higher accuracy in posterior approximation

Effective on diverse benchmark datasets

Abstract

Bayesian coresets have emerged as a promising approach for implementing scalable Bayesian inference. The Bayesian coreset problem involves selecting a (weighted) subset of the data samples, such that the posterior inference using the selected subset closely approximates the posterior inference using the full dataset. This manuscript revisits Bayesian coresets through the lens of sparsity constrained optimization. Leveraging recent advances in accelerated optimization methods, we propose and analyze a novel algorithm for coreset selection. We provide explicit convergence rate guarantees and present an empirical evaluation on a variety of benchmark datasets to highlight our proposed algorithm's superior performance compared to state-of-the-art on speed and accuracy.