A Particle Algorithm for Mean-Field Variational Inference

TL;DR

This paper introduces PAVI, a novel particle-based algorithm for nonparametric mean-field variational inference, providing the first end-to-end error guarantees for such methods, enhancing scalability and flexibility in posterior inference.

Contribution

The paper presents PAVI, a new particle algorithm for nonparametric MFVI with non-asymptotic error bounds and end-to-end theoretical guarantees.

Findings

First end-to-end guarantee for particle-based MFVI.

Non-asymptotic error bounds established.

Applicable to scalable posterior inference.

Abstract

Variational inference is a fast and scalable alternative to Markov chain Monte Carlo and has been widely applied to posterior inference tasks in statistics and machine learning. A traditional approach for implementing mean-field variational inference (MFVI) is coordinate ascent variational inference (CAVI), which relies crucially on parametric assumptions on complete conditionals. We introduce a novel particle-based algorithm for MFVI, named PArticle VI (PAVI), for nonparametric mean-field approximation. We obtain non-asymptotic error bounds for our algorithm. To our knowledge, this is the first end-to-end guarantee for particle-based MFVI.