Quasi-Bayes properties of a recursive procedure for mixtures

TL;DR

This paper demonstrates that a fast recursive algorithm for online nonparametric mixture models can be interpreted as a quasi-Bayesian method, providing asymptotic posterior distributions and credible intervals, thus bridging computational efficiency and Bayesian inference.

Contribution

It develops a methodology to interpret a recursive mixture model algorithm as a quasi-Bayesian solution, offering asymptotic posterior insights and credible intervals.

Findings

The recursive algorithm asymptotically approximates a Bayesian exchangeable mixture model.

It provides asymptotic credible intervals for the mixing distribution.

Simulation studies illustrate parameter tuning and potential extensions.

Abstract

Bayesian methods are often optimal, yet increasing pressure for fast computations, especially with streaming data, brings renewed interest in faster, possibly sub-optimal, solutions. The extent to which these algorithms approximate Bayesian solutions is a question of interest, but often unanswered. We propose a methodology to address this question in predictive settings, when the algorithm can be reinterpreted as a probabilistic predictive rule. We specifically develop the proposed methodology for a recursive procedure for online learning in nonparametric mixture models, often refereed to as Newton's algorithm. This algorithm is simple and fast; however, its approximation properties are unclear. By reinterpreting it as a predictive rule, we can show that it underlies a statistical model which is, asymptotically, a Bayesian, exchangeable mixture model. In this sense, the recursive rule provides a quasi-Bayes solution. While the algorithm only offers a point estimate, our clean statistical formulation allows us to provide the asymptotic posterior distribution and asymptotic credible intervals for the mixing distribution. Moreover, it gives insights for tuning the parameters, as we illustrate in simulation studies, and paves the way to extensions in various directions. Beyond mixture models, our approach can be applied to other predictive algorithms.