Amortized Bayesian Mixture Models

TL;DR

This paper introduces a novel amortized Bayesian inference framework for mixture models that leverages neural networks to efficiently estimate model parameters and mixture components without explicit likelihoods.

Contribution

It extends ABI to mixture models by combining generative and classification networks, enabling fast, likelihood-free inference for both independent and dependent mixtures.

Findings

Effective in synthetic datasets

Applicable to real-world data

Handles both independent and dependent mixtures

Abstract

Finite mixtures are a broad class of models useful in scenarios where observed data is generated by multiple distinct processes but without explicit information about the responsible process for each data point. Estimating Bayesian mixture models is computationally challenging due to issues such as high-dimensional posterior inference and label switching. Furthermore, traditional methods such as MCMC are applicable only if the likelihoods for each mixture component are analytically tractable. Amortized Bayesian Inference (ABI) is a simulation-based framework for estimating Bayesian models using generative neural networks. This allows the fitting of models without explicit likelihoods, and provides fast inference. ABI is therefore an attractive framework for estimating mixture models. This paper introduces a novel extension of ABI tailored to mixture models. We factorize the posterior into a distribution of the parameters and a distribution of (categorical) mixture indicators, which allows us to use a combination of generative neural networks for parameter inference, and classification networks for mixture membership identification. The proposed framework accommodates both independent and dependent mixture models, enabling filtering and smoothing. We validate and demonstrate our approach through synthetic and real-world datasets.