Auxiliary Variables for Multi-Dirichlet Priors

TL;DR

This paper introduces an auxiliary variable scheme that simplifies inference in Bayesian models with Multi-Dirichlet priors, enabling efficient and fully collapsed inference methods for hierarchical models.

Contribution

It proposes a novel auxiliary variable approach that makes inference in hierarchical Multi-Dirichlet models more tractable and efficient.

Findings

Simplifies inference in hierarchical Multi-Dirichlet models

Enables derivation of fully collapsed inference schemes

Improves computational efficiency of Bayesian inference

Abstract

Bayesian models that mix multiple Dirichlet prior parameters, called Multi-Dirichlet priors (MD) in this paper, are gaining popularity. Inferring mixing weights and parameters of mixed prior distributions seems tricky, as sums over Dirichlet parameters complicate the joint distribution of model parameters. This paper shows a novel auxiliary variable scheme which helps to simplify the inference for models involving hierarchical MDs and MDPs. Using this scheme, it is easy to derive fully collapsed inference schemes which allow for an efficient inference.