Variational Bayesian Methods for a Tree-Structured Stick-Breaking Process Mixture of Gaussians by Application of the Bayes Codes for Context Tree Models

TL;DR

This paper introduces a variational Bayesian approach for the tree-structured stick-breaking process mixture of Gaussians, significantly reducing computational costs compared to traditional MCMC methods by leveraging Bayes coding techniques.

Contribution

It presents the first variational Bayesian algorithm for TS-SBP mixture models, enabling efficient learning under finite tree assumptions.

Findings

VB method is computationally more efficient than MCMC.

The proposed VB approach accurately models hierarchical structures.

Experimental results confirm reduced computational costs.

Abstract

The tree-structured stick-breaking process (TS-SBP) mixture model is a non-parametric Bayesian model that can represent tree-like hierarchical structures among the mixture components. For TS-SBP mixture models, only a Markov chain Monte Carlo (MCMC) method has been proposed and any variational Bayesian (VB) methods has not been proposed. In general, MCMC methods are computationally more expensive than VB methods. Therefore, we require a large computational cost to learn the TS-SBP mixture model. In this paper, we propose a learning algorithm with less computational cost for the TS-SBP mixture of Gaussians by using the VB method under an assumption of finite tree width and depth. When constructing such VB method, the main challenge is efficient calculation of a sum over all possible trees. To solve this challenge, we utilizes a subroutine in the Bayes coding algorithm for context tree models. We confirm the computational efficiency of our VB method through an experiments on a benchmark dataset.