Stochastic Collapsed Variational Inference for Sequential Data

TL;DR

This paper introduces a stochastic collapsed variational inference algorithm tailored for sequential data, applicable to hidden Markov models and hierarchical Dirichlet process models, demonstrating improved efficiency and accuracy over uncollapsed methods.

Contribution

It presents a novel stochastic collapsed variational inference method for sequential models, extending applicability to various HMMs and exponential family emissions.

Findings

More efficient inference compared to uncollapsed methods

Higher accuracy demonstrated on discrete datasets

Applicable to both finite and hierarchical Dirichlet process HMMs

Abstract

Stochastic variational inference for collapsed models has recently been successfully applied to large scale topic modelling. In this paper, we propose a stochastic collapsed variational inference algorithm in the sequential data setting. Our algorithm is applicable to both finite hidden Markov models and hierarchical Dirichlet process hidden Markov models, and to any datasets generated by emission distributions in the exponential family. Our experiment results on two discrete datasets show that our inference is both more efficient and more accurate than its uncollapsed version, stochastic variational inference.