MCMC for Hierarchical Semi-Markov Conditional Random Fields

TL;DR

This paper introduces an approximation method combining Gibbs sampling and Rao-Blackwellisation for hierarchical semi-Markov models, aiming to reduce inference complexity from cubic or exponential to sub-cubic or linear, with some quality trade-offs.

Contribution

It proposes a novel approximation technique that improves inference efficiency in hierarchical semi-Markov models, enabling scalable analysis of large and deep nested sequential data.

Findings

RGBS offers faster inference with longer sequences.

Trade-off between computational speed and inference quality.

Simulation results demonstrate potential for sub-cubic complexity.

Abstract

Deep architecture such as hierarchical semi-Markov models is an important class of models for nested sequential data. Current exact inference schemes either cost cubic time in sequence length, or exponential time in model depth. These costs are prohibitive for large-scale problems with arbitrary length and depth. In this contribution, we propose a new approximation technique that may have the potential to achieve sub-cubic time complexity in length and linear time depth, at the cost of some loss of quality. The idea is based on two well-known methods: Gibbs sampling and Rao-Blackwellisation. We provide some simulation-based evaluation of the quality of the RGBS with respect to run time and sequence length.