Temporal Parallelization of Inference in Hidden Markov Models

TL;DR

This paper introduces parallel algorithms for inference in hidden Markov models, enabling efficient processing of long sequences by leveraging parallel scan techniques on GPUs.

Contribution

It proposes novel parallel backward-forward and Viterbi algorithms for HMM inference, formulated as parallel prefix-sum computations, improving efficiency for long sequences.

Findings

Significant speedup over classical methods on GPU

Effective parallelization of filtering and smoothing algorithms

Enhanced scalability for long time horizon HMMs

Abstract

This paper presents algorithms for parallelization of inference in hidden Markov models (HMMs). In particular, we propose parallel backward-forward type of filtering and smoothing algorithm as well as parallel Viterbi-type maximum-a-posteriori (MAP) algorithm. We define associative elements and operators to pose these inference problems as parallel-prefix-sum computations in sum-product and max-product algorithms and parallelize them using parallel-scan algorithms. The advantage of the proposed algorithms is that they are computationally efficient in HMM inference problems with long time horizons. We empirically compare the performance of the proposed methods to classical methods on a highly parallel graphical processing unit (GPU).