Estimation of An Infinite Dimensional Transition Probability Matrix Using a Generalized Hierarchical Stick-Breaking Process

TL;DR

This paper introduces a Bayesian nonparametric method using a Generalized Hierarchical Stick-Breaking prior to estimate transition matrices in infinite or expanding state spaces, addressing limitations of classical methods.

Contribution

It develops a novel prior that extends traditional stick-breaking processes, enabling flexible modeling of infinite-dimensional transition matrices for complex stochastic processes.

Findings

Provides a new Bayesian framework for infinite state spaces.

Enables inference in high-dimensional and sparse settings.

Extends existing priors to more flexible hierarchical models.

Abstract

Markov chains provide a foundational framework for modeling sequential stochastic processes, with the transition probability matrix characterizing the dynamics of state evolution. While classical estimation methods such as maximum likelihood and empirical Bayes approaches are effective in finite-state settings, they become inadequate in applications involving countably infinite or dynamically expanding state spaces, which frequently arise in domains such as natural language processing, population dynamics, and behavioral modeling. In this work, we introduce a novel Bayesian nonparametric framework for estimating infinite-dimensional transition probability matrices by employing a new class of priors, termed the Generalized Hierarchical Stick-Breaking prior. This prior extends traditional Dirichlet process and stick-breaking constructions, enabling highly flexible modelling of transition probability matrices. The proposed approach offers a principled methodology for inferring transition probabilities in settings characterized by sparsity, high dimensionality, and unobserved state spaces, thereby contributing to the advancement of statistical inference for infinite-dimensional transition probability matrices.