Complexity bounds for Dirichlet process slice samplers

TL;DR

This paper provides theoretical bounds on the computational complexity of Dirichlet process slice samplers, showing they scale efficiently with high probability across various posterior regimes, supporting their practical use.

Contribution

It offers the first high-probability complexity bounds for DP slice samplers, demonstrating their scalability regardless of dataset or posterior cluster growth.

Findings

Overhead of slice variables is O(log n) with high probability

Superlinear computational costs are rare in worst-case scenarios

Results apply broadly to DP models without likelihood restrictions

Abstract

Slice sampling is a standard Monte Carlo technique for Dirichlet process (DP)-based models, widely used in posterior simulation. However, formal assessments of the scalability of posterior slice samplers have remained largely unexplored, primarily because the computational cost of a slice-sampling iteration is random and potentially unbounded. In this work, we obtain high-probability bounds on the computational complexity of DP slice samplers. Our main results show that, uniformly across posterior cluster-growth regimes, the overhead induced by slice variables, relatively to the number of clusters supported by the posterior, is $O_{\mathbb P}(\log n)$. As a consequence, even in worst-case configurations, superlinear blow-ups in per-iteration computational cost occur with vanishing probability. Our analysis applies broadly to DP-based models without any likelihood-specific assumptions, still providing complexity guarantees for posterior sampling on arbitrary datasets. These results establish a theoretical foundation for assessing the practical scalability of slice sampling in DP-based models.