Controlled Accuracy Gibbs Sampling of Order Constrained Non-IID Ordered Random Variates

TL;DR

This paper introduces a novel Gibbs sampling method for accurately simulating order statistics from non-iid variables with order constraints, applicable in Bayesian PCA and other statistical models.

Contribution

It proposes a coupling-from-the-past algorithm for perfect simulation of order-constrained non-iid order statistics, improving accuracy over existing methods.

Findings

Effective simulation demonstrated in Bayesian PCA applications

Algorithm achieves high accuracy in generating order statistics

Applicable to various non-iid order statistic problems

Abstract

Order statistics arising from $m$ independent but not identically distributed random variables are typically constructed by arranging some $X_{1}, X_{2}, \ldots, X_{m}$, with $X_{i}$ having distribution function $F_{i}(x)$, in increasing order denoted as $X_{(1)} \leq X_{(2)} \leq \ldots \leq X_{(m)}$. In this case, $X_{(i)}$ is not necessarily associated with $F_{i}(x)$. Assuming one can simulate values from each distribution, one can generate such "non-iid" order statistics by simulating $X_{i}$ from $F_{i}$, for $i=1,2,\ldots, m$, and arranging them in order. In this paper, we consider the problem of simulating ordered values $X_{(1)}, X_{(2)}, \ldots, X_{(m)}$ such that the marginal distribution of $X_{(i)}$ is $F_{i}(x)$. This problem arises in Bayesian principal components analysis (BPCA) where the $X_{i}$ are ordered eigenvalues that are a posteriori independent but not identically distributed. We propose a novel coupling-from-the-past algorithm to "perfectly" (up to computable order of accuracy) simulate such {\emph{order-constrained non-iid}} order statistics. We demonstrate the effectiveness of our approach for several examples, including the BPCA problem.