Analysis of two-component Gibbs samplers using the theory of two projections

TL;DR

This paper applies the theory of two projections to analyze two-component Gibbs samplers, revealing conditions under which random-scan can outperform deterministic-scan in terms of asymptotic variance, especially with cost disparities.

Contribution

It introduces a novel application of the theory of two projections to simplify analysis of Gibbs samplers and compares their asymptotic variances under different scanning strategies.

Findings

Random-scan Gibbs sampler can be less or more efficient than deterministic-scan depending on parameters.

Deterministic-scan has faster convergence rate but may have higher asymptotic variance.

Modified deterministic-scan considering computation cost can outperform random-scan.

Abstract

The theory of two projections is utilized to study two-component Gibbs samplers. Through this theory, previously intractable problems regarding the asymptotic variances of two-component Gibbs samplers are reduced to elementary matrix algebra exercises. It is found that in terms of asymptotic variance, the two-component random-scan Gibbs sampler is never much worse, and could be considerably better than its deterministic-scan counterpart, provided that the selection probability is appropriately chosen. This is especially the case when there is a large discrepancy in computation cost between the two components. The result contrasts with the known fact that the deterministic-scan version has a faster convergence rate, which can also be derived from the method herein. On the other hand, a modified version of the deterministic-scan sampler that accounts for computation cost can outperform the random-scan version.