A Theoretical Comparison of No-U-Turn Sampler Variants: Necessary and Sufficient Convergence Conditions and Mixing Time Analysis under Gaussian Targets

TL;DR

This paper provides a comprehensive theoretical comparison of NUTS variants, establishing convergence conditions and analyzing mixing times, revealing both qualitative similarities and quantitative differences in their efficiency.

Contribution

It derives the first necessary and sufficient conditions for geometric ergodicity and provides the first mixing time analysis for NUTS-BPS on Gaussian targets.

Findings

Both NUTS-mul and NUTS-BPS are geometrically ergodic depending on target tail properties.

Mixing times scale as O(d^{1/4}) for both variants in Gaussian settings.

Constants in mixing time bounds are smaller for NUTS-BPS, indicating faster convergence.

Abstract

The No-U-Turn Sampler (NUTS) is the computational workhorse of modern Bayesian software libraries, yet its qualitative and quantitative convergence guarantees were established only recently. A significant gap remains in the theoretical comparison of its two main variants: NUTS-mul and NUTS-BPS, which use multinomial sampling and biased progressive sampling, respectively, for index selection. In this paper, we address this gap in three contributions. First, we derive the first necessary conditions for geometric ergodicity for both variants. Second, we establish the first sufficient conditions for geometric ergodicity and ergodicity for NUTS-mul. Third, we obtain the first mixing time result for NUTS-BPS on a standard Gaussian distribution. Our results show that NUTS-mul and NUTS-BPS exhibit nearly identical qualitative behavior, with geometric ergodicity depending on the tail properties of the target distribution. However, they differ quantitatively in their convergence rates. More precisely, when initialized in the typical set of the canonical Gaussian measure, the mixing times of both NUTS-mul and NUTS-BPS scale as $O(d^{1/4})$ up to logarithmic factors, where $d$ denotes the dimension. Nevertheless, the associated constants are strictly smaller for NUTS-BPS.