Incorporating Local Step-Size Adaptivity into the No-U-Turn Sampler using Gibbs Self Tuning

TL;DR

This paper introduces a novel method for local step-size adaptation in the No-U-Turn Sampler (NUTS) using Gibbs self-tuning, improving efficiency in complex, high-dimensional distributions.

Contribution

It presents a reversible, Gibbs self-tuning approach for local step-size adaptation in NUTS, addressing interdependence issues in tuning parameters.

Findings

Effective adaptation on Neal's funnel density

Improved performance on high-dimensional normal distributions

Guarantees reversibility and acceptance probability based on conditional step size

Abstract

Adapting the step size locally in the no-U-turn sampler (NUTS) is challenging because the step-size and path-length tuning parameters are interdependent. The determination of an optimal path length requires a predefined step size, while the ideal step size must account for errors along the selected path. Ensuring reversibility further complicates this tuning problem. In this paper, we present a method for locally adapting the step size in NUTS that is an instance of the Gibbs self-tuning (GIST) framework. Our approach guarantees reversibility with an acceptance probability that depends exclusively on the conditional distribution of the step size. We validate our step-size-adaptive NUTS method on Neal's funnel density and a high-dimensional normal distribution, demonstrating its effectiveness in challenging scenarios.