A Separation in Heavy-Tailed Sampling: Gaussian vs. Stable Oracles for Proximal Samplers

TL;DR

This paper demonstrates a fundamental separation in heavy-tailed sampling, showing Gaussian-based samplers are limited to low-accuracy guarantees while stable-based samplers can achieve high-accuracy results.

Contribution

It introduces a separation result between Gaussian and stable oracles in proximal samplers, highlighting the limitations and advantages of each in heavy-tailed sampling.

Findings

Gaussian oracles have a fundamental barrier to high-accuracy guarantees.

Stable oracles enable high-accuracy guarantees in heavy-tailed sampling.

Lower bounds match the upper bounds, confirming the optimality of the results.

Abstract

We study the complexity of heavy-tailed sampling and present a separation result in terms of obtaining high-accuracy versus low-accuracy guarantees i.e., samplers that require only $O(\log(1/\varepsilon))$ versus $\Omega(\text{poly}(1/\varepsilon))$ iterations to output a sample which is $\varepsilon$-close to the target in $\chi^2$-divergence. Our results are presented for proximal samplers that are based on Gaussian versus stable oracles. We show that proximal samplers based on the Gaussian oracle have a fundamental barrier in that they necessarily achieve only low-accuracy guarantees when sampling from a class of heavy-tailed targets. In contrast, proximal samplers based on the stable oracle exhibit high-accuracy guarantees, thereby overcoming the aforementioned limitation. We also prove lower bounds for samplers under the stable oracle and show that our upper bounds cannot be fundamentally improved.