# Stochastic Zeroth-order Discretizations of Langevin Diffusions for Bayesian Inference

## TL;DR

This paper introduces stochastic zeroth-order discretizations for Langevin diffusions, enabling Bayesian sampling without gradient evaluations, and provides theoretical analysis of their sample complexity and effectiveness in noisy, high-dimensional scenarios.

## Contribution

The work develops and analyzes zeroth-order Langevin discretizations using Gaussian Stein's identities, extending sampling methods to black-box and high-dimensional contexts.

## Key findings

- Provides sample complexity bounds for zeroth-order Langevin algorithms.
- Introduces a variable selection technique with theoretical guarantees.
- Extends Langevin sampling to noisy black-box and high-dimensional settings.

## Abstract

Discretizations of Langevin diffusions provide a powerful method for sampling and Bayesian inference. However, such discretizations require evaluation of the gradient of the potential function. In several real-world scenarios, obtaining gradient evaluations might either be computationally expensive, or simply impossible. In this work, we propose and analyze stochastic zeroth-order sampling algorithms for discretizing overdamped and underdamped Langevin diffusions. Our approach is based on estimating the gradients, based on Gaussian Stein's identities, widely used in the stochastic optimization literature. We provide a comprehensive sample complexity analysis -- number noisy function evaluations to be made to obtain an $\epsilon$-approximate sample in Wasserstein distance -- of stochastic zeroth-order discretizations of both overdamped and underdamped Langevin diffusions, under various noise models. We also propose a variable selection technique based on zeroth-order gradient estimates and establish its theoretical guarantees. Our theoretical contributions extend the practical applicability of sampling algorithms to the noisy black-box and high-dimensional settings.