Finite-Particle Rates for Regularized Stein Variational Gradient Descent

TL;DR

This paper establishes explicit finite-particle convergence rates for the regularized Stein variational gradient descent (R-SVGD), improving understanding of its accuracy and providing practical tuning guidelines.

Contribution

It derives non-asymptotic bounds for R-SVGD, connecting convergence in Fisher information and Wasserstein distance with finite-particle effects and regularization.

Findings

Explicit non-asymptotic convergence bounds for R-SVGD.

Guidelines for tuning regularization, step size, and averaging.

Convergence in Fisher information and Wasserstein distance.

Abstract

We derive finite-particle rates for the regularized Stein variational gradient descent (R-SVGD) algorithm introduced by He et al. (2024) that corrects the constant-order bias of the SVGD by applying a resolvent-type preconditioner to the kernelized Wasserstein gradient. For the resulting interacting $N$-particle system, we establish explicit non-asymptotic bounds for time-averaged (annealed) empirical measures, illustrating convergence in the \emph{true} (non-kernelized) Fisher information and, under a $\mathrm{W}_1\mathrm{I}$ condition on the target, corresponding $\mathrm{W}_1$ convergence for a large class of smooth kernels. Our analysis covers both continuous- and discrete-time dynamics and yields principled tuning rules for the regularization parameter, step size, and averaging horizon that quantify the trade-off between approximating the Wasserstein gradient flow and controlling finite-particle estimation error.