Stein Variational Gradient Descent dynamics for highly concentrated kernels

TL;DR

This paper investigates the behavior of Stein Variational Gradient Descent (SVGD) as the kernel bandwidth approaches zero, revealing its convergence to a local gradient flow and clarifying the collapse from a nonlocal to a local dynamics.

Contribution

It provides a rigorous analysis of SVGD's singular limit, showing convergence to a local evolution equation and employing Stein-log-Sobolev inequalities for weighted kernels.

Findings

SVGD dynamics converge to a local gradient flow as kernel bandwidth tends to zero.

The analysis covers both integrable and weighted kernels.

Stein-log-Sobolev inequalities support the convergence proof for weighted kernels.

Abstract

Stein Variational Gradient Descent (SVGD) is a widely used in practice algorithm for scalable sampling with deterministic particle updates. We study its behavior in the singular limit where the kernel bandwidth tends to zero. In this regime, we show that the nonlocal SVGD dynamics converge to a local evolution equation that can be formally interpreted as a Wasserstein gradient flow with quadratic mobility. We analyze this singular limit in two settings: integrable kernels and weighted kernels. In the weighted case, the proof is supported by recently established Stein-log-Sobolev inequalities, which provide the necessary functional control. Overall, our results clarify how SVGD collapses from a nonlocal interacting particle system to a local gradient-flow dynamics as the kernel concentrates.