Random Feature Stein Discrepancies

TL;DR

This paper introduces feature Stein discrepancies ($\

Contribution

It proposes a new family of Stein discrepancies, $\\Phi$SDs, that are computationally efficient and maintain strong convergence guarantees, improving over existing methods.

Findings

R$\Phi$SDs are computable in near-linear time.

R$\Phi$SDs perform as well or better than quadratic-time KSDs.

R$\Phi$SDs are effective for sampler selection and goodness-of-fit testing.

Abstract

Computable Stein discrepancies have been deployed for a variety of applications, ranging from sampler selection in posterior inference to approximate Bayesian inference to goodness-of-fit testing. Existing convergence-determining Stein discrepancies admit strong theoretical guarantees but suffer from a computational cost that grows quadratically in the sample size. While linear-time Stein discrepancies have been proposed for goodness-of-fit testing, they exhibit avoidable degradations in testing power -- even when power is explicitly optimized. To address these shortcomings, we introduce feature Stein discrepancies ($\Phi$SDs), a new family of quality measures that can be cheaply approximated using importance sampling. We show how to construct $\Phi$SDs that provably determine the convergence of a sample to its target and develop high-accuracy approximations -- random $\Phi$SDs (R$\Phi$SDs) -- which are computable in near-linear time. In our experiments with sampler selection for approximate posterior inference and goodness-of-fit testing, R$\Phi$SDs perform as well or better than quadratic-time KSDs while being orders of magnitude faster to compute.