A Kernel Test of Goodness of Fit

TL;DR

This paper introduces a nonparametric goodness-of-fit test using kernel methods and Stein's divergence, applicable to i.i.d. and dependent samples, with practical applications in MCMC convergence and model criticism.

Contribution

It develops a novel kernel-based divergence measure for goodness-of-fit testing, with a bootstrap-based null distribution estimation applicable to various data dependencies.

Findings

Effective in quantifying MCMC convergence

Applicable to model criticism and density estimation

Uses wild bootstrap for null distribution estimation

Abstract

We propose a nonparametric statistical test for goodness-of-fit: given a set of samples, the test determines how likely it is that these were generated from a target density function. The measure of goodness-of-fit is a divergence constructed via Stein's method using functions from a Reproducing Kernel Hilbert Space. Our test statistic is based on an empirical estimate of this divergence, taking the form of a V-statistic in terms of the log gradients of the target density and the kernel. We derive a statistical test, both for i.i.d. and non-i.i.d. samples, where we estimate the null distribution quantiles using a wild bootstrap procedure. We apply our test to quantifying convergence of approximate Markov Chain Monte Carlo methods, statistical model criticism, and evaluating quality of fit vs model complexity in nonparametric density estimation.