Kernelized Complete Conditional Stein Discrepancy

TL;DR

This paper introduces kernelized complete conditional Stein discrepancies (KCC-SDs), a new method for distinguishing distributions and assessing sample quality, with improved power over existing baselines, especially in high-dimensional settings.

Contribution

The paper proposes KCC-SDs, a novel discrepancy measure based on complete conditionals, enhancing the ability to distinguish distributions and evaluate MCMC samples.

Findings

KCC-SDs effectively distinguish distributions.

KCC-SDs outperform baseline methods in power.

KCC-SDs are useful for assessing MCMC sample quality.

Abstract

Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepancies (KSDs). While kernels make Stein discrepancies tractable, they pose several challenges in high dimensions. We introduce kernelized complete conditional Stein discrepancies (KCC-SDs). Complete conditionals turn a multivariate distribution into multiple univariate distributions. We show that KCC-SDs distinguish distributions. To show the efficacy of KCC-SDs in distinguishing distributions, we introduce a goodness-of-fit test using KCC-SDs. We empirically show that KCC-SDs have higher power over baselines and use KCC-SDs to assess sample quality in Markov chain Monte Carlo.