Kernel Tests of Equivalence

TL;DR

This paper introduces new kernel-based equivalence tests for distributions that effectively determine the absence of meaningful differences, overcoming limitations of traditional goodness-of-fit tests by focusing on full distributional equivalence.

Contribution

It develops kernel Stein discrepancy and MMD-based tests for distribution equivalence, with novel critical value computation methods including asymptotic and bootstrap approaches.

Findings

The proposed tests accurately assess distributional equivalence.

They outperform existing methods in various numerical experiments.

The tests control error rates effectively.

Abstract

We propose novel kernel-based tests for assessing the equivalence between distributions. Traditional goodness-of-fit testing is inappropriate for concluding the absence of distributional differences, because failure to reject the null hypothesis may simply be a result of lack of test power, also known as the Type-II error. This motivates \emph{equivalence testing}, which aims to assess the \emph{absence} of a statistically meaningful effect under controlled error rates. However, existing equivalence tests are either limited to parametric distributions or focus only on specific moments rather than the full distribution. We address these limitations using two kernel-based statistical discrepancies: the \emph{kernel Stein discrepancy} and the \emph{Maximum Mean Discrepancy}. The null hypothesis of our proposed tests assumes the candidate distribution differs from the nominal distribution by at least a pre-defined margin, which is measured by these discrepancies. We propose two approaches for computing the critical values of the tests, one using an asymptotic normality approximation, and another based on bootstrapping. Numerical experiments are conducted to assess the performance of these tests.