A kernel Stein test of goodness of fit for sequential models

TL;DR

This paper introduces a novel kernel Stein discrepancy-based goodness-of-fit test tailored for sequential models with variable-dimensional data, extending existing methods to handle sequences of differing lengths without requiring normalized densities.

Contribution

The authors extend the kernel Stein discrepancy to variable-dimensional data, enabling goodness-of-fit testing for sequential models with varying sequence lengths.

Findings

Test performs well on discrete sequential data benchmarks

Does not require the density to be normalized

Extends KSD to variable-dimension setting

Abstract

We propose a goodness-of-fit measure for probability densities modeling observations with varying dimensionality, such as text documents of differing lengths or variable-length sequences. The proposed measure is an instance of the kernel Stein discrepancy (KSD), which has been used to construct goodness-of-fit tests for unnormalized densities. The KSD is defined by its Stein operator: current operators used in testing apply to fixed-dimensional spaces. As our main contribution, we extend the KSD to the variable-dimension setting by identifying appropriate Stein operators, and propose a novel KSD goodness-of-fit test. As with the previous variants, the proposed KSD does not require the density to be normalized, allowing the evaluation of a large class of models. Our test is shown to perform well in practice on discrete sequential data benchmarks.