Signature Maximum Mean Discrepancy Two-Sample Statistical Tests

TL;DR

This paper explores the signature Maximum Mean Discrepancy (sig-MMD), a kernel-based method for comparing path space distributions, analyzing its practical application, challenges like Type 2 errors, and mitigation techniques.

Contribution

It introduces the sig-MMD for two-sample testing on path space, providing practical examples, identifying challenges, and proposing solutions for effective use.

Findings

sig-MMD can be used to compare distributions on path space

Type 2 errors can occur in limited data settings

Techniques to reduce false negatives are proposed

Abstract

Maximum Mean Discrepancy (MMD) is a widely used concept in machine learning research which has gained popularity in recent years as a highly effective tool for comparing (finite-dimensional) distributions. Since it is designed as a kernel-based method, the MMD can be extended to path space valued distributions using the signature kernel. The resulting signature MMD (sig-MMD) can be used to define a metric between distributions on path space. Similarly to the original use case of the MMD as a test statistic within a two-sample testing framework, the sig-MMD can be applied to determine if two sets of paths are drawn from the same stochastic process. This work is dedicated to understanding the possibilities and challenges associated with applying the sig-MMD as a statistical tool in practice. We introduce and explain the sig-MMD, and provide easily accessible and verifiable examples for its practical use. We present examples that can lead to Type 2 errors in the hypothesis test, falsely indicating that samples have been drawn from the same underlying process (which generally occurs in a limited data setting). We then present techniques to mitigate the occurrence of this type of error.