MMD Two-sample Testing in the Presence of Arbitrarily Missing Data

TL;DR

This paper introduces a novel MMD-based two-sample test that effectively handles arbitrarily missing data without assumptions on missingness, maintaining Type I error control and demonstrating good power in simulations.

Contribution

It develops the first two-sample testing method that guarantees Type I error control with arbitrarily missing data, applicable to both univariate and multivariate cases.

Findings

Controls Type I error in presence of missing data

Maintains good statistical power with 5-10% missing data

Effective even when data are missing not at random

Abstract

In many real-world applications, it is common that a proportion of the data may be missing or only partially observed. We develop a novel two-sample testing method based on the Maximum Mean Discrepancy (MMD) which accounts for missing data in both samples, without making assumptions about the missingness mechanism. Our approach is based on deriving the mathematically precise bounds of the MMD test statistic after accounting for all possible missing values. To the best of our knowledge, it is the only two-sample testing method that is guaranteed to control the Type I error for both univariate and multivariate data where data may be arbitrarily missing. Simulation results show that our method has good statistical power, typically for cases where 5% to 10% of the data are missing. We highlight the value of our approach when the data are missing not at random, a context in which either ignoring the missing values or using common imputation methods may not control the Type I error.