Variable Selection for Kernel Two-Sample Tests

TL;DR

This paper introduces a new kernel-based variable selection method for two-sample tests that improves detection power in high-dimensional data by selecting the most informative variables.

Contribution

It proposes a novel framework using kernel MMD with mixed-integer programming and algorithms, providing theoretical power analysis and superior empirical performance.

Findings

Sample size depends on selected variables, not data dimension.

Method outperforms existing variable selection techniques.

Effective for high-dimensional two-sample testing.

Abstract

We consider the variable selection problem for two-sample tests, aiming to select the most informative variables to determine whether two collections of samples follow the same distribution. To address this, we propose a novel framework based on the kernel maximum mean discrepancy (MMD). Our approach seeks a subset of variables with a pre-specified size that maximizes the variance-regularized kernel MMD statistic. We focus on three commonly used types of kernels: linear, quadratic, and Gaussian. From a computational perspective, we derive mixed-integer programming formulations and propose exact and approximation algorithms with performance guarantees to solve these formulations. From a statistical viewpoint, we derive the rate of testing power of our framework under appropriate conditions. These results show that the sample size requirements for the three kernels depend crucially on the number of selected variables, rather than the data dimension. Experimental results on synthetic and real datasets demonstrate the superior performance of our method, compared to other variable selection frameworks, particularly in high-dimensional settings.