A framework for paired-sample hypothesis testing for high-dimensional data

TL;DR

This paper introduces a novel high-dimensional paired-sample testing framework that leverages hyperplane-based scoring functions and the Hodges-Lehmann estimator, improving accuracy over traditional methods.

Contribution

It proposes a new two-step testing procedure using hyperplanes and the Hodges-Lehmann estimator to enhance high-dimensional paired-sample hypothesis testing.

Findings

Substantial performance gains in testing accuracy.

Effective estimation of feature contributions.

Outperforms traditional multivariate and multiple testing methods.

Abstract

The standard paired-sample testing approach in the multidimensional setting applies multiple univariate tests on the individual features, followed by p-value adjustments. Such an approach suffers when the data carry numerous features. A number of studies have shown that classification accuracy can be seen as a proxy for two-sample testing. However, neither theoretical foundations nor practical recipes have been proposed so far on how this strategy could be extended to multidimensional paired-sample testing. In this work, we put forward the idea that scoring functions can be produced by the decision rules defined by the perpendicular bisecting hyperplanes of the line segments connecting each pair of instances. Then, the optimal scoring function can be obtained by the pseudomedian of those rules, which we estimate by extending naturally the Hodges-Lehmann estimator. We accordingly propose a framework of a two-step testing procedure. First, we estimate the bisecting hyperplanes for each pair of instances and an aggregated rule derived through the Hodges-Lehmann estimator. The paired samples are scored by this aggregated rule to produce a unidimensional representation. Second, we perform a Wilcoxon signed-rank test on the obtained representation. Our experiments indicate that our approach has substantial performance gains in testing accuracy compared to the traditional multivariate and multiple testing, while at the same time estimates each feature's contribution to the final result.