High Probability Lower Bounds for the Total Variation Distance

TL;DR

This paper proposes a new framework for constructing high-probability lower bounds on the total variation distance using classification or regression methods, providing a refined measure of distributional differences beyond rejection decisions.

Contribution

It introduces a novel approach to estimate lower bounds on total variation distance via one-dimensional projections, enhancing interpretability in two-sample testing.

Findings

Derived asymptotic power and detection rates for proposed estimators

Demonstrated application on climate dataset reanalysis

Provided a framework for refined distributional difference measurement

Abstract

The statistics and machine learning communities have recently seen a growing interest in classification-based approaches to two-sample testing. The outcome of a classification-based two-sample test remains a rejection decision, which is not always informative since the null hypothesis is seldom strictly true. Therefore, when a test rejects, it would be beneficial to provide an additional quantity serving as a refined measure of distributional difference. In this work, we introduce a framework for the construction of high-probability lower bounds on the total variation distance. These bounds are based on a one-dimensional projection, such as a classification or regression method, and can be interpreted as the minimal fraction of samples pointing towards a distributional difference. We further derive asymptotic power and detection rates of two proposed estimators and discuss potential uses through an application to a reanalysis climate dataset.