Moment Expansions of the Energy Distance

TL;DR

This paper analyzes the sensitivity of the energy distance to differences in moments of distributions, especially when the distributions are close, revealing a focus on mean differences over covariance differences and the effects of off-diagonal components.

Contribution

It provides a moment expansion analysis of the energy distance, highlighting its sensitivity to mean differences and the structure of covariance differences in near-identical distributions.

Findings

Energy distance is more sensitive to mean differences than covariance differences when distributions are close.

Off-diagonal covariance components contribute less to the energy distance in isotropic cases.

Numerical results support the theoretical sensitivity analysis even with non-ideal distributions.

Abstract

The energy distance is used to test distributional equality, and as a loss function in machine learning. While $D^2(X, Y)=0$ only when $X\sim Y$, the sensitivity to different moments is of practical importance. This work considers $D^2(X, Y)$ in the case where the distributions are close. In this regime, $D^2(X, Y)$ is more sensitive to differences in the means $\bar{X}-\bar{Y}$, than differences in the covariances $\Delta$. This is due to the structure of the energy distance and is independent of dimension. The sensitivity to on versus off diagonal components of $\Delta$ is examined when $X$ and $Y$ are close to isotropic. Here a dimension dependent averaging occurs and, in many cases, off diagonal correlations contribute significantly less. Numerical results verify these relationships hold even when distributional assumptions are not strictly met.