Physics-driven Comparative Analysis of Various Statistical Distance Metrics and Normalizing Functions

TL;DR

This paper systematically compares various statistical distance metrics and normalizing functions using experimental data from a radioactive isotope, analyzing their stability and properties in scientific data analysis.

Contribution

It introduces a data-driven framework for comparing statistical distances and normalizing functions, emphasizing their stability under different conditions.

Findings

Wasserstein and Hellinger distances showed high stability across conditions.

Normalizing functions with specific properties improved metric performance.

The study provides guidelines for selecting distance metrics in scientific analyses.

Abstract

Comparison of two probability density/mass functions (PDF/PMFs) is ubiquitous in various forms of scientific analysis, including machine learning, optimization problems, and hypothesis tests. A copious amount of distance metrics have already been proposed and are regularly being used in this regard. In this document, we report a data-driven systematic comparison among a few of such metrics. The metrics considered here are Hellinger distance, Wasserstein distances (1D), $\sqrt{JS}$ distance, $L_\infty$ norm, Kolmogorov-Smirnov distance, and Fisher-Rao metric. We perform this comparison using electron and photon events from a decaying \iso{Kr}{83} isotope, collected through an HPGe spectrometer operating under cryo-vacuum conditions. To accomplish this, first, a dimensionless Parameter of Interest (PoI) was established, then PDF/PMFs were generated from the data, and finally the stabilities of the PoI under various criteria, such as sample size, discretization length, and normalizing functions, were studied and the results were summarized. In this report, we also propose a list of properties that a normalizing function should have and utilize them in the comparison.