Magnitude Distance: A Geometric Measure of Dataset Similarity

TL;DR

This paper introduces magnitude distance, a new geometric metric for dataset similarity that adapts to different scales and remains effective in high-dimensional spaces, with applications in training generative models.

Contribution

The paper proposes magnitude distance, a novel dataset distance metric based on the magnitude of metric spaces, with theoretical properties and practical use in generative model training.

Findings

Magnitude distance is discriminative in high-dimensional settings.

It has desirable theoretical properties across scales.

It performs comparably to established distances in generative tasks.

Abstract

Quantifying the distance between datasets is a fundamental question in mathematics and machine learning. We propose \textit{magnitude distance}, a novel distance metric defined on finite datasets using the notion of the \emph{magnitude} of a metric space. The proposed distance incorporates a tunable scaling parameter, $t$, that controls the sensitivity to global structure (small $t$) and finer details (large $t$). We prove several theoretical properties of magnitude distance, including its limiting behavior across scales and conditions under which it satisfies key metric properties. In contrast to classical distances, we show that magnitude distance remains discriminative in high-dimensional settings when the scale is appropriately tuned. We further demonstrate how magnitude distance can be used as a training objective for push-forward generative models. Our experimental results support our theoretical analysis and demonstrate that magnitude distance provides meaningful signals, comparable to established distance-based generative approaches.