Cross-Fusion Distance: A Novel Metric for Measuring Fusion and Separability Between Data Groups in Representation Space

TL;DR

This paper introduces Cross-Fusion Distance, a new metric that accurately measures the true fusion and separability of data groups in representation space, especially under domain shifts, by isolating geometric factors.

Contribution

The paper proposes CFD, a novel metric that isolates fusion-altering geometry from other variations, with theoretical analysis and validation on synthetic and real-world datasets.

Findings

CFD aligns better with downstream generalization than existing metrics.

CFD is invariant to fusion-preserving factors like scaling and layout changes.

CFD has linear computational complexity.

Abstract

Quantifying degrees of fusion and separability between data groups in representation space is a fundamental problem in representation learning, particularly under domain shift. A meaningful metric should capture fusion-altering factors like geometric displacement between representation groups, whose variations change the extent of fusion, while remaining invariant to fusion-preserving factors such as global scaling and sampling-induced layout changes, whose variations do not. Existing distributional distance metrics conflate these factors, leading to measures that are not informative of the true extent of fusion between data groups. We introduce Cross-Fusion Distance (CFD), a principled measure that isolates fusion-altering geometry while remaining robust to fusion-preserving variations, with linear computational complexity. We characterize the invariance and sensitivity properties of CFD theoretically and validate them in controlled synthetic experiments. For practical utility on real-world datasets with domain shift, CFD aligns more closely with downstream generalization degradation than commonly used alternatives. Overall, CFD provides a theoretically grounded and interpretable distance measure for representation learning.