GSVD for Geometry-Grounded Dataset Comparison: An Alignment Angle Is All You Need

TL;DR

This paper introduces an angle-based diagnostic score derived from the generalized singular value decomposition (GSVD) to compare datasets in a geometry-grounded manner, providing interpretable insights into data relations.

Contribution

It revisits the classical GSVD primitive, operationalizes it for dataset comparison, and introduces an interpretable angle score for geometric diagnostics.

Findings

The angle score effectively quantifies dataset similarities and differences.

The method provides interpretable geometric diagnostics for data analysis.

Application to MNIST demonstrates practical utility.

Abstract

Geometry-grounded learning asks models to respect structure in the problem domain rather than treating observations as arbitrary vectors. Motivated by this view, we revisit a classical but underused primitive for comparing datasets: linear relations between two data matrices, expressed via the co-span constraint $Ax = By = z$ in a shared ambient space. To operationalize this comparison, we use the generalized singular value decomposition (GSVD) as a joint coordinate system for two subspaces. In particular, we exploit the GSVD form $A = HCU$, $B = HSV$ with $C^{\top}C + S^{\top}S = I$, which separates shared versus dataset-specific directions through the diagonal structure of $(C, S)$. From these factors we derive an interpretable *angle score* $\theta(z) \in [0, \pi/2]$ for a sample $z$, quantifying whether z is explained relatively more by $A$, more by $B$, or comparably by both. The primary role of $\theta(z)$ is as a *per-sample geometric diagnostic*. We illustrate the behavior of the score on MNIST through angle distributions and representative GSVD directions. A binary classifier derived from $\theta(z)$ is presented as an illustrative application of the score as an interpretable diagnostic tool.