No-Rank Tensor Decomposition Using Metric Learning

TL;DR

This paper introduces a novel no-rank tensor decomposition method based on metric learning, replacing traditional reconstruction objectives with similarity-driven optimization to produce semantically meaningful embeddings.

Contribution

It presents a new tensor decomposition framework using metric learning with triplet loss and regularization, providing theoretical guarantees and effective performance across diverse datasets.

Findings

Produces embeddings reflecting semantic and physical relationships

Achieves competitive clustering performance compared to classical and deep learning methods

Effective in small-data regimes where transformers are less feasible

Abstract

Tensor decomposition of high-dimensional data often struggles to capture semantically or physically meaningful structures, particularly when relying on reconstruction objectives and fixed-rank constraints. We introduce a no-rank tensor decomposition framework based on metric learning, which replaces reconstruction objectives with a similarity-driven optimization. By combining a triplet loss with diversity and uniformity regularization, the method learns embeddings where distances naturally reflect semantic and physical relationships, supported by theoretical guarantees on convergence and metric properties. We evaluate the approach on diverse datasets, including face recognition (LFW, Olivetti), brain connectivity (ABIDE), and simulated physical systems (galaxies, crystals). In comprehensive comparisons against classical methods (PCA, t-SNE, UMAP), tensor decompositions (CP, Tucker, t-SVD), and deep learning models (VAE, DEC, transformer based embeddings), our method produces embeddings that preserve physically and semantically relevant relationships and achieve competitive clustering performance. While transformers often excel in predictive accuracy on large datasets, our method provides interpretable embeddings and remains effective in small-data regimes where transformer training may be infeasible. This work establishes metric learning as a principled paradigm for tensor analysis, emphasizing physical interpretability and semantic relevance over pixel-level reconstruction, and offering an efficient and robust alternative in data-scarce scientific domains.