Regularized Orthogonal Tensor Decompositions for Multi-Relational Learning

TL;DR

This paper introduces a scalable tensor decomposition method with regularization for multi-relational learning, addressing computational efficiency and rank sensitivity issues, and demonstrates its effectiveness through extensive experiments.

Contribution

It proposes a novel regularized orthogonal iteration decomposition method with convergence guarantees, reducing computational cost and improving robustness in tensor analytics.

Findings

Efficient algorithms with convergence guarantees for tensor decomposition.

Effective handling of incomplete and full tensor data in multi-relational learning.

Validated performance on real and synthetic datasets, even with limited observations.

Abstract

Multi-relational learning has received lots of attention from researchers in various research communities. Most existing methods either suffer from superlinear per-iteration cost, or are sensitive to the given ranks. To address both issues, we propose a scalable core tensor trace norm Regularized Orthogonal Iteration Decomposition (ROID) method for full or incomplete tensor analytics, which can be generalized as a graph Laplacian regularized version by using auxiliary information or a sparse higher-order orthogonal iteration (SHOOI) version. We first induce the equivalence relation of the Schatten p-norm (0<p<\infty) of a low multi-linear rank tensor and its core tensor. Then we achieve a much smaller matrix trace norm minimization problem. Finally, we develop two efficient augmented Lagrange multiplier algorithms to solve our problems with convergence guarantees. Extensive experiments using both real and synthetic datasets, even though with only a few observations, verified both the efficiency and effectiveness of our methods.