Fast and Accurate CP-HIFI Tensor Decompositions: Exploiting Kronecker Structure

TL;DR

This paper introduces fast algorithms for CP-HIFI tensor decompositions that exploit Kronecker structure, significantly reducing computational time for large-scale problems involving scattered data and infinite-dimensional factors.

Contribution

The paper develops novel algorithms leveraging Kronecker structure and preconditioning to efficiently compute CP-HIFI tensor decompositions, outperforming existing methods in speed and scalability.

Findings

Speed up to 500x over naive methods

Reduced computational complexity through structure exploitation

Effective handling of scattered and infinite-dimensional data

Abstract

Tensor decompositions are a fundamental tool in scientific computing and data analysis. In many applications -- such as simulation data on irregular grids, surrogate modeling for parameterized PDEs, or spectroscopic measurements -- the data has both discrete and continuous structure, and may only be observed at scattered sample points. The CP-HIFI (hybrid infinite-finite) decomposition generalizes the Canonical Polyadic (CP) tensor decomposition to settings where some factors are finite-dimensional vectors and others are functions drawn from infinite-dimensional spaces. The decomposition can be applied to a fully observed tensor (aligned) or, when only scattered observations are available, to a sparsely sampled tensor (unaligned). Current methods compute CP-HIFI factors by solving a sequence of dense linear systems arising from regularized least-squares problems to fit reproducing Kernel Hilbert space (RKHS) representations to the data, but these direct solves become computationally prohibitive as problem size grows. We propose new algorithms that achieve the same accuracy while being orders of magnitude faster. For aligned tensors, we exploit the Kronecker structure of the system to efficiently compute its eigendecomposition without ever forming the full system, reducing the solve to independent scalar equations. For unaligned tensors, we introduce a preconditioned conjugate gradient method, exploiting the problem's structure for fast matrix-vector products and efficient preconditioning. In our experiments, the proposed methods speed up the solution up to 500x compared to the prior naive direct methods, in line with the reduction in the theoretical computational complexity.