Oblivious subspace embeddings for compressed Tucker decompositions

TL;DR

This paper introduces Johnson-Lindenstrauss type guarantees for Tucker tensor decompositions using oblivious random embeddings, enabling efficient dimension reduction with minimal error increase, demonstrated on large datasets.

Contribution

It provides the first JL-type guarantees for Tucker decompositions with oblivious embeddings and implements a practical HOOI algorithm that improves efficiency on large tensors.

Findings

Dimension reduction with ≤5% error increase on datasets

50%-60% lower computation time with 50% dimension reduction

Outperforms traditional HOSVD and TensorSketch on large tensors

Abstract

Emphasis in the tensor literature on random embeddings (tools for low-distortion dimension reduction) for the canonical polyadic (CP) tensor decomposition has left analogous results for the more expressive Tucker decomposition comparatively lacking. This work establishes general Johnson-Lindenstrauss (JL) type guarantees for the estimation of Tucker decompositions when an oblivious random embedding is applied along each mode. When these embeddings are drawn from a JL-optimal family, the decomposition can be estimated within $\varepsilon$ relative error under restrictions on the embedding dimension that are in line with recent CP results. We implement a higher-order orthogonal iteration (HOOI) decomposition algorithm with random embeddings to demonstrate the practical benefits of this approach and its potential to improve the accessibility of otherwise prohibitive tensor analyses. On moderately large face image and fMRI neuroimaging datasets, empirical results show that substantial dimension reduction is possible with minimal increase in reconstruction error relative to traditional HOOI ($\leq$5% larger error, 50%-60% lower computation time for large models with 50% dimension reduction along each mode). Especially for large tensors, our method outperforms traditional higher-order singular value decomposition (HOSVD) and recently proposed TensorSketch methods.