Complexity and computation for the spectral norm and nuclear norm of order three tensors with one fixed dimension

TL;DR

This paper investigates the computational complexity of spectral and nuclear norms for third-order tensors with one fixed dimension, establishing polynomial-time computability and proposing approximation schemes.

Contribution

It demonstrates polynomial-time algorithms for these tensor norms when one dimension is fixed and introduces the first FPTAS for the nuclear norm of general asymmetric tensors.

Findings

Spectral norm computation is polynomial-time for fixed first dimension.

Nuclear norm computation is polynomial-time for fixed first dimension.

Proposed FPTAS effectively computes norms for small fixed dimensions and large other dimensions.

Abstract

The recent decade has witnessed a surge of research in modelling and computing from two-way data (matrices) to multiway data (tensors). However, there is a drastic phase transition for most tensor optimization problems when the order of a tensor increases from two (a matrix) to three: Most tensor problems are NP-hard while that for matrices are easy. It triggers a question on where exactly the transition occurs. The paper aims to study this kind of question for the spectral norm and the nuclear norm. Although computing the spectral norm for a general $\ell\times m\times n$ tensor is NP-hard, we show that it can be computed in polynomial time if $\ell$ is fixed. This is the same for the nuclear norm. While these polynomial-time methods are not implementable in practice, we propose fully polynomial-time approximation schemes (FPTAS) for the spectral norm based on spherical grids and for the nuclear norm with further help of duality theory and semidefinite optimization. Numerical experiments on simulated data show that our FPTAS can compute these tensor norms for small $\ell \le 6$ but large $m, n\ge50$. To the best of our knowledge, this is the first method that can compute the nuclear norm of general asymmetric tensors. Both our polynomial-time algorithms and FPTAS can be extended to higher-order tensors as well.