Cramer-Rao-Induced Bounds for CANDECOMP/PARAFAC tensor decomposition

TL;DR

This paper derives a computationally efficient Cramer-Rao lower bound for the accuracy of tensor factorization in CANDECOMP/PARAFAC decompositions, applicable to noisy data and useful for algorithm performance assessment.

Contribution

It introduces a new, less computationally intensive CRLB expression for tensor factor estimation, including special cases and applications to algorithm stability and design.

Findings

Derived a CRLB with O(NR^6) complexity, less than previous algorithms.

Provided explicit bounds for rank 1 and rank 2 tensors.

Enabled performance prediction and stability analysis of tensor decomposition algorithms.

Abstract

This paper presents a Cramer-Rao lower bound (CRLB) on the variance of unbiased estimates of factor matrices in Canonical Polyadic (CP) or CANDECOMP/PARAFAC (CP) decompositions of a tensor from noisy observations, (i.e., the tensor plus a random Gaussian i.i.d. tensor). A novel expression is derived for a bound on the mean square angular error of factors along a selected dimension of a tensor of an arbitrary dimension. The expression needs less operations for computing the bound, O(NR^6), than the best existing state-of-the art algorithm, O(N^3R^6) operations, where N and R are the tensor order and the tensor rank. Insightful expressions are derived for tensors of rank 1 and rank 2 of arbitrary dimension and for tensors of arbitrary dimension and rank, where two factor matrices have orthogonal columns. The results can be used as a gauge of performance of different approximate CP decomposition algorithms, prediction of their accuracy, and for checking stability of a given decomposition of a tensor (condition whether the CRLB is finite or not). A novel expression is derived for a Hessian matrix needed in popular damped Gauss-Newton method for solving the CP decomposition of tensors with missing elements. Beside computing the CRLB for these tensors the expression may serve for design of damped Gauss-Newton algorithm for the decomposition.