On the rank-one approximation of symmetric tensors

TL;DR

This paper investigates the symmetric rank-one approximation of symmetric tensors, providing bounds on accuracy, analyzing eigenvector relations, and improving iterative methods for better convergence and stability in tensor decomposition.

Contribution

It offers new perturbation bounds, insights into eigenvector relations, and enhanced convergence analysis for the SS-HOPM algorithm in symmetric tensor approximation.

Findings

Bounds on approximation accuracy in noisy conditions

Eigenvector relations for large eigenvalues

Improved convergence and stability of SS-HOPM

Abstract

The problem of symmetric rank-one approximation of symmetric tensors is important in Independent Components Analysis, also known as Blind Source Separation, as well as polynomial optimization. We analyze the symmetric rank-one approximation problem for symmetric tensors and derive several perturbation results. Given a symmetric rank-one tensor obscured by noise, we provide bounds on the accuracy of the best symmetric rank-one approximation for recovering the original rank-one structure, and we show that any eigenvector with sufficiently large eigenvalue is related to the rank-one structure as well. Further, we show that for high-dimensional symmetric approximately-rank-one tensors, the generalized Rayleigh quotient is mostly close to zero, so the best symmetric rank-one approximation corresponds to a prominent global extreme value. We show that each iteration of the Shifted Symmetric Higher Order Power Method (SS-HOPM), when applied to a rank-one symmetric tensor, moves towards the principal eigenvector for any input and shift parameter, under mild conditions. Finally, we explore the best choice of shift parameter for SS-HOPM to recover the principal eigenvector. We show that SS-HOPM is guaranteed to converge to an eigenvector of an approximately rank-one even-mode tensor for a wider choice of shift parameter than it is for a general symmetric tensor. We also show that the principal eigenvector is a stable fixed point of the SS-HOPM iteration for a wide range of shift parameters; together with a numerical experiment, these results lead to a non-obvious recommendation for shift parameter for the symmetric rank-one approximation problem.