Quasi-Newton methods on Grassmannians and multilinear approximations of tensors

TL;DR

This paper develops quasi-Newton and limited memory quasi-Newton methods tailored for optimization on Grassmannians, enabling efficient multilinear tensor approximations with proven optimality and robustness.

Contribution

It introduces BFGS and L-BFGS updates on Grassmannians, demonstrating their optimality and effectiveness for large, high-order tensor approximation problems.

Findings

Fast, robust, and accurate tensor approximation algorithms.

Effective on large-dimensional and high-order tensors.

Theoretical guarantees for BFGS updates on Grassmannians.

Abstract

In this paper we proposed quasi-Newton and limited memory quasi-Newton methods for objective functions defined on Grassmannians or a product of Grassmannians. Specifically we defined BFGS and L-BFGS updates in local and global coordinates on Grassmannians or a product of these. We proved that, when local coordinates are used, our BFGS updates on Grassmannians share the same optimality property as the usual BFGS updates on Euclidean spaces. When applied to the best multilinear rank approximation problem for general and symmetric tensors, our approach yields fast, robust, and accurate algorithms that exploit the special Grassmannian structure of the respective problems, and which work on tensors of large dimensions and arbitrarily high order. Extensive numerical experiments are included to substantiate our claims.