Computation over Tensor Stiefel Manifold: A Preliminary Study

TL;DR

This paper introduces the tensor Stiefel manifold, establishes its Riemannian structure, and derives fundamental geometric tools like tangent space, gradients, Hessians, and retractions for optimization on this manifold.

Contribution

It is the first to define and analyze the tensor Stiefel manifold, providing formulas for geometric operations crucial for tensor optimization algorithms.

Findings

Verified that the tensor Stiefel manifold is a Riemannian manifold.

Derived tangent space, Riemannian gradient, and Hessian formulas.

Presented various retraction methods for optimization on the manifold.

Abstract

Let $*$ denote the t-product between two third-order tensors. The purpose of this work is to study fundamental computation over the set $St(n,p,l):= \{\mathcal Q\in \mathbb R^{n\times p\times l} \mid \mathcal Q^{\top}* \mathcal Q = \mathcal I \}$, where $\mathcal Q$ is a third-order tensor of size $n\times p \times l$ and $\mathcal I$ ($n\geq p$) is the identity tensor. It is first verified that $St(n,p,l)$ endowed with the usual Frobenius norm forms a Riemannian manifold, which is termed as the (third-order) \emph{tensor Stiefel manifold} in this work. We then derive the tangent space, Riemannian gradient, and Riemannian Hessian on $St(n,p,l)$. In addition, formulas of various retractions based on t-QR, t-polar decomposition, Cayley transform, and t-exponential, as well as vector transports, are presented. It is expected that analogous to their matrix counterparts, the formulas derived in this study may serve as building blocks for analyzing optimization problems over the tensor Stiefel manifold and designing Riemannian algorithms for them.