Second order optimality on orthogonal Stiefel manifolds

TL;DR

This paper derives a matrix formula for the Hessian of functions on orthogonal Stiefel manifolds, introduces a local frame for computations, and applies these to second order optimality conditions and Newton algorithms.

Contribution

It provides a new explicit matrix formula for the Hessian on Stiefel manifolds and develops a local frame for practical computations, enhancing optimization analysis.

Findings

Revealed second order optimality conditions for Procrustes and Penrose problems.

Established necessary and sufficient conditions for local minima in Brockett problem.

Provided an explicit Newton algorithm for optimization on orthogonal Stiefel manifolds.

Abstract

The main tool to study a second order optimality problem is the Hessian operator associated to the cost function that defines the optimization problem. By regarding an orthogonal Stiefel manifold as a constraint manifold embedded in an Euclidean space we obtain a concise matrix formula for the Hessian of a cost function defined on such a manifold. We introduce an explicit local frame on an orthogonal Stiefel manifold in order to compute the components of the Hessian matrix of a cost function. We present some important properties of this frame. As applications we rediscover second order conditions of optimality for the Procrustes and the Penrose regression problems (previously found in the literature). For the Brockett problem we find necessary and sufficient conditions for a critical point to be a local minimum. Since many optimization problems are approached using numerical algorithms, we give an explicit description of the Newton algorithm on orthogonal Stiefel manifolds.