Foundations of Riemannian Geometry for Riemannian Optimization: A Monograph with Detailed Derivations

TL;DR

This monograph offers a detailed, step-by-step derivation of Riemannian geometric concepts tailored for optimization on matrix manifolds, bridging theory and practical implementation.

Contribution

It provides explicit coordinate-level derivations and formulas for Riemannian structures, gradient, Hessian, and exponential maps on key matrix manifolds for optimization.

Findings

Explicit formulas for Riemannian gradient and Hessian on matrix manifolds

Step-by-step derivations suitable for implementation

Specialized formulas for Stiefel, Grassmann, and SPD manifolds

Abstract

Riemannian geometry provides the fundamental framework for optimization on nonlinear spaces such as matrix manifolds, which arise in machine learning, signal processing, and robotics. While the underlying theory is classical, existing literature often presents results at a high level of abstraction, omitting the detailed coordinate-level derivations required for implementation and algorithm development. This work provides a self-contained and rigorous treatment of the foundations of Riemannian geometry, with a focus on explicit derivations tailored to Riemannian optimization. We systematically develop the key geometric structures -- including tangent and cotangent spaces, tensor calculus, metric tensors, Levi-Civita connections, curvature, and geodesics -- emphasizing step-by-step derivations in coordinates and matrix form. Building on these foundations, we derive the Riemannian gradient, Hessian, exponential map, and retraction in a form suitable for numerical computation. We further specialize these constructions to important matrix manifolds, including the Stiefel, Grassmann, and SPD (Symmetric Positive Definite) manifolds, providing explicit formulas widely used in optimization and geometric machine learning. This monograph develops a unified and implementation-oriented treatment of Riemannian geometry for optimization on manifolds. Its main contribution is the systematic organization and detailed derivation of classical geometric constructions in forms directly usable for algorithm design and numerical implementation. By connecting coordinate-level differential geometry with matrix-manifold formulas, the monograph bridges the gap between abstract theory and practical computation, and provides a reference for researchers and practitioners working in Riemannian optimization and related fields.