A thorough study of Riemannian Newton's Method

TL;DR

This paper provides a comprehensive numerical analysis of Riemannian Newton's Method (RNM) on Grassmannian and Stiefel manifolds, demonstrating its advantages over classical methods in quantum chemistry energy minimization.

Contribution

It offers a detailed comparison between Riemannian and Euclidean Newton's methods, introduces a modified RNM for stability, and analyzes numerical issues on quotient manifolds.

Findings

RNM achieves higher convergence rates and fewer iterations.

RNM shows greater robustness to initial guesses.

Modified RNM is stable and performs comparably to RNM on quotient manifolds.

Abstract

This work presents a thorough numerical study of Riemannian Newton's Method (RNM) for optimization problems, with a focus on the Grassmannian and on the Stiefel manifold. We compare the Riemannian formulation of Newton's Method with its classical Euclidean counterpart based on Lagrange multipliers by applying both approaches to the important and challenging Hartree--Fock energy minimization problem from Quantum Chemistry. Experiments on a dataset of 125 molecules show that the Riemannian approaches achieve higher convergence rates, require fewer iterations, and exhibit greater robustness to the choice of initial guess. In this work we also analyze the numerical issues that arise from using Newton's Method on the total manifold when the cost function is defined on the quotient manifold. We investigate the performance of a modified RNM in which we ignore the small eigenvalues of the Hessian and the results indicate that this modified method is stable and performs on par with the RNM on the quotient manifold.