On Sharp Stochastic Zeroth Order Hessian Estimators over Riemannian Manifolds

TL;DR

This paper introduces new stochastic zeroth-order Hessian estimators for functions on Riemannian manifolds, providing the first bias bounds that incorporate manifold geometry, with empirical validation of their effectiveness.

Contribution

It presents the first bias bounds for Hessian estimators on Riemannian manifolds that explicitly depend on geometric properties, using only O(1) function evaluations.

Findings

Achieves bias bound of order O(γδ^2) depending on manifold geometry

Provides empirical evidence of estimator effectiveness

First to explicitly incorporate Riemannian geometry into Hessian estimation

Abstract

We study Hessian estimators for functions defined over an $n$-dimensional complete analytic Riemannian manifold. We introduce new stochastic zeroth-order Hessian estimators using $O (1)$ function evaluations. We show that, for an analytic real-valued function $f$, our estimator achieves a bias bound of order $ O \left( \gamma \delta^2 \right) $, where $ \gamma $ depends on both the Levi-Civita connection and function $f$, and $\delta$ is the finite difference step size. To the best of our knowledge, our results provide the first bias bound for Hessian estimators that explicitly depends on the geometry of the underlying Riemannian manifold. We also study downstream computations based on our Hessian estimators. The supremacy of our method is evidenced by empirical evaluations.