Minimization of Functions on Dually Flat Spaces Using Geodesic Descent Based on Dual Connections

TL;DR

This paper introduces geodesic-based optimization techniques on dually flat spaces, enabling efficient maximum likelihood estimation, with theoretical guarantees and practical advantages demonstrated through numerical experiments.

Contribution

It develops novel geodesic descent methods tailored for dually flat spaces, improving optimization efficiency in statistical models like exponential families.

Findings

Single-step convergence to MLE via m-geodesic updates

e-geodesic updates facilitate unconstrained optimization in complete spaces

Methods are validated through numerical experiments

Abstract

We propose geodesic-based optimization methods on dually flat spaces, where the geometric structure of the parameter manifold is closely related to the form of the objective function. A primary application is maximum likelihood estimation in statistical models, especially exponential families, whose model manifolds are dually flat. We show that an m-geodesic update, which directly optimizes the log-likelihood, can theoretically reach the maximum likelihood estimator in a single step. In contrast, an e-geodesic update has a practical advantage in cases where the parameter space is geodesically complete, allowing optimization without explicitly handling parameter constraints. We establish the theoretical properties of the proposed methods and validate their effectiveness through numerical experiments.