On the minmax regret for statistical manifolds: the role of curvature

TL;DR

This paper explores how the curvature of statistical manifolds influences model complexity and regret, using Riemannian geometry to improve model selection and dimensionality reduction techniques.

Contribution

It introduces a curvature-based refinement of stochastic complexity and derives the minmax regret for statistical manifolds, with applications to PCA.

Findings

Scalar curvature significantly impacts model complexity measures.

Derived a sharper expression for stochastic complexity incorporating curvature.

Applied results to optimize dimensional reduction in PCA.

Abstract

Model complexity plays an essential role in its selection, namely, by choosing a model that fits the data and is also succinct. Two-part codes and the minimum description length have been successful in delivering procedures to single out the best models, avoiding overfitting. In this work, we pursue this approach and complement it by performing further assumptions in the parameter space. Concretely, we assume that the parameter space is a smooth manifold, and by using tools of Riemannian geometry, we derive a sharper expression than the standard one given by the stochastic complexity, where the scalar curvature of the Fisher information metric plays a dominant role. Furthermore, we derive the minmax regret for general statistical manifolds and apply our results to derive optimal dimensional reduction in the context of principal component analysis.