An approach to Fisher-Rao metric for infinite dimensional non-parametric information geometry

TL;DR

This paper introduces a novel geometric framework for infinite-dimensional non-parametric information geometry, making the Fisher-Rao metric computationally feasible and linking it to statistical limits and intrinsic data dimensionality.

Contribution

It develops a finite-dimensional Covariate Fisher Information Matrix via orthogonal decomposition, enabling practical analysis of infinite-dimensional models and connecting geometry with statistical efficiency.

Findings

Derived the Covariate Fisher Information Matrix (cFIM) as a finite-dimensional representation.

Proved the Trace Theorem linking G-entropy to the trace of cFIM.

Established the connection between cFIM and the curvature of KL-divergence, leading to a Covariate Cramér-Rao Lower Bound.

Abstract

Being infinite dimensional, non-parametric information geometry has long faced an "intractability barrier" due to the fact that the Fisher-Rao metric is now a functional incurring difficulties in defining its inverse. This paper introduces a novel framework to resolve the intractability with an Orthogonal Decomposition of the Tangent Space ($T_fM = S \oplus S^{\perp}$), where $S$ represents an observable covariate subspace. Through the decomposition, we derive the Covariate Fisher Information Matrix (cFIM), denoted as ${\bf G}_f$, which is a finite-dimensional and computable representative of information extractable from the manifold's geometry. Significantly, by proving the Trace Theorem: $H_G(f) = \text{Tr}({\bf G}_f)$, we establish a rigorous foundation for the G-entropy previously introduced by us, thereby identifying it as a fundamental geometric invariant representing the total explainable statistical information captured by the probability distribution associated with a model. Furthermore, we establish a link between ${\bf G}_f$ and the second derivative (i.e. the curvature) of the KL-divergence, leading to the notion of Covariate Cram\'er-Rao Lower Bound(CRLB). We demonstrate that ${\bf G}_f$ is congruent to the Efficient Fisher Information Matrix, thereby providing fundamental limits of variance for semi-parametric estimators. Finally, we apply our geometric framework to the Manifold Hypothesis, lifting the latter from a heuristic assumption into a testable condition of rank-deficiency within the cFIM. By defining the Information Capture Ratio, we provide a rigorous method for estimating intrinsic dimensionality in high-dimensional data. In short, our work bridges the gap between abstract information geometry and the demand of explainable AI, by providing a tractable path for assessing the statistical coverage and the efficiency of non-parametric models.