Information geometry of Bayes computations

TL;DR

This paper explores the application of Amari's Information Geometry to Bayesian computations, emphasizing the Fisher score and divergence measures within a nonparametric framework, providing a geometric perspective on probabilistic models.

Contribution

It introduces a nonparametric affine geometric framework for Bayesian computations based on classical Weyl's axioms, connecting Fisher scores and divergence measures.

Findings

Fisher's score aligns with affine velocity in the geometric model

The framework effectively handles Bayes and Kullback-Leibler divergence calculations

Provides a geometric interpretation of Bayesian inference processes

Abstract

Amari's Information Geometry is a dually affine formalism for parametric probability models. The literature proposes various nonparametric functional versions. Our approach uses classical Weyl's axioms so that the affine velocity of a one-parameter statistical model equals the classical Fisher's score. In the present note, we first offer a concise review of the notion of a statistical bundle as a set of couples of probability densities and Fisher's scores. Then, we show how the nonparametric dually affine setup deals with the basic Bayes and Kullback-Leibler divergence computations.