Information Geometry of the Probability Simplex: A Short Course

TL;DR

This paper provides an accessible introduction to the information geometry of the probability simplex, comparing non-parametric methods with classical statistical physics approaches, suitable for learners and researchers new to the field.

Contribution

It offers an almost self-contained, elementary overview of the information geometry of the probability simplex, bridging classical statistical physics and modern IG formalism.

Findings

Clarifies the geometric structure of the probability simplex.

Connects IG concepts with statistical physics methods.

Provides educational material for newcomers to the field.

Abstract

This set of notes is intended for a short course aiming to provide an (almost) self-contained and (almost) elementary introduction to the topic of Information Geometry (IG) of the probability simplex. Such a course can be considered an introduction to the original monograph by Amari and Nagaoka (2000), and to the recent monographs by Amari (2016} and by Ay et al. (2017). The focus is on a non-parametric approach, that is, I consider the geometry of the full probability simplex and compare the IG formalism with what is classically done in Statistical Physics.