The EM Algorithm in Information Geometry

TL;DR

This thesis introduces the concepts of information geometry and the EM algorithm, illustrating their applications with Python implementations and exploring their relevance to deep learning.

Contribution

It provides an accessible overview of information geometry and the EM algorithm, including explicit calculations and a novel Python implementation for deep learning applications.

Findings

Explicit calculation of e and m projections

Python implementation of EM algorithm

Application to deep learning

Abstract

The purpose of this thesis is to convey the basic concepts of information geometry and its applications to non-specialists and those in applied fields, assuming only a first-year undergraduate background in calculus, linear algebra, and probability theory / statistics. We first begin with an introduction to the EM algorithm, providing a typical use case in Python, before moving to an overview of basic Riemannian geometry. We then introduce the core concepts of information geometry and the $em$ algorithm, with an explicit calculation of both the $e$ and $m$ projection, before closing with a discussion of an important application of this research to the field of deep learning, providing a novel implementation in Python.