# GAIT: A Geometric Approach to Information Theory

## TL;DR

This paper introduces GAIT, a geometric information theory framework that incorporates symbol similarities into entropy measures, providing efficient divergence computations and versatile applications across generative modeling, image analysis, and empirical data approximation.

## Contribution

The paper presents a novel geometry-aware entropy and divergence framework that integrates symbol similarities, offering computational efficiency and broad applicability.

## Key findings

- Performance comparable to Wasserstein-based methods
- Closed-form divergence expression for efficiency
- Versatile applications demonstrated across domains

## Abstract

We advocate the use of a notion of entropy that reflects the relative abundances of the symbols in an alphabet, as well as the similarities between them. This concept was originally introduced in theoretical ecology to study the diversity of ecosystems. Based on this notion of entropy, we introduce geometry-aware counterparts for several concepts and theorems in information theory. Notably, our proposed divergence exhibits performance on par with state-of-the-art methods based on the Wasserstein distance, but enjoys a closed-form expression that can be computed efficiently. We demonstrate the versatility of our method via experiments on a broad range of domains: training generative models, computing image barycenters, approximating empirical measures and counting modes.