From Kinetic Theory to AI: a Rediscovery of High-Dimensional Divergences and Their Properties

TL;DR

This paper reviews divergence measures from kinetic theory, emphasizing their theoretical properties and potential applications in machine learning and AI, highlighting the importance of choosing appropriate measures for model performance.

Contribution

It provides a comparative analysis of divergence measures rooted in kinetic theory, connecting their theoretical foundations to applications in AI and machine learning.

Findings

Highlights the theoretical properties of high-dimensional divergences

Explores applications of kinetic theory divergences in AI

Provides a comparative review of divergence measures

Abstract

Selecting an appropriate divergence measure is a critical aspect of machine learning, as it directly impacts model performance. Among the most widely used, we find the Kullback-Leibler (KL) divergence, originally introduced in kinetic theory as a measure of relative entropy between probability distributions. Just as in machine learning, the ability to quantify the proximity of probability distributions plays a central role in kinetic theory. In this paper, we present a comparative review of divergence measures rooted in kinetic theory, highlighting their theoretical foundations and exploring their potential applications in machine learning and artificial intelligence.