Learning Physics from Data: a Thermodynamic Interpretation

TL;DR

This paper presents a thermodynamic perspective on machine learning, viewing the learning process as a dissipative evolution driven by entropy and connecting it to thermodynamic and geometric principles.

Contribution

It introduces a thermodynamic framework for understanding learning as entropy-driven dissipation and links reversible Hamiltonian dynamics to Poisson geometry in machine learning.

Findings

Learning as entropy-driven dissipation from detailed to less detailed descriptions

Reversible Hamiltonian evolution corresponds to propagation within levels of description

Application to free surface liquids and rigid body rotation

Abstract

Experimental data bases are typically very large and high dimensional. To learn from them requires to recognize important features (a pattern), often present at scales different to that of the recorded data. Following the experience collected in statistical mechanics and thermodynamics, the process of recognizing the pattern (the learning process) can be seen as a dissipative time evolution driven by entropy from a detailed level of description to less detailed. This is the way thermodynamics enters machine learning. On the other hand, reversible (typically Hamiltonian) evolution is propagation within the levels of description, that is also to be recognized. This is how Poisson geometry enters machine learning. Learning to handle free surface liquids and damped rigid body rotation serves as an illustration.