Thermodynamics of learning physical phenomena

TL;DR

This paper explores how thermodynamics offers valuable insights and a useful inductive bias for improving machine learning processes, emphasizing the importance of scale, variable selection, and learning techniques.

Contribution

It provides a comprehensive review of thermodynamics' role in understanding and enhancing machine learning, highlighting new perspectives on physical phenomena modeling.

Findings

Thermodynamics can serve as an inductive bias in machine learning.

Scale and variable choice significantly influence learning outcomes.

Different techniques impact the effectiveness of thermodynamics-inspired learning.

Abstract

Thermodynamics could be seen as an expression of physics at a high epistemic level. As such, its potential as an inductive bias to help machine learning procedures attain accurate and credible predictions has been recently realized in many fields. We review how thermodynamics provides helpful insights in the learning process. At the same time, we study the influence of aspects such as the scale at which a given phenomenon is to be described, the choice of relevant variables for this description or the different techniques available for the learning process.