What entropy really is : the contribution of information theory

TL;DR

This paper clarifies the concept of entropy by emphasizing its subjective nature and demonstrates how information theory offers a more consistent and intuitive understanding, addressing common misconceptions and paradoxes.

Contribution

It provides a didactic analysis of entropy's foundations, highlighting how information theory can introduce entropy more coherently and simply.

Findings

Entropy's subjectivity is central to understanding.

Information theory offers a consistent framework for entropy.

The paper clarifies misconceptions and paradoxes related to entropy.

Abstract

Even today, the concept of entropy is perceived by many as quite obscure. The main difficulty is analyzed as being fundamentally due to the subjectivity and anthropocentrism of the concept that prevent us to have a sufficient distance to embrace it. However, it is pointed out that the lack of coherence of certain presentations or certain preconceived ideas do not help. They are of three kinds : 1) axiomatic thermodynamics; 2) inconsistent solutions of certain paradoxes; 3) reluctance of physicists to the simplification provided by information theory. The purpose of this paper is to examine these points in a didactic way by paying attention to the structure of the theory, what are the foundations, how ideas articulate, with a peculiar focus on their consistency and economy. It is shown how entropy can be introduced in a more consistent and economical manner with the help of information theory, which from the start takes into account its subjective nature, finally allowing a more intuitive understanding.