A Light Discussion and Derivation of Entropy

TL;DR

This paper clarifies the foundational assumptions and provides a straightforward derivation of entropy, demystifying its often mysterious expression in information theory and thermodynamics.

Contribution

It offers a clear discussion of the assumptions and a simple derivation of entropy, enhancing understanding of this fundamental concept.

Findings

Clarifies the assumptions behind entropy

Provides a simple derivation of entropy

Demystifies the expression of entropy

Abstract

The expression for entropy sometimes appears mysterious - as it often is asserted without justification. This short manuscript contains a discussion of the underlying assumptions behind entropy as well as simple derivation of this ubiquitous quantity.