TL;DR
This paper clarifies the mathematical definition of information, distinguishes it from entropy, and emphasizes its importance in prediction across various disciplines like physics and biology.
Contribution
It provides a clear, rigorous explanation of entropy and information, highlighting their relationship and significance beyond engineering.
Findings
Information is rigorously defined and distinguished from entropy.
Understanding information as prediction is crucial across multiple scientific fields.
The paper offers an intuitive yet precise explanation of core concepts.
Abstract
Information is a precise concept that can be defined mathematically, but its relationship to what we call "knowledge" is not always made clear. Furthermore, the concepts "entropy" and "information", while deeply related, are distinct and must be used with care, something that is not always achieved in the literature. In this elementary introduction, the concepts of entropy and information are laid out one by one, explained intuitively, but defined rigorously. I argue that a proper understanding of information in terms of prediction is key to a number of disciplines beyond engineering, such as physics and biology.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.