Jaynes & Shannon's Constrained Ignorance and Surprise
Thomas Cailleteau

TL;DR
This paper presents a variational approach to deriving Shannon's entropy based on constraints, inspired by Jaynes's work, aiming to establish a consistent formalism where the maximum entropy principle emerges naturally.
Contribution
It introduces a novel variational framework for deriving Shannon's entropy from ignorance and constraints, expanding the theoretical understanding of entropy in physics.
Findings
Derivation of Shannon's entropy from a variational principle.
Comparison of biased and unbiased approaches to entropy.
Potential applications in theoretical and applied physics.
Abstract
In this simple article, with possible applications in theoretical and applied physics, we suggest an original way to derive the expression of Shannon's entropy from a purely variational approach,using constraints. Based on the work of Edwin T. Jaynes, our results are not fundamentally new but the context in which they are derived might, however, lead to a remarkably consistent formalism,where the maximum entropy principle appears naturally. After having given a general definition of "ignorance" in this framework, we derive the somehow general expected expression for the entropy using two approaches. In the first, one is biased and has a vague idea of the shape of the entropy function. In the second, we consider the general case, where nothing is a priori known. The merits of both ways of thinking are compared.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEpistemology, Ethics, and Metaphysics
