TL;DR
This paper critically examines the definition and calculation of conditional entropy, proposing corrections and challenging the traditional view that information always reduces uncertainty.
Contribution
It provides a corrected formula for conditional entropy and demonstrates that information may not always decrease uncertainty, revising foundational assumptions.
Findings
Corrected the formula for conditional entropy
Showed that conditional entropy can increase in certain cases
Challenged the traditional view of information always reducing uncertainty
Abstract
The problems of conditional entropy's definition and the formula to compute conditional entropy are analyzed from various perspectives, and the corrected computing formula is presented. Examples are given to prove the conclusion that conditional entropy never be increased is not absolute, thus the representation that information is to decrease uncertainty in the definition of information is not absolutely correct.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Thermodynamics and Statistical Mechanics · Matrix Theory and Algorithms