TL;DR
This paper derives an upper bound on the entropy of real documents based on their power law distribution parameters, connecting entropy estimation with power law characteristics.
Contribution
It introduces a novel method to estimate entropy bounds from power law parameters, linking statistical distribution properties with information theory.
Findings
Entropy can be bounded using power law parameters.
The approach provides a new perspective on entropy estimation.
The method applies to real document data.
Abstract
Shannon(1951) and Yavuz(1974) estimated the entropy of real documents. This note drives an upper bound of entropy from the parameter of the power law.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Text Analysis Techniques