A Scalable Shannon Entropy Estimator
Priyanka Golia, Brendan Juba, Kuldeep S. Meel

TL;DR
This paper introduces a highly efficient algorithm for estimating Shannon entropy using a conditional sampling oracle, significantly improving query complexity and practical performance over previous methods, especially for small entropy regimes.
Contribution
The authors present a new multiplicative approximation algorithm for Shannon entropy that requires fewer queries and has explicit constants, advancing the state-of-the-art in entropy estimation with conditional samples.
Findings
Achieves $(1+psilon)$-approximation with $rac{m}{psilon^2}lograc{1}{elta}$ queries.
Outperforms previous polynomial-query algorithms in practice.
Provides practical improvements for entropy estimation in security applications.
Abstract
We revisit the well-studied problem of estimating the Shannon entropy of a probability distribution, now given access to a probability-revealing conditional sampling oracle. In this model, the oracle takes as input the representation of a set and returns a sample from the distribution obtained by conditioning on , together with the probability of that sample in the distribution. Our work is motivated by applications of such algorithms in Quantitative Information Flow analysis (QIF) in programming-language-based security. Here, information-theoretic quantities capture the effort required on the part of an adversary to obtain access to confidential information. These applications demand accurate measurements when the entropy is small. Existing algorithms that do not use conditional samples require a number of queries that scale inversely with the entropy, which is unacceptable in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Malware Detection Techniques · Internet Traffic Analysis and Secure E-voting · Network Security and Intrusion Detection
