Information without rolling dice
Massimo Franceschetti, Taehyung J. Lim

TL;DR
This paper investigates deterministic measures of information capacity and entropy for bandlimited signals, introducing new bounds and concepts that unify stochastic and deterministic perspectives in information theory.
Contribution
It introduces the $(psilon,elta)$-capacity, extends Kolmogorov $psilon$-capacity, and provides new bounds and asymptotic expressions for capacity and entropy of bandlimited signals.
Findings
Capacity and entropy grow linearly with degrees of freedom.
Capacity and entropy grow logarithmically with SNR.
New bounds on Kolmogorov $psilon$-capacity and entropy.
Abstract
The deterministic notions of capacity and entropy are studied in the context of communication and storage of information using square-integrable, bandlimited signals subject to perturbation. The -capacity, that extends the Kolmogorov -capacity to packing sets of overlap at most , is introduced and compared to the Shannon capacity. The functional form of the results indicates that in both Kolmogorov and Shannon's settings, capacity and entropy grow linearly with the number of degrees of freedom, but only logarithmically with the signal to noise ratio. This basic insight transcends the details of the stochastic or deterministic description of the information-theoretic model. For , the analysis leads to new bounds on the Kolmogorov -capacity, and to a tight asymptotic expression of the Kolmogorov -entropy of bandlimited…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputability, Logic, AI Algorithms · Wireless Communication Security Techniques · Chaos-based Image/Signal Encryption
