# Entropy and Compression: A simple proof of an inequality of Khinchin-Ornstein-Shields

## TL;DR

This paper provides a simple proof of the inequality relating entropy and data compression, including a pointwise version and an elementary proof of Khinchin's inequality, with historical context.

## Contribution

It offers a straightforward proof of the entropy-compression inequality and Khinchin's inequality, enhancing understanding and educational value in information theory.

## Key findings

- Proof of a pointwise entropy inequality
- Elementary proof of Khinchin's inequality
- Historical and technical insights on the inequalities

## Abstract

This paper concerns the folklore statement that ``entropy is a lower bound for compression''. More precisely we derive from the entropy theorem a simple proof of a pointwise inequality firstly stated by Ornstein and Shields and which is the almost-sure version of an average inequality firstly stated by Khinchin in 1953. We further give an elementary proof of original Khinchin inequality that can be used as an exercise for Information Theory students and we conclude by giving historical and technical notes of such inequality.