Huffman Coding as a Non-linear Dynamical System

TL;DR

This paper models stochastic sources as chaotic dynamical systems and derives a binary coding algorithm that re-discovers Huffman coding, linking data compression to dynamical systems theory.

Contribution

It introduces a novel dynamical systems perspective to source coding, showing Huffman coding as a minimal redundancy approximation of chaotic models.

Findings

Lyapunov exponent of GLS equals source entropy

Derived binary coding algorithm with minimal redundancy

Re-discovery of Huffman coding through dynamical systems approach

Abstract

In this paper, source coding or data compression is viewed as a measurement problem. Given a measurement device with fewer states than the observable of a stochastic source, how can one capture the essential information? We propose modeling stochastic sources as piecewise linear discrete chaotic dynamical systems known as Generalized Lur\"{o}th Series (GLS) which dates back to Georg Cantor's work in 1869. The Lyapunov exponent of GLS is equal to the Shannon's entropy of the source (up to a constant of proportionality). By successively approximating the source with GLS having fewer states (with the closest Lyapunov exponent), we derive a binary coding algorithm which exhibits minimum redundancy (the least average codeword length with integer codeword lengths). This turns out to be a re-discovery of Huffman coding, the popular lossless compression algorithm used in the JPEG international standard for still image compression.