Entropy, Derivation Operators and Huffman Trees

TL;DR

This paper develops a theoretical framework connecting entropy, derivation operators, and Huffman trees, providing a categorical perspective on how entropy can be operationalized and derived within binary tree structures.

Contribution

It introduces a novel theory that categorifies entropy using binary trees on multisets, linking derivation properties to Huffman trees.

Findings

Operationalizes entropy through Huffman trees.

Shows how derivation properties extend to joint distributions.

Provides a categorical perspective on entropy in binary trees.

Abstract

We build a theory of binary trees on finite multisets that categorifies, or operationalizes, the entropy of a finite probability distribution. Multisets operationalize probabilities as the event outcomes of an experiment. Huffman trees operationalize the entropy of the distribution of these events. We show how the derivation property of the entropy of a joint distribution lifts to Huffman trees.