Rooted Trees with Probabilities Revisited

TL;DR

This paper revisits rooted trees with probabilities, introducing the differential LANSIT to analyze divergence and entropy, with applications to data compression, distribution matching, and approximation of product distributions.

Contribution

It introduces the differential LANSIT, extending existing theorems to analyze normalized functionals and divergence in rooted trees with probabilities.

Findings

Re-states the Leaf-Average Node-Sum Interchange Theorem (LANSIT) and its applications.

Derives the differential LANSIT for normalized functionals.

Formulates Pinsker's inequality for rooted trees with probabilities.

Abstract

Rooted trees with probabilities are convenient to represent a class of random processes with memory. They allow to describe and analyze variable length codes for data compression and distribution matching. In this work, the Leaf-Average Node-Sum Interchange Theorem (LANSIT) and the well-known applications to path length and leaf entropy are re-stated. The LANSIT is then applied to informational divergence. Next, the differential LANSIT is derived, which allows to write normalized functionals of leaf distributions as an average of functionals of branching distributions. Joint distributions of random variables and the corresponding conditional distributions are special cases of leaf distributions and branching distributions. Using the differential LANSIT, Pinsker's inequality is formulated for rooted trees with probabilities, with an application to the approximation of product distributions. In particular, it is shown that if the normalized informational divergence of a distribution and a product distribution approaches zero, then the entropy rate approaches the entropy rate of the product distribution.