Varentropy Decreases Under the Polar Transform

TL;DR

This paper investigates how the variance of entropy, called varentropy, evolves during polar transformations of binary data elements, showing it decreases and approaches zero asymptotically, which has implications for information theory.

Contribution

It proves that varentropy decreases under polar transforms and extends this to higher orders, demonstrating asymptotic convergence to zero for i.i.d. inputs.

Findings

Varentropy decreases under the polar transform.

Sum of output varentropies is less than or equal to input sum.

Varentropy approaches zero asymptotically for i.i.d. inputs.

Abstract

We consider the evolution of variance of entropy (varentropy) in the course of a polar transform operation on binary data elements (BDEs). A BDE is a pair $(X,Y)$ consisting of a binary random variable $X$ and an arbitrary side information random variable $Y$. The varentropy of $(X,Y)$ is defined as the variance of the random variable $-\log p_{X|Y}(X|Y)$. A polar transform of order two is a certain mapping that takes two independent BDEs and produces two new BDEs that are correlated with each other. It is shown that the sum of the varentropies at the output of the polar transform is less than or equal to the sum of the varentropies at the input, with equality if and only if at least one of the inputs has zero varentropy. This result is extended to polar transforms of higher orders and it is shown that the varentropy decreases to zero asymptotically when the BDEs at the input are independent and identially distributed.