Generalized Forms of the Kraft Inequality for Finite-State Encoders

TL;DR

This paper extends the Kraft inequality to finite-state encoders by introducing a Kraft matrix and establishing spectral radius conditions, with special cases and extensions for side information and lossy compression.

Contribution

It introduces a Kraft matrix framework and spectral radius conditions as necessary criteria for lossless finite-state encoding, including special cases and extensions.

Findings

Eigenvalues of the Kraft matrix must not exceed unity in modulus.

Kraft sums are bounded by a constant for irreducible encoders.

Extensions include side information and lossy compression scenarios.

Abstract

We derive a few extended versions of the Kraft inequality for information lossless finite-state encoders. The main basic contribution is in defining a notion of a Kraft matrix and in establishing the fact that a necessary condition for information losslessness of a finite-state encoder is that none of the eigenvalues of this matrix have modulus larger than unity, or equivalently, the generalized Kraft inequality asserts that the spectral radius of the Kraft matrix cannot exceed one. For the important special case where the FS encoder is irreducible, we derive several equivalent forms of this inequality, which are based on well known formulas for spectral radius. It also turns out that in the irreducible case, Kraft sums are bounded by a constant, independent of the block length, and thus cannot grow even in any subexponential rate. Finally, two extensions are outlined - one concerns the case of side information available to both encoder and decoder, and the other is for lossy compression.