Practical applications of Set Shaping Theory in Huffman coding

TL;DR

This paper demonstrates a practical method to apply Set Shaping Theory in Huffman coding by replacing large correspondence tables with a functional transform, enabling real-world implementation and validation of the theory.

Contribution

The authors introduce a function-based approach to implement Set Shaping Theory in Huffman coding, overcoming the limitations of large tables for practical use.

Findings

The new method confirms theoretical predictions.

It enables application of Set Shaping Theory to larger A and N values.

Results show improved encoding efficiency.

Abstract

One of the biggest criticisms of the Set Shaping Theory is the lack of a practical application. This is due to the difficulty of its application. In fact, to apply this technique from an experimental point of view we must use a table that defines the correspondences between two sets. However, this approach is not usable in practice, because the table has A^N elements, with A number of symbols and N length of the message to be encoded. Consequently, these tables can be implemented in a program only when A and N have a low value. Unfortunately, in these cases, there are no compression algorithms with such efficiency as to detect the improvement introduced by this method. In this article, we use a function capable of performing the transform without using the correspondence table; this allows us to apply this theory to a wide range of values of A and N. The results obtained confirm the theoretical predictions.