Higher Order Context Transformations

TL;DR

This paper presents a new algorithm for higher order context transformations that significantly reduces zero-order entropy by transforming longer words, improving data compression potential but with increased overhead.

Contribution

It introduces a recursive generalized context transformation algorithm that extends previous methods to higher order contexts for better entropy reduction.

Findings

Zero order entropy drops significantly after transformation.

Overhead from describing transformations limits effectiveness on small files.

Higher order transformations enhance compression potential.

Abstract

The context transformation and generalized context transformation methods, we introduced recently, were able to reduce zero order entropy by exchanging digrams, and as a consequence, they were removing mutual information between consecutive symbols of the input message. These transformations were intended to be used as a preprocessor for zero-order entropy coding algorithms like Arithmetic or Huffman coding, since we know, that especially Arithmetic coding can achieve a compression rate almost of the size of Shannon's entropy. This paper introduces a novel algorithm based on the concept of generalized context transformation, that allows transformation of words longer than simple digrams. The higher order contexts are exploited using recursive form of a generalized context transformation. It is shown that the zero order entropy of transformed data drops significantly, but on the other hand, the overhead given by a description of individual transformations increases and it has become a limiting factor in a successful transformation of smaller files.