Improving Run Length Encoding by Preprocessing

TL;DR

This paper introduces a preprocessing technique that transforms arbitrary data into a form highly suitable for run length encoding, significantly improving compression efficiency over standard RLE methods.

Contribution

The authors propose a novel combination of preprocessing steps, including bit-wise reordering, byte remapping, and transformations, to enhance RLE compression effectiveness.

Findings

Achieves 8x better average compression than standard RLE.

Effective preprocessing makes RLE applicable to natural text and similar data.

Combines multiple transformations with Huffman encoding for optimal results.

Abstract

The Run Length Encoding (RLE) compression method is a long standing simple lossless compression scheme which is easy to implement and achieves a good compression on input data which contains repeating consecutive symbols. In its pure form RLE is not applicable on natural text or other input data with short sequences of identical symbols. We present a combination of preprocessing steps that turn arbitrary input data in a byte-wise encoding into a bit-string which is highly suitable for RLE compression. The main idea is to first read all most significant bits of the input byte-string, followed by the second most significant bit, and so on. We combine this approach by a dynamic byte remapping as well as a Burrows-Wheeler-Scott transform on a byte level. Finally, we apply a Huffman Encoding on the output of the bit-wise RLE encoding to allow for more dynamic lengths of code words encoding runs of the RLE. With our technique we can achieve a lossless average compression which is better than the standard RLE compression by a factor of 8 on average.