Time-universal data compression and prediction

TL;DR

This paper introduces a method for selecting the best data compressor from a set with minimal additional time, effectively integrating compressor selection into the compression process.

Contribution

It presents a novel approach that encodes data using the optimal compressor while limiting the extra computational time required.

Findings

Achieves near-optimal compression with limited additional time.

Automates the selection of the best compressor within a set.

Reduces the computational overhead of compressor selection.

Abstract

Suppose there is a large file which should be transmitted (or stored) and there are several (say, m) admissible data-compressors. It seems natural to try all the compressors and then choose the best, i.e. the one that gives the shortest compressed file. Then transfer (or store) the index number of the best compressor (it requires log m bits) and the compressed file.The only problem is the time, which essentially increases due to the need to compress the file m times (in order to find the best compressor). We propose a method that encodes the file with the optimal compressor, but uses a relatively small additional time: the ratio of this extra time and the total time of calculation can be limited by an arbitrary positive constant. Generally speaking, in many situations it may be necessary find the best data compressor out of a given set, which is often done by comparing them empirically. One of the goals of this work is to turn such a selection process into a part of the data compression method, automating and optimizing it.