Mixing Strategies in Data Compression

TL;DR

This paper introduces geometric weighting as a new method for model combination in data compression, generalizing PAQ-weighting to non-binary alphabets and demonstrating its superiority through experimental evaluation.

Contribution

It proposes a novel geometric weighting method for model combination, generalizes PAQ-weighting, and introduces a new linear mixture technique with an optimal weight optimization approach.

Findings

Geometric weighting outperforms linear weighting in experiments.

Optimal weight selection is formulated as a convex optimization problem.

The methods are applicable to non-binary alphabets.

Abstract

We propose geometric weighting as a novel method to combine multiple models in data compression. Our results reveal the rationale behind PAQ-weighting and generalize it to a non-binary alphabet. Based on a similar technique we present a new, generic linear mixture technique. All novel mixture techniques rely on given weight vectors. We consider the problem of finding optimal weights and show that the weight optimization leads to a strictly convex (and thus, good-natured) optimization problem. Finally, an experimental evaluation compares the two presented mixture techniques for a binary alphabet. The results indicate that geometric weighting is superior to linear weighting.