Combining non-stationary prediction, optimization and mixing for data compression

TL;DR

This paper introduces a new approach for modeling nonstationary binary sequences for data compression, combining prediction, optimization, and mixing techniques, and evaluates its performance against existing estimators and in ensemble models.

Contribution

It presents a novel nonstationary prediction model and an ensemble approach for data compression, with systematic parameter optimization and empirical evaluation.

Findings

The model outperforms Laplace and Krichevsky-Trofimov estimators in tests.

Ensemble model improves compression efficiency on BWT output.

Systematic parameter optimization enhances model performance.

Abstract

In this paper an approach to modelling nonstationary binary sequences, i.e., predicting the probability of upcoming symbols, is presented. After studying the prediction model we evaluate its performance in two non-artificial test cases. First the model is compared to the Laplace and Krichevsky-Trofimov estimators. Secondly a statistical ensemble model for compressing Burrows-Wheeler-Transform output is worked out and evaluated. A systematic approach to the parameter optimization of an individual model and the ensemble model is stated.