Prediction of Large Alphabet Processes and Its Application to Adaptive Source Coding

TL;DR

This paper addresses predicting sequences from large alphabets, proposing a method that improves prediction precision over existing algorithms, with applications to adaptive source coding.

Contribution

It introduces a novel prediction method for large alphabet sources that outperforms known algorithms in terms of Kullback-Leibler divergence.

Findings

Enhanced prediction accuracy for large alphabet sequences

Method applicable to adaptive source coding

Improved divergence bounds compared to prior algorithms

Abstract

The problem of predicting a sequence $x_1,x_2,...$ generated by a discrete source with unknown statistics is considered. Each letter $x_{t+1}$ is predicted using information on the word $x_1x_2... x_t$ only. In fact, this problem is a classical problem which has received much attention. Its history can be traced back to Laplace. We address the problem where each $x_i$ belongs to some large (or even infinite) alphabet. A method is presented for which the precision is greater than for known algorithms, where precision is estimated by the Kullback-Leibler divergence. The results can readily be translated to results about adaptive coding.