Variable Length Lossless Coding for Variational Distance Class: An Optimal Merging Algorithm

TL;DR

This paper develops an optimal merging algorithm for lossless source coding within a total variational distance ball, providing a minimax solution with a waterfilling approach and an efficient ${ m O}(n)$ algorithm.

Contribution

It introduces a novel minimax coding framework for sources within a variational distance ball, with a new optimal merging algorithm and a fast computational method.

Findings

Derived the optimal codeword lengths via waterfilling.

Proposed an efficient ${ m O}(n)$ algorithm for code computation.

Established a partition-based solution for the optimization problem.

Abstract

In this paper we consider lossless source coding for a class of sources specified by the total variational distance ball centred at a fixed nominal probability distribution. The objective is to find a minimax average length source code, where the minimizers are the codeword lengths -- real numbers for arithmetic or Shannon codes -- while the maximizers are the source distributions from the total variational distance ball. Firstly, we examine the maximization of the average codeword length by converting it into an equivalent optimization problem, and we give the optimal codeword lenghts via a waterfilling solution. Secondly, we show that the equivalent optimization problem can be solved via an optimal partition of the source alphabet, and re-normalization and merging of the fixed nominal probabilities. For the computation of the optimal codeword lengths we also develop a fast algorithm with a computational complexity of order ${\cal O}(n)$.