Local Decode and Update for Big Data Compression

TL;DR

This paper introduces a universal data compression scheme that enables efficient local decoding and updating, achieving near-entropy rates with constant or linear probing, significantly improving over previous methods.

Contribution

It presents a novel compression scheme that allows local decoding and updates with constant or linear probing, maintaining near-entropy compression rates and low complexity.

Findings

Local decoding and update require probing a constant number of bits.

Compression rate approaches the source entropy.

Encoding and decoding are quasilinear in blocklength.

Abstract

This paper investigates data compression that simultaneously allows local decoding and local update. The main result is a universal compression scheme for memoryless sources with the following features. The rate can be made arbitrarily close to the entropy of the underlying source, contiguous fragments of the source can be recovered or updated by probing or modifying a number of codeword bits that is on average linear in the size of the fragment, and the overall encoding and decoding complexity is quasilinear in the blocklength of the source. In particular, the local decoding or update of a single message symbol can be performed by probing or modifying a constant number of codeword bits. This latter part improves over previous best known results for which local decodability or update efficiency grows logarithmically with blocklength.