CRAM: Compressed Random Access Memory

TL;DR

CRAM is a novel data structure that compresses a dynamic string efficiently, allowing quick access, modifications, and support for insertions and deletions without full decompression, advancing compressed data management.

Contribution

Introduces CRAM, a dynamic compressed data structure that supports efficient access, modification, and updates, with near-optimal compression bounds based on empirical entropy.

Findings

Supports fast substring decompression and character modification.

Allows updates without decompressing the entire string.

Extensible to insertions and deletions with minor performance trade-offs.

Abstract

We present a new data structure called the \emph{Compressed Random Access Memory} (CRAM) that can store a dynamic string $T$ of characters, e.g., representing the memory of a computer, in compressed form while achieving asymptotically almost-optimal bounds (in terms of empirical entropy) on the compression ratio. It allows short substrings of $T$ to be decompressed and retrieved efficiently and, significantly, characters at arbitrary positions of $T$ to be modified quickly during execution \emph{without decompressing the entire string}. This can be regarded as a new type of data compression that can update a compressed file directly. Moreover, at the cost of slightly increasing the time spent per operation, the CRAM can be extended to also support insertions and deletions. Our key observation that the empirical entropy of a string does not change much after a small change to the string, as well as our simple yet efficient method for maintaining an array of variable-length blocks under length modifications, may be useful for many other applications as well.