On the Value of Multiple Read/Write Streams for Data Compression

TL;DR

This paper investigates the potential benefits of using multiple read/write streams for data compression under limited memory and passes, demonstrating that two streams are necessary for optimal entropy-based compression.

Contribution

It establishes the minimal number of streams needed for different levels of data compression quality, highlighting the importance of multiple streams for entropy-only bounds.

Findings

One stream suffices for universal compression.

One stream is insufficient for grammar-based compression.

Two streams are necessary and sufficient for entropy-only bounds.

Abstract

We study whether, when restricted to using polylogarithmic memory and polylogarithmic passes, we can achieve qualitatively better data compression with multiple read/write streams than we can with only one. We first show how we can achieve universal compression using only one pass over one stream. We then show that one stream is not sufficient for us to achieve good grammar-based compression. Finally, we show that two streams are necessary and sufficient for us to achieve entropy-only bounds.