Adaptive Encoding Strategies for Erasing-Based Lossless Floating-Point Compression

TL;DR

This paper introduces Elf*, an optimized adaptive encoding strategy for lossless floating-point time series compression, improving compression ratios and efficiency over existing methods, especially in streaming scenarios.

Contribution

It proposes Elf*, a set of adaptive encoding optimizations with a theoretical proof of optimality, and extends it to Streaming Elf* for high efficiency in streaming environments.

Findings

SElf* achieves 9.2% better compression ratio than competitors.

Elf* ranks among the most competitive batch compressors.

SElf* maintains high efficiency in streaming scenarios.

Abstract

Lossless floating-point time series compression is crucial for a wide range of critical scenarios. Nevertheless, it is a big challenge to compress time series losslessly due to the complex underlying layouts of floating-point values. The state-of-the-art erasing-based compression algorithm Elf demonstrates a rather impressive performance. We give an in-depth exploration of the encoding strategies of Elf, and find that there is still much room for improvement. In this paper, we propose Elf*, which employs a set of optimizations for leading zeros, center bits and sharing condition. Specifically, we develop a dynamic programming algorithm with a set of pruning strategies to compute the adaptive approximation rules efficiently. We theoretically prove that the adaptive approximation rules are globally optimal. We further extend Elf* to Streaming Elf*, i.e., SElf*, which achieves almost the same compression ratio as Elf*, while enjoying even higher efficiency in streaming scenarios. We compare Elf* and SElf* with 8 competitors using 22 datasets. The results demonstrate that SElf* achieves 9.2% relative compression ratio improvement over the best streaming competitor while maintaining similar efficiency, and that Elf* ranks among the most competitive batch compressors. All source codes are publicly released.