DeXOR: Enabling XOR in Decimal Space for Streaming Lossless Compression of Floating-point Data

TL;DR

DeXOR is a novel compression framework that efficiently encodes streaming floating-point data in decimal space, achieving higher compression ratios and faster decompression while maintaining robustness under extreme conditions.

Contribution

DeXOR introduces decimal XOR procedures and error-tolerant strategies for effective lossless compression of floating-point streams, outperforming existing methods.

Findings

15% higher compression ratio than state-of-the-art schemes

20% faster decompression speed

Robust performance under extreme conditions

Abstract

With streaming floating-point numbers being increasingly prevalent, effective and efficient compression of such data is critical. Compression schemes must be able to exploit the similarity, or smoothness, of consecutive numbers and must be able to contend with extreme conditions, such as high-precision values or the absence of smoothness. We present DeXOR, a novel framework that enables decimal XOR procedure to encode decimal-space longest common prefixes and suffixes, achieving optimal prefix reuse and effective redundancy elimination. To ensure accurate and low-cost decompression even with binary-decimal conversion errors, DeXOR incorporates 1) scaled truncation with error-tolerant rounding and 2) different bit management strategies optimized for decimal XOR. Additionally, a robust exception handler enhances stability by managing floating-point exponents, maintaining high compression ratios under extreme conditions. In evaluations across 22 datasets, DeXOR surpasses state-of-the-art schemes, achieving a 15% higher compression ratio and a 20% faster decompression speed while maintaining a competitive compression speed. DeXOR also offers scalability under varying conditions and exhibits robustness in extreme scenarios where other schemes fail.