Seq2Seq2Seq: Lossless Data Compression via Discrete Latent Transformers and Reinforcement Learning

TL;DR

This paper introduces a novel lossless data compression method using discrete latent transformers and reinforcement learning, which preserves token structure and improves compression ratios over traditional techniques.

Contribution

It proposes a reinforcement learning-based approach applied to a T5 model to enable token-sequence compression, maintaining data structure and enhancing efficiency.

Findings

Achieves higher compression ratios than conventional methods.

Preserves token-based data structure for better data integrity.

Effective without external grammatical or world knowledge.

Abstract

Efficient lossless compression is essential for minimizing storage costs and transmission overhead while preserving data integrity. Traditional compression techniques, such as dictionary-based and statistical methods, often struggle to optimally exploit the structure and redundancy in complex data formats. Recent advancements in deep learning have opened new avenues for compression; however, many existing approaches depend on dense vector representations that obscure the underlying token structure. To address these limitations, we propose a novel lossless compression method that leverages Reinforcement Learning applied to a T5 language model architecture. This approach enables the compression of data into sequences of tokens rather than traditional vector representations. Unlike auto-encoders, which typically encode information into continuous latent spaces, our method preserves the token-based structure, aligning more closely with the original data format. This preservation allows for higher compression ratios while maintaining semantic integrity. By training the model using an off-policy Reinforcement Learning algorithm, we optimize sequence length to minimize redundancy and enhance compression efficiency. Our method introduces an efficient and adaptive data compression system built upon advanced Reinforcement Learning techniques, functioning independently of external grammatical or world knowledge. This approach shows significant improvements in compression ratios compared to conventional methods. By leveraging the latent information within language models, our system effectively compresses data without requiring explicit content understanding, paving the way for more robust and practical compression solutions across various applications.