A Fast Transformer-based General-Purpose Lossless Compressor
Yu Mao, Yufei Cui, Tei-Wei Kuo, Chun Jason Xue

TL;DR
This paper introduces TRACE, a transformer-based lossless compressor that significantly reduces execution time while maintaining competitive compression ratios, by designing a lightweight, compression-friendly transformer structure and acceleration strategies.
Contribution
The paper proposes a novel, fast, general-purpose lossless compressor using a single-layer transformer with new model selection metrics and acceleration techniques, addressing computational inefficiency in existing methods.
Findings
TRACE achieves approximately 3x speedup over state-of-the-art compressors.
It maintains comparable compression ratios to existing methods.
The design enables efficient parallel history-dependency modeling in compression.
Abstract
Deep-learning-based compressor has received interests recently due to much improved compression ratio. However, modern approaches suffer from long execution time. To ease this problem, this paper targets on cutting down the execution time of deep-learning-based compressors. Building history-dependencies sequentially (e.g., recurrent neural networks) is responsible for long inference latency. Instead, we introduce transformer into deep learning compressors to build history-dependencies in parallel. However, existing transformer is too heavy in computation and incompatible to compression tasks. This paper proposes a fast general-purpose lossless compressor, TRACE, by designing a compression-friendly structure based on a single-layer transformer. We first design a new metric to advise the selection part of compression model structures. Byte-grouping and Shared-ffn schemes are further…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsParallel Computing and Optimization Techniques · Advanced Neural Network Applications · Advanced Data Storage Technologies
