Significant improvement of lossy compression rate and speed of HPC data using perceptron parallelized compression

TL;DR

This paper presents a perceptron-based lossy compression framework that significantly improves compression rate and speed for HPC data, addressing the challenge of efficient data storage amid exponential data growth.

Contribution

It introduces a novel compression framework with enhanced predictive accuracy and parallelization, achieving up to 17.78% better compression ratios compared to existing methods.

Findings

Maximum compression ratio reduction of 17.78%

Enhanced predictive accuracy with a three-layer perceptron

Effective parallelized compression within blocks

Abstract

The escalating surge in data generation presents formidable challenges to information technology, necessitating advancements in storage, retrieval, and utilization. With the proliferation of artificial intelligence and big data, the "Data Age 2025" report forecasts an exponential increase in global data production. The escalating data volumes raise concerns about efficient data processing. The paper addresses the predicament of achieving a lower compression ratio while maintaining or surpassing the compression performance of state-of-the-art techniques. This paper introduces a lossy compression framework grounded in the perceptron model for data prediction, striving for high compression quality. The contributions of this study encompass the introduction of positive and negative factors within the relative-to-absolute domain transformation algorithm, the utilization of a three-layer perceptron for improved predictive accuracy, and data selection rule modifications for parallelized compression within compression blocks. Comparative experiments with SZ2.1's PW_REL mode demonstrate a maximum compression ratio reduction of 17.78%. The article is structured as follows: the introduction highlights the data explosion challenge; related work delves into existing solutions; optimization of mapping algorithms in the relative and absolute domains is expounded in Section 3,the design of the new compression framework is detailed in Section 4,In Section 5 we describe the whole process and give pseudo-code, and in Section 6, our solution is evaluated. Finally, in Section 7, we provide an outlook for future work.