Exploring Autoencoder-based Error-bounded Compression for Scientific Data

TL;DR

This paper introduces a convolutional autoencoder framework for error-bounded lossy compression of scientific data, achieving significantly better compression ratios while maintaining data fidelity, compared to existing methods.

Contribution

It develops an error-bounded autoencoder-based compression framework tailored for scientific data, optimizing model parameters and demonstrating superior performance over existing solutions.

Findings

Achieves 100% to 800% better compression ratio than SZ2.1 and ZFP at similar data distortion.

Provides an in-depth analysis of autoencoder characteristics for error-bounded compression.

Demonstrates competitive compression quality on five real-world scientific datasets.

Abstract

Error-bounded lossy compression is becoming an indispensable technique for the success of today's scientific projects with vast volumes of data produced during simulations or instrument data acquisitions. Not only can it significantly reduce data size, but it also can control the compression errors based on user-specified error bounds. Autoencoder (AE) models have been widely used in image compression, but few AE-based compression approaches support error-bounding features, which are highly required by scientific applications. To address this issue, we explore using convolutional autoencoders to improve error-bounded lossy compression for scientific data, with the following three key contributions. (1) We provide an in-depth investigation of the characteristics of various autoencoder models and develop an error-bounded autoencoder-based framework in terms of the SZ model. (2) We optimize the compression quality for the main stages in our designed AE-based error-bounded compression framework, fine-tuning the block sizes and latent sizes and also optimizing the compression efficiency of latent vectors. (3) We evaluate our proposed solution using five real-world scientific datasets and compare them with six other related works. Experiments show that our solution exhibits a very competitive compression quality among all the compressors in our tests. In absolute terms, it can obtain a much better compression quality (100% ~ 800% improvement in compression ratio with the same data distortion) compared with SZ2.1 and ZFP in cases with a high compression ratio.