Attention Based Machine Learning Methods for Data Reduction with Guaranteed Error Bounds

TL;DR

This paper introduces an attention-based hierarchical data compression method for scientific datasets that leverages spatial, temporal, and inter-variable correlations, achieving significantly higher compression ratios with guaranteed error bounds.

Contribution

It presents a novel attention-based autoencoder framework with error-bound guarantees, outperforming existing methods like SZ3 in scientific data reduction.

Findings

Up to 8x higher compression ratio on multi-variable datasets.

Achieves 3x and 2x higher compression ratios on single-variable datasets.

Effectively captures spatiotemporal and inter-variable correlations.

Abstract

Scientific applications in fields such as high energy physics, computational fluid dynamics, and climate science generate vast amounts of data at high velocities. This exponential growth in data production is surpassing the advancements in computing power, network capabilities, and storage capacities. To address this challenge, data compression or reduction techniques are crucial. These scientific datasets have underlying data structures that consist of structured and block structured multidimensional meshes where each grid point corresponds to a tensor. It is important that data reduction techniques leverage strong spatial and temporal correlations that are ubiquitous in these applications. Additionally, applications such as CFD, process tensors comprising hundred plus species and their attributes at each grid point. Reduction techniques should be able to leverage interrelationships between the elements in each tensor. In this paper, we propose an attention-based hierarchical compression method utilizing a block-wise compression setup. We introduce an attention-based hyper-block autoencoder to capture inter-block correlations, followed by a block-wise encoder to capture block-specific information. A PCA-based post-processing step is employed to guarantee error bounds for each data block. Our method effectively captures both spatiotemporal and inter-variable correlations within and between data blocks. Compared to the state-of-the-art SZ3, our method achieves up to 8 times higher compression ratio on the multi-variable S3D dataset. When evaluated on single-variable setups using the E3SM and XGC datasets, our method still achieves up to 3 times and 2 times higher compression ratio, respectively.