Lossy Data Compression By Adaptive Mesh Coarsening

TL;DR

This paper introduces an error-bounded lossy data compression method based on adaptive mesh coarsening, suitable for high-performance scientific data and compatible with existing adaptive mesh refinement applications.

Contribution

It presents a novel adaptive mesh coarsening approach for lossy compression that is easily integrable and applicable to various multi-dimensional array data types.

Findings

Effective compression of ERA5 data demonstrated

Supports exclusion of regions of interest

Allows nested error domains during compression

Abstract

Today's scientific simulations, for example in the high-performance exascale sector, produce huge amounts of data. Due to limited I/O bandwidth and available storage space, there is the necessity to reduce scientific data of high performance computing applications. Error-bounded lossy compression has been proven to be an effective approach tackling the trade-off between accuracy and storage space. Within this work, we are exploring and discussing error-bounded lossy compression solely based on adaptive mesh refinement techniques. This compression technique is not only easily integrated into existing adaptive mesh refinement applications but also suits as a general lossy compression approach for arbitrary data in form of multi-dimensional arrays, irrespective of the data type. Moreover, these techniques permit the exclusion of regions of interest and even allows for nested error domains during the compression. The described data compression technique is presented exemplary on ERA5 data.