Prioritized Data Compression using Wavelets

TL;DR

This paper introduces a wavelet-based data compression method tailored for high-performance computing, focusing on preserving salient information efficiently in large-scale simulations.

Contribution

It presents a novel decision-theoretic approach to prioritized data compression using wavelets, with practical heuristics and application insights for HPC environments.

Findings

Effective preservation of salient data regions

Applicability to large-scale HPC simulations

Potential for improved storage efficiency

Abstract

The volume of data and the velocity with which it is being generated by com- putational experiments on high performance computing (HPC) systems is quickly outpacing our ability to effectively store this information in its full fidelity. There- fore, it is critically important to identify and study compression methodologies that retain as much information as possible, particularly in the most salient regions of the simulation space. In this paper, we cast this in terms of a general decision-theoretic problem and discuss a wavelet-based compression strategy for its solution. We pro- vide a heuristic argument as justification and illustrate our methodology on several examples. Finally, we will discuss how our proposed methodology may be utilized in an HPC environment on large-scale computational experiments.