Accurate Performance Modeling And Uncertainty Analysis of Lossy Compression in Scientific Applications

TL;DR

This paper introduces surrogate models for predicting lossy compression times in scientific applications, achieving high accuracy and providing uncertainty analysis to improve scheduling and performance optimization.

Contribution

It presents novel surrogate models that accurately predict compression times and incorporate uncertainty analysis, addressing limitations of prior empirical prediction methods.

Findings

Achieves 5% average prediction error across datasets

Provides 95% confidence intervals for compression time estimates

Enhances scheduling accuracy in scientific workflows

Abstract

Scientific applications typically generate large volumes of floating-point data, making lossy compression one of the most effective methods for data reduction, thereby lowering storage requirements and improving performance in large-scale applications. However, variations in compression time can significantly impact overall performance improvement, due to inaccurate scheduling, workload imbalances, etc. Existing approaches rely on empirical methods to predict the compression performance, which often lack interpretability and suffer from limitations in accuracy and generalizability. In this paper, we propose surrogate models for predicting the compression time of prediction-based lossy compression and provide a detailed analysis of the factors influencing time variability with uncertainty analysis. Our evaluation shows that our solution can accuratly predict the compression time with 5% average error across six scientific datasets. It also provides accurate 95% confidence interval, which is essential for time-sensitive scheduling and applications.