Mitigating Artifacts in Pre-quantization Based Scientific Data Compressors with Quantization-aware Interpolation

TL;DR

This paper introduces a quantization-aware interpolation method to reduce artifacts in pre-quantization based scientific data compressors, significantly improving data quality while preserving high throughput in parallel computing environments.

Contribution

It presents a novel artifact mitigation algorithm that characterizes compression errors, improves decompressed data quality, and is efficiently parallelized for high-performance computing.

Findings

Effective artifact reduction in decompressed data

Maintains high compression throughput

Validated on real-world datasets

Abstract

Error-bounded lossy compression has been regarded as a promising way to address the ever-increasing amount of scientific data in today's high-performance computing systems. Pre-quantization, a critical technique to remove sequential dependency and enable high parallelism, is widely used to design and develop high-throughput error-controlled data compressors. Despite the extremely high throughput of pre-quantization based compressors, they generally suffer from low data quality with medium or large user-specified error bounds. In this paper, we investigate the artifacts generated by pre-quantization based compressors and propose a novel algorithm to mitigate them. Our contributions are fourfold: (1) We carefully characterize the artifacts in pre-quantization based compressors to understand the correlation between the quantization index and compression error; (2) We propose a novel quantization-aware interpolation algorithm to improve the decompressed data; (3) We parallelize our algorithm in both shared-memory and distributed-memory environments to obtain high performance; (4) We evaluate our algorithm and validate it with two leading pre-quantization based compressors using five real-world datasets. Experiments demonstrate that our artifact mitigation algorithm can effectively improve the quality of decompressed data produced by pre-quantization based compressors while maintaining their high compression throughput.