Guaranteed Conditional Diffusion: 3D Block-based Models for Scientific Data Compression

TL;DR

This paper introduces Guaranteed Conditional Diffusion with Tensor Correction (GCDTC), a novel lossy scientific data compression method that combines conditional diffusion models, tensor correction, and error guarantees to improve compression quality and reliability.

Contribution

The paper presents a new framework integrating conditional diffusion models with tensor correction and error guarantees for scientific data compression, ensuring deterministic reconstruction with controlled error.

Findings

Outperforms standard convolutional autoencoders in experiments.

Achieves competitive compression quality with existing algorithms.

Ensures maximum error distortion in lossy compression.

Abstract

This paper proposes a new compression paradigm -- Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) -- for lossy scientific data compression. The framework is based on recent conditional diffusion (CD) generative models, and it consists of a conditional diffusion model, tensor correction, and error guarantee. Our diffusion model is a mixture of 3D conditioning and 2D denoising U-Net. The approach leverages a 3D block-based compressing module to address spatiotemporal correlations in structured scientific data. Then, the reverse diffusion process for 2D spatial data is conditioned on the ``slices'' of content latent variables produced by the compressing module. After training, the denoising decoder reconstructs the data with zero noise and content latent variables, and thus it is entirely deterministic. The reconstructed outputs of the CD model are further post-processed by our tensor correction and error guarantee steps to control and ensure a maximum error distortion, which is an inevitable requirement in lossy scientific data compression. Our experiments involving two datasets generated by climate and chemical combustion simulations show that our framework outperforms standard convolutional autoencoders and yields competitive compression quality with an existing scientific data compression algorithm.