ANTIC: Adaptive Neural Temporal In-situ Compressor

TL;DR

ANTIC is an in situ compression pipeline that adaptively filters and compresses high-dimensional simulation data, significantly reducing storage needs while maintaining accuracy.

Contribution

It introduces a novel adaptive temporal selector and neural spatial compression module for efficient in situ data reduction in large-scale physics simulations.

Findings

Achieves several orders of magnitude reduction in storage requirements.

Maintains physics accuracy despite aggressive compression.

Operates in a single streaming pass for efficiency.

Abstract

The persistent storage requirements for high-resolution, spatiotemporally evolving fields governed by large-scale and high-dimensional partial differential equations (PDEs) have reached the petabyte-to-exabyte scale. Transient simulations modeling Navier-Stokes equations, magnetohydrodynamics, plasma physics, or binary black hole mergers generate data volumes that are prohibitive for modern high-performance computing (HPC) infrastructures. To address this bottleneck, we introduce ANTIC (Adaptive Neural Temporal in situ Compressor), an end-to-end in situ compression pipeline. ANTIC consists of an adaptive temporal selector tailored to high-dimensional physics that identifies and filters informative snapshots at simulation time, combined with a spatial neural compression module based on continual fine-tuning that learns residual updates between adjacent snapshots using neural fields. By operating in a single streaming pass, ANTIC enables a combined compression of temporal and spatial components and effectively alleviates the need for explicit on-disk storage of entire time-evolved trajectories. Experimental results demonstrate how storage reductions of several orders of magnitude relate to physics accuracy.