In Situ Training of Implicit Neural Compressors for Scientific Simulations via Sketch-Based Regularization

TL;DR

This paper introduces a novel in situ training method for implicit neural compressors in scientific simulations, using sketch-based regularization to prevent forgetting and achieve high compression with performance close to offline methods.

Contribution

The paper presents a new in situ training protocol employing sketching for implicit neural representations, enabling efficient compression in complex scientific simulations.

Findings

Strong reconstruction performance at high compression rates.

Sketching enables in situ training to match offline method performance.

Method effective across 2D, 3D, unstructured, and non-Cartesian geometries.

Abstract

Focusing on implicit neural representations, we present a novel in situ training protocol that employs limited memory buffers of full and sketched data samples, where the sketched data are leveraged to prevent catastrophic forgetting. The theoretical motivation for our use of sketching as a regularizer is presented via a simple Johnson-Lindenstrauss-informed result. While our methods may be of wider interest in the field of continual learning, we specifically target in situ neural compression using implicit neural representation-based hypernetworks. We evaluate our method on a variety of complex simulation data in two and three dimensions, over long time horizons, and across unstructured grids and non-Cartesian geometries. On these tasks, we show strong reconstruction performance at high compression rates. Most importantly, we demonstrate that sketching enables the presented in situ scheme to approximately match the performance of the equivalent offline method.