Intelligent Sampling of Extreme-Scale Turbulence Datasets for Accurate and Efficient Spatiotemporal Model Training

TL;DR

This paper introduces SICKLE, an intelligent subsampling framework using MaxEnt sampling to efficiently train turbulence models, significantly reducing data and energy consumption while improving accuracy.

Contribution

The paper presents a novel MaxEnt-based sampling method for turbulence datasets, enabling scalable training and energy-efficient model development.

Findings

MaxEnt sampling outperforms random and phase-space sampling.

Subsampling reduces energy consumption by up to 38 times.

Model accuracy can be improved with intelligent data selection.

Abstract

With the end of Moore's law and Dennard scaling, efficient training increasingly requires rethinking data volume. Can we train better models with significantly less data via intelligent subsampling? To explore this, we develop SICKLE, a sparse intelligent curation framework for efficient learning, featuring a novel maximum entropy (MaxEnt) sampling approach, scalable training, and energy benchmarking. We compare MaxEnt with random and phase-space sampling on large direct numerical simulation (DNS) datasets of turbulence. Evaluating SICKLE at scale on Frontier, we show that subsampling as a preprocessing step can, in many cases, improve model accuracy and substantially lower energy consumption, with observed reductions of up to 38x.