Continual learning autoencoder training for a particle-in-cell simulation via streaming

TL;DR

This paper introduces a streaming-based continual learning approach for training autoencoders concurrently with high-resolution particle-in-cell simulations, enabling efficient model training without disk storage and improving generalization.

Contribution

It proposes a novel in-memory streaming training pipeline combined with continual learning methods for physics simulations, addressing data storage challenges at exascale.

Findings

Effective autoencoder training during simulations

Improved model generalization with continual learning methods

Feasibility demonstrated on laser wakefield acceleration simulations

Abstract

The upcoming exascale era will provide a new generation of physics simulations. These simulations will have a high spatiotemporal resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible. Therefore, we need to rethink the training of machine learning models for simulations for the upcoming exascale era. This work presents an approach that trains a neural network concurrently to a running simulation without storing data on a disk. The training pipeline accesses the training data by in-memory streaming. Furthermore, we apply methods from the domain of continual learning to enhance the generalization of the model. We tested our pipeline on the training of a 3d autoencoder trained concurrently to laser wakefield acceleration particle-in-cell simulation. Furthermore, we experimented with various continual learning methods and their effect on the generalization.