Simulation-Based Parallel Training

TL;DR

This paper introduces a simulation-based parallel training framework for neural networks that generates training data concurrently with model training, reducing data bottlenecks in scientific machine learning.

Contribution

It proposes a novel parallel training approach with a bias mitigation strategy using a memory buffer, demonstrated on chaotic Lorenz system dynamics.

Findings

Framework reduces training data bottlenecks

Bias mitigation improves chaotic dynamics capture

Outperforms offline training methods

Abstract

Numerical simulations are ubiquitous in science and engineering. Machine learning for science investigates how artificial neural architectures can learn from these simulations to speed up scientific discovery and engineering processes. Most of these architectures are trained in a supervised manner. They require tremendous amounts of data from simulations that are slow to generate and memory greedy. In this article, we present our ongoing work to design a training framework that alleviates those bottlenecks. It generates data in parallel with the training process. Such simultaneity induces a bias in the data available during the training. We present a strategy to mitigate this bias with a memory buffer. We test our framework on the multi-parametric Lorenz's attractor. We show the benefit of our framework compared to offline training and the success of our data bias mitigation strategy to capture the complex chaotic dynamics of the system.