Comparison of neural network training strategies for the simulation of dynamical systems

TL;DR

This paper compares parallel and series-parallel neural network training strategies for simulating dynamical systems, finding that parallel training offers superior long-term prediction accuracy across multiple architectures and applications.

Contribution

It provides an empirical comparison of training strategies, clarifies terminology, and recommends parallel training as the default for neural network-based system simulation.

Findings

Parallel training yields better long-term accuracy.

Series-parallel training is more common but less effective.

Clarifies terminology and links strategies to system identification.

Abstract

Neural networks have become a widely adopted tool for modeling nonlinear dynamical systems from data. However, the choice of training strategy remains a key design decision, particularly for simulation tasks. This paper compares two predominant strategies: parallel and series-parallel training. The conducted empirical analysis spans five neural network architectures and two examples: a pneumatic valve test bench and an industrial robot benchmark. The study reveals that, even though series-parallel training dominates current practice, parallel training consistently yields better long-term prediction accuracy. Additionally, this work clarifies the often inconsistent terminology in the literature and relate both strategies to concepts from system identification. The findings suggest that parallel training should be considered the default training strategy for neural network-based simulation of dynamical systems.