Neural Scaling Laws for Learning-based Identification of Nonlinear Systems

TL;DR

This paper investigates how the performance of machine learning models in nonlinear system identification scales with resources like data and model size, using neural scaling laws to guide future model design and data collection.

Contribution

It verifies neural scaling laws for system identification across various architectures and metrics, providing a framework to predict performance and optimize resource allocation.

Findings

Neural scaling laws hold for nonlinear system identification.

Performance can be forecasted based on data and model size.

Guidelines for model design and data acquisition are derived.

Abstract

The use of machine learning models in system identification has increased due to their ability to approximate complex nonlinear dynamics with high accuracy. However, often it is not clear how the performance of trained models scales with given resources such as data, compute, and model size. To allow for a better understanding of the scalability of the performance of machine learning models, we verify neural scaling laws (NSLs) in the context of system identification from input-state-output data using different evaluation metrics for accuracy and different system architectures, including input-affine and physics-informed port-Hamiltonian representations. Our verified NSLs can help to forecast performance improvements and guide model design or data acquisition.