Random Feature Models for Learning Interacting Dynamical Systems

TL;DR

This paper introduces a randomized feature-based method with sparsity promotion for learning interaction kernels in high-dimensional dynamical systems from noisy data, improving prediction accuracy and computational efficiency.

Contribution

It develops a novel sparse randomized feature algorithm for data-driven approximation of interaction kernels in multi-agent systems, reducing overfitting and computational costs.

Findings

Effective in first-order and second-order systems

Reduces overfitting with limited data

Lowers simulation costs significantly

Abstract

Particle dynamics and multi-agent systems provide accurate dynamical models for studying and forecasting the behavior of complex interacting systems. They often take the form of a high-dimensional system of differential equations parameterized by an interaction kernel that models the underlying attractive or repulsive forces between agents. We consider the problem of constructing a data-based approximation of the interacting forces directly from noisy observations of the paths of the agents in time. The learned interaction kernels are then used to predict the agents behavior over a longer time interval. The approximation developed in this work uses a randomized feature algorithm and a sparse randomized feature approach. Sparsity-promoting regression provides a mechanism for pruning the randomly generated features which was observed to be beneficial when one has limited data, in particular, leading to less overfitting than other approaches. In addition, imposing sparsity reduces the kernel evaluation cost which significantly lowers the simulation cost for forecasting the multi-agent systems. Our method is applied to various examples, including first-order systems with homogeneous and heterogeneous interactions, second order homogeneous systems, and a new sheep swarming system.