Learning dynamical systems with hit-and-run random feature maps

TL;DR

This paper introduces an advanced random feature map approach with deep variants and localization techniques, achieving state-of-the-art forecasting of complex chaotic dynamical systems with minimal hyperparameter tuning.

Contribution

It presents a novel deep and localized random feature map method for forecasting dynamical systems, reducing hyperparameter tuning and improving accuracy over existing methods.

Findings

Excellent forecasting skill for systems up to 512 dimensions

Effective long-term statistical property estimation

Requires tuning only a single hyperparameter

Abstract

We show how random feature maps can be used to forecast dynamical systems with excellent forecasting skill. We consider the tanh activation function and judiciously choose the internal weights in a data-driven manner such that the resulting features explore the nonlinear, non-saturated regions of the activation function. We introduce skip connections and construct a deep variant of random feature maps by combining several units. To mitigate the curse of dimensionality, we introduce localization where we learn local maps, employing conditional independence. Our modified random feature maps provide excellent forecasting skill for both single trajectory forecasts as well as long-time estimates of statistical properties, for a range of chaotic dynamical systems with dimensions up to 512. In contrast to other methods such as reservoir computers which require extensive hyperparameter tuning, we effectively need to tune only a single hyperparameter, and are able to achieve state-of-the-art forecast skill with much smaller networks.