No-propagate algorithm for linear responses of random chaotic systems

TL;DR

This paper introduces a novel no-propagate algorithm for efficiently sampling the linear response of noisy chaotic systems, avoiding common computational issues like gradient explosion and high dimensionality.

Contribution

The paper presents a new algorithm that differs from existing methods by not propagating vectors or covectors, simplifying the computation of linear responses in complex systems.

Findings

Successfully applied to a noisy tent map and a deep neural network

Avoids gradient explosion and high-dimensional issues

Provides a linear response approximation with controllable noise-related error

Abstract

We develop the no-propagate algorithm for sampling the linear response of random dynamical systems, which are non-uniform hyperbolic deterministic systems perturbed by noise with smooth density. We first derive a Monte-Carlo type formula and then the algorithm, which is different from the ensemble (stochastic gradient) algorithms, finite-element algorithms, and fast-response algorithms; it does not involve the propagation of vectors or covectors, and only the density of the noise is differentiated, so the formula is not cursed by gradient explosion, dimensionality, or non-hyperbolicity. We demonstrate our algorithm on a tent map perturbed by noise and a chaotic neural network with 51 layers $\times$ 9 neurons. By itself, this algorithm approximates the linear response of non-hyperbolic deterministic systems, with an additional error proportional to the noise. We also discuss the potential of using this algorithm as a part of a bigger algorithm with smaller error.