Backpropagation in hyperbolic chaos via adjoint shadowing

TL;DR

This paper extends backpropagation to hyperbolic chaos systems using the adjoint shadowing operator, enabling efficient computation of linear response contributions in both discrete and continuous time.

Contribution

It introduces the adjoint shadowing operator $\\mathcal{S}$, providing a unified framework for backpropagation in hyperbolic chaos systems and linking it to shadowing and unstable contributions.

Findings

The adjoint shadowing operator $\\mathcal{S}$ is equivalent to the adjoint of the linear shadowing operator.

$\\mathcal{S}$ can be computed efficiently using a shadowing algorithm similar to backpropagation.

The linear response in continuous-time systems can be decomposed into shadowing and unstable contributions.

Abstract

To generalize the backpropagation method to both discrete-time and continuous-time hyperbolic chaos, we introduce the adjoint shadowing operator $\mathcal{S}$ acting on covector fields. We show that $\mathcal{S}$ can be equivalently defined as: (a) $\mathcal{S}$ is the adjoint of the linear shadowing operator $S$; (b) $\mathcal{S}$ is given by a `split then propagate' expansion formula; (c) $\mathcal{S}(\omega)$ is the only bounded inhomogeneous adjoint solution of $\omega$. By (a), $\mathcal{S}$ adjointly expresses the shadowing contribution, a significant part of the linear response, where the linear response is the derivative of the long-time statistics with respect to system parameters. By (b), $\mathcal{S}$ also expresses the other part of the linear response, the unstable contribution. By (c), $\mathcal{S}$ can be efficiently computed by the nonintrusive shadowing algorithm in Ni and Talnikar (2019 J. Comput. Phys. 395 690-709), which is similar to the conventional backpropagation algorithm. For continuous-time cases, we additionally show that the linear response admits a well-defined decomposition into shadowing and unstable contributions.