Existence of n-cycles and border-collision bifurcations in piecewise-linear continuous maps with applications to recurrent neural networks

TL;DR

This paper analyzes the existence of n-cycles and border-collision bifurcations in piecewise linear recurrent neural networks, providing theoretical conditions and numerical validation to understand their dynamics and potential for chaos.

Contribution

It offers the first theoretical analysis of n-cycle existence and border-collision bifurcations in n-dimensional PLRNNs, extending known 1D results to higher dimensions.

Findings

Derived parametric regions for stable and unstable n-cycles.

Showed border-collision bifurcations occur at switching boundaries.

Numerical simulations confirm theoretical predictions.

Abstract

Piecewise linear recurrent neural networks (PLRNNs) form the basis of many successful machine learning applications for time series prediction and dynamical systems identification, but rigorous mathematical analysis of their dynamics and properties is lagging behind. Here we contribute to this topic by investigating the existence of n-cycles $(n\geq 3)$ and border-collision bifurcations in a class of n-dimensional piecewise linear continuous maps which have the general form of a PLRNN. This is particularly important as for one-dimensional maps the existence of 3-cycles implies chaos. It is shown that these n-cycles collide with the switching boundary in a border-collision bifurcation, and parametric regions for the existence of both stable and unstable n-cycles and border-collision bifurcations will be derived theoretically. We then discuss how our results can be extended and applied to PLRNNs. Finally, numerical simulations demonstrate the implementation of our results and are found to be in good agreement with the theoretical derivations. Our findings thus provide a basis for understanding periodic behavior in PLRNNs, how it emerges in bifurcations, and how it may lead into chaos.