Attention-Enhanced Reservoir Computing as a Multiple Dynamical System Approximator

TL;DR

This paper introduces an attention-enhanced reservoir computing model that dynamically prioritizes features, enabling it to predict multiple chaotic systems simultaneously with high accuracy and adaptability, surpassing traditional methods.

Contribution

The study presents a novel attention mechanism integrated into reservoir computing, allowing for multi-attractor learning and transition without retraining, improving prediction and modeling of complex dynamical systems.

Findings

Achieves superior prediction accuracy over traditional reservoir computing.

Successfully predicts multiple chaotic attractors with a single model.

Demonstrates robustness across various benchmark dynamical systems.

Abstract

Reservoir computing has proven effective for tasks such as time-series prediction, particularly in the context of chaotic systems. However, conventional reservoir computing frameworks often face challenges in achieving high prediction accuracy and adapting to diverse dynamical problems due to their reliance on fixed weight structures. A concept of an attention-enhanced reservoir computer has been proposed, which integrates an attention mechanism into the output layer of the reservoir computing model. This addition enables the system to prioritize distinct features dynamically, enhancing adaptability and prediction performance. In this study, we demonstrate the capability of the attention-enhanced reservoir computer to learn and predict multiple chaotic attractors simultaneously with a single set of weights, thus enabling transitions between attractors without explicit retraining. The method is validated using benchmark tasks, including the Lorenz system, R\"ossler system, Henon map, Duffing oscillator, and Mackey-Glass delay-differential equation. Our results indicate that the attention-enhanced reservoir computer achieves superior prediction accuracy, valid prediction times, and improved representation of spectral and histogram characteristics compared to traditional reservoir computing methods, establishing it as a robust tool for modeling complex dynamical systems.