Parallel BiLSTM-Transformer networks for forecasting chaotic dynamics

TL;DR

This paper introduces a parallel hybrid model combining Transformer and BiLSTM networks to improve the prediction of chaotic systems by capturing both local and global features, outperforming single-branch models.

Contribution

The study proposes a novel dual-branch architecture integrating Transformer and BiLSTM for chaotic time series prediction, effectively capturing local and global features simultaneously.

Findings

Outperforms single-branch models in chaotic system prediction

Demonstrates robustness in long-term trajectory forecasting

Effective in reconstructing unmeasured states from partial data

Abstract

The nonlinear nature of chaotic systems results in extreme sensitivity to initial conditions and highly intricate dynamical behaviors, posing fundamental challenges for accurately predicting their evolution. To overcome the limitation that conventional approaches fail to capture both local features and global dependencies in chaotic time series simultaneously, this study proposes a parallel predictive framework integrating Transformer and Bidirectional Long Short-Term Memory (BiLSTM) networks. The hybrid model employs a dual-branch architecture, where the Transformer branch mainly captures long-range dependencies while the BiLSTM branch focuses on extracting local temporal features. The complementary representations from the two branches are fused in a dedicated feature-fusion layer to enhance predictive accuracy. As illustrating examples, the model's performance is systematically evaluated on two representative tasks in the Lorenz system. The first is autonomous evolution prediction, in which the model recursively extrapolates system trajectories from the time-delay embeddings of the state vector to evaluate long-term tracking accuracy and stability. The second is inference of unmeasured variable, where the model reconstructs the unobserved states from the time-delay embeddings of partial observations to assess its state-completion capability. The results consistently indicate that the proposed hybrid framework outperforms both single-branch architectures across tasks, demonstrating its robustness and effectiveness in chaotic system prediction.