COTN: A Chaotic Oscillatory Transformer Network for Complex Volatile Systems under Extreme Conditions

TL;DR

COTN is a novel Transformer-based model with a chaotic oscillatory activation and an autoencoder module, designed to improve prediction accuracy and robustness in highly volatile and chaotic financial and electricity markets.

Contribution

The paper introduces COTN, combining a new Lee Oscillator activation with an Autoencoder Self-Regressive module, enhancing modeling of chaotic dynamics and abnormal market patterns.

Findings

Outperforms state-of-the-art models like Informer by up to 17%.

Achieves up to 40% better accuracy than traditional GARCH models.

Demonstrates robustness and effectiveness in real-world market scenarios.

Abstract

Accurate prediction of financial and electricity markets, especially under extreme conditions, remains a significant challenge due to their intrinsic nonlinearity, rapid fluctuations, and chaotic patterns. To address these limitations, we propose the Chaotic Oscillatory Transformer Network (COTN). COTN innovatively combines a Transformer architecture with a novel Lee Oscillator activation function, processed through Max-over-Time pooling and a lambda-gating mechanism. This design is specifically tailored to effectively capture chaotic dynamics and improve responsiveness during periods of heightened volatility, where conventional activation functions (e.g., ReLU, GELU) tend to saturate. Furthermore, COTN incorporates an Autoencoder Self-Regressive (ASR) module to detect and isolate abnormal market patterns, such as sudden price spikes or crashes, thereby preventing corruption of the core prediction process and enhancing robustness. Extensive experiments across electricity spot markets and financial markets demonstrate the practical applicability and resilience of COTN. Our approach outperforms state-of-the-art deep learning models like Informer by up to 17% and traditional statistical methods like GARCH by as much as 40%. These results underscore COTN's effectiveness in navigating real-world market uncertainty and complexity, offering a powerful tool for forecasting highly volatile systems under duress.