DB2-TransF: All You Need Is Learnable Daubechies Wavelets for Time Series Forecasting

TL;DR

DB2-TransF introduces a wavelet-based module replacing self-attention in Transformers, capturing multi-scale patterns efficiently for scalable and accurate time series forecasting, outperforming traditional models in resource usage.

Contribution

It proposes a novel Transformer-inspired architecture using learnable Daubechies wavelets, improving scalability and efficiency in time series forecasting.

Findings

Achieves comparable or better accuracy than standard Transformers.

Reduces memory usage significantly.

Demonstrates effectiveness on 13 benchmark datasets.

Abstract

Time series forecasting requires models that can efficiently capture complex temporal dependencies, especially in large-scale and high-dimensional settings. While Transformer-based architectures excel at modeling long-range dependencies, their quadratic computational complexity poses limitations on scalability and adaptability. To overcome these challenges, we introduce DB2-TransF, a novel Transformer-inspired architecture that replaces the self-attention mechanism with a learnable Daubechies wavelet coefficient layer. This wavelet-based module efficiently captures multi-scale local and global patterns and enhances the modeling of correlations across multiple time series for the time series forecasting task. Extensive experiments on 13 standard forecasting benchmarks demonstrate that DB2-TransF achieves comparable or superior predictive accuracy to conventional Transformers, while substantially reducing memory usage for the time series forecasting task. The obtained experimental results position DB2-TransF as a scalable and resource-efficient framework for advanced time series forecasting. Our code is available at https://github.com/SteadySurfdom/DB2-TransF