BSAT: B-Spline Adaptive Tokenizer for Long-Term Time Series Forecasting

TL;DR

The paper introduces BSAT, a parameter-free B-spline based tokenizer for long-term time series forecasting that adaptively segments data, reducing complexity and improving performance under memory constraints.

Contribution

It proposes a novel adaptive segmentation method using B-splines and a hybrid positional encoding, enhancing transformer efficiency for long-term forecasting.

Findings

Competitive performance at high compression rates

Effective segmentation in high-curvature regions

Improved efficiency for memory-constrained scenarios

Abstract

Long-term time series forecasting using transformers is hampered by the quadratic complexity of self-attention and the rigidity of uniform patching, which may be misaligned with the data's semantic structure. In this paper, we introduce the \textit{B-Spline Adaptive Tokenizer (BSAT)}, a novel, parameter-free method that adaptively segments a time series by fitting it with B-splines. BSAT algorithmically places tokens in high-curvature regions and represents each variable-length basis function as a fixed-size token, composed of its coefficient and position. Further, we propose a hybrid positional encoding that combines a additive learnable positional encoding with Rotary Positional Embedding featuring a layer-wise learnable base: L-RoPE. This allows each layer to attend to different temporal dependencies. Our experiments on several public benchmarks show that our model is competitive with strong performance at high compression rates. This makes it particularly well-suited for use cases with strong memory constraints.