StretchTime: Adaptive Time Series Forecasting via Symplectic Attention

TL;DR

StretchTime introduces Symplectic Positional Embeddings (SyPE) to enable transformers to adaptively model non-linear, non-stationary time-warped dynamics in time series forecasting, outperforming traditional methods on benchmarks.

Contribution

The paper proposes SyPE, a novel learnable positional encoding based on Hamiltonian mechanics, extending rotary embeddings to handle complex temporal warping in time series forecasting.

Findings

Achieves state-of-the-art results on benchmark datasets.

Demonstrates robustness on non-stationary, time-warped data.

Outperforms existing transformer-based forecasting methods.

Abstract

Transformer architectures have established strong baselines in time series forecasting, yet they typically rely on positional encodings that assume uniform, index-based temporal progression. However, real-world systems, from shifting financial cycles to elastic biological rhythms, frequently exhibit "time-warped" dynamics where the effective flow of time decouples from the sampling index. In this work, we first formalize this misalignment and prove that rotary position embedding (RoPE) is mathematically incapable of representing non-affine temporal warping. To address this, we propose Symplectic Positional Embeddings (SyPE), a learnable encoding framework derived from Hamiltonian mechanics. SyPE strictly generalizes RoPE by extending the rotation group $\mathrm{SO}(2)$ to the symplectic group $\mathrm{Sp}(2,\mathbb{R})$, modulated by a novel input-dependent adaptive warp module. By allowing the attention mechanism to adaptively dilate or contract temporal coordinates end-to-end, our approach captures locally varying periodicities without requiring pre-defined warping functions. We implement this mechanism in StretchTime, a multivariate forecasting architecture that achieves state-of-the-art performance on standard benchmarks, demonstrating superior robustness on datasets exhibiting non-stationary temporal dynamics.