ss-Mamba: Semantic-Spline Selective State-Space Model

TL;DR

ss-Mamba is a new time series forecasting model that combines semantic embeddings and spline-based temporal encoding within a selective state-space framework, offering high accuracy and efficiency.

Contribution

It introduces a novel foundation model integrating semantic-aware embeddings and adaptive spline-based encoding in a selective state-space framework for improved forecasting.

Findings

Achieves comparable accuracy to Transformers with linear computational complexity.

Effectively generalizes to unseen series using pretrained semantic embeddings.

Provides interpretable modeling of complex seasonalities and non-stationarities.

Abstract

We propose ss-Mamba, a novel foundation model that enhances time series forecasting by integrating semantic-aware embeddings and adaptive spline-based temporal encoding within a selective state-space modeling framework. Building upon the recent success of Transformer architectures, ss-Mamba adopts the Mamba selective state space model as an efficient alternative that achieves comparable performance while significantly reducing computational complexity from quadratic to linear time. Semantic index embeddings, initialized from pretrained language models, allow effective generalization to previously unseen series through meaningful semantic priors. Additionally, spline-based Kolmogorov-Arnold Networks (KAN) dynamically and interpretably capture complex seasonalities and non-stationary temporal effects, providing a powerful enhancement over conventional temporal feature encodings. Extensive experimental evaluations confirm that ss-Mamba delivers superior accuracy, robustness, and interpretability, demonstrating its capability as a versatile and computationally efficient alternative to traditional Transformer-based models in time-series forecasting.