Tiny-TSM: Efficiently Training a Lightweight SOTA Time Series Foundation Model

TL;DR

Tiny-TSM is a small, efficient, and high-performing time series foundation model trained on synthetic data, achieving state-of-the-art results with minimal resources and no extensive tuning.

Contribution

The paper introduces Tiny-TSM, a lightweight time series foundation model trained efficiently on synthetic data, matching or surpassing larger models without extensive tuning.

Findings

Outperforms larger models on medium- and long-term forecasting

Achieves state-of-the-art performance with only 23M parameters

Trained efficiently on a single GPU in less than a week

Abstract

We present Tiny-TSM, a time series foundation model characterized by small scale, economical training, and state-of-the-art performance. It comprises 23M total parameters, trained on a single A100 GPU in less than a week using a new synthetic data generation and data augmentation pipeline (SynthTS). Without any neural architecture search, hyperparameter tuning, or scaling up model size, Tiny-TSM achieves state-of-the-art performance on a wide range of time series benchmark datasets, often outperforming much larger models and even matching the performance of much larger, industrial-scale, likely highly tuned foundation models. Specifically, Tiny-TSM outperforms all other time series foundation models we evaluated on medium- and long-term forecasting tasks under MSE loss, while short-term accuracy is still competitive with state-of-the-art models. We also introduce a causal input normalization scheme that enables time series models to be trained with dense next-token prediction loss, significantly accelerating convergence speed and reducing training time. All experiments were conducted on a single A100 GPU, illustrating the practicality of the proposed approach in a resource-constrained setting.