TS-Arena -- A Live Forecast Pre-Registration Platform

TL;DR

TS-Arena is a live forecasting platform that evaluates models on future data with strict pre-registration, preventing data leakage and enabling continuous, real-time benchmarking of time series models.

Contribution

It introduces a novel live benchmarking platform with a pre-registration protocol that assesses models on future data, enhancing evaluation integrity and immediacy.

Findings

Established TSFMs show robust performance over time.

Newcomers can demonstrate immediate competitiveness.

The platform prevents test-set contamination.

Abstract

Time Series Foundation Models (TSFMs) are transforming the field of forecasting. However, evaluating them on historical data is increasingly difficult due to the risks of train-test sample overlaps and temporal overlaps between correlated train and test time series. To address this, we introduce TS-Arena, a live forecasting platform that shifts evaluation from the known past to the unknown future. Building on the concept of continuous benchmarking, TS-Arena evaluates models on future data. Crucially, we introduce a strict forecasting pre-registration protocol: models must submit predictions before the ground-truth data physically exists. This makes test-set contamination impossible by design. The platform relies on a modular microservice architecture that harmonizes and structures data from different sources and orchestrates containerized model submissions. By enforcing a strict pre-registration protocol on live data streams, TS-Arena prevents information leakage offers a faster alternative to traditional static, infrequently repeated competitions (e.g. the M-Competitions). First empirical results derived from operating TS-Arena over one year of energy time series demonstrate that established TSFMs accumulate robust longitudinal scores over time, while the continuous nature of the benchmark simultaneously allows newcomers to demonstrate immediate competitiveness. TS-Arena provides the necessary infrastructure to assess the true generalization capabilities of modern forecasting models. The platform and corresponding code are available at https://ts-arena.live/.