Evaluating Time Series Foundation Models on Noisy Periodic Time Series

TL;DR

This paper empirically evaluates the zero-shot forecasting performance of time series foundation models on noisy periodic data, revealing their strengths and limitations across various noise and sampling conditions.

Contribution

It provides a systematic assessment of TSFMs' capabilities on synthetic noisy periodic time series, highlighting their performance boundaries compared to traditional statistical methods.

Findings

TSFMs perform well with bounded periods and high sampling rates.

Forecasting deteriorates with increased noise and longer periods.

Traditional methods like FFT and AR remain competitive under certain conditions.

Abstract

While recent advancements in foundation models have significantly impacted machine learning, rigorous tests on the performance of time series foundation models (TSFMs) remain largely underexplored. This paper presents an empirical study evaluating the zero-shot, long-horizon forecasting abilities of several leading TSFMs over two synthetic datasets constituting noisy periodic time series. We assess model efficacy across different noise levels, underlying frequencies, and sampling rates. As benchmarks for comparison, we choose two statistical techniques: a Fourier transform (FFT)-based approach and a linear autoregressive (AR) model. Our findings demonstrate that while for time series with bounded periods and higher sampling rates, TSFMs can match or outperform the statistical approaches, their forecasting abilities deteriorate with longer periods, higher noise levels, lower sampling rates and more complex shapes of the time series.