In-Context and Few-Shots Learning for Forecasting Time Series Data based on Large Language Models

TL;DR

This paper evaluates the effectiveness of large language models and foundation models like TimesFM in forecasting time series data, comparing their performance with traditional deep learning models such as LSTM and TCN.

Contribution

It demonstrates that pre-trained time series foundation models, especially TimesFM, outperform traditional models in accuracy and inference time, highlighting their potential for real-time forecasting.

Findings

TimesFM achieves the lowest RMSE (0.3023) among tested models.

OpenAI's o4-mini performs well in zero-shot settings.

Foundation models show promise for scalable, real-time time series forecasting.

Abstract

Existing data-driven approaches in modeling and predicting time series data include ARIMA (Autoregressive Integrated Moving Average), Transformer-based models, LSTM (Long Short-Term Memory) and TCN (Temporal Convolutional Network). These approaches, and in particular deep learning-based models such as LSTM and TCN, have shown great results in predicting time series data. With the advancement of leveraging pre-trained foundation models such as Large Language Models (LLMs) and more notably Google's recent foundation model for time series data, {\it TimesFM} (Time Series Foundation Model), it is of interest to investigate whether these foundation models have the capability of outperforming existing modeling approaches in analyzing and predicting time series data. This paper investigates the performance of using LLM models for time series data prediction. We investigate the in-context learning methodology in the training of LLM models that are specific to the underlying application domain. More specifically, the paper explores training LLMs through in-context, zero-shot and few-shot learning and forecasting time series data with OpenAI {\tt o4-mini} and Gemini 2.5 Flash Lite, as well as the recent Google's Transformer-based TimesFM, a time series-specific foundation model, along with two deep learning models, namely TCN and LSTM networks. The findings indicate that TimesFM has the best overall performance with the lowest RMSE value (0.3023) and the competitive inference time (266 seconds). Furthermore, OpenAI's o4-mini also exhibits a good performance based on Zero Shot learning. These findings highlight pre-trained time series foundation models as a promising direction for real-time forecasting, enabling accurate and scalable deployment with minimal model adaptation.