TimeGPT in Load Forecasting: A Large Time Series Model Perspective

TL;DR

This paper introduces TimeGPT, a large time series model inspired by LLMs, which is pre-trained on diverse datasets and fine-tuned for load forecasting with scarce data, showing improved accuracy over traditional models.

Contribution

The paper presents a novel large time series model, TimeGPT, trained on massive datasets and adapted for load forecasting with limited historical data, demonstrating its potential in this domain.

Findings

TimeGPT outperforms traditional models on several real datasets.

Fine-tuning improves load forecasting accuracy with scarce data.

Performance varies depending on data distribution differences.

Abstract

Machine learning models have made significant progress in load forecasting, but their forecast accuracy is limited in cases where historical load data is scarce. Inspired by the outstanding performance of large language models (LLMs) in computer vision and natural language processing, this paper aims to discuss the potential of large time series models in load forecasting with scarce historical data. Specifically, the large time series model is constructed as a time series generative pre-trained transformer (TimeGPT), which is trained on massive and diverse time series datasets consisting of 100 billion data points (e.g., finance, transportation, banking, web traffic, weather, energy, healthcare, etc.). Then, the scarce historical load data is used to fine-tune the TimeGPT, which helps it to adapt to the data distribution and characteristics associated with load forecasting. Simulation results show that TimeGPT outperforms the benchmarks (e.g., popular machine learning models and statistical models) for load forecasting on several real datasets with scarce training samples, particularly for short look-ahead times. However, it cannot be guaranteed that TimeGPT is always superior to benchmarks for load forecasting with scarce data, since the performance of TimeGPT may be affected by the distribution differences between the load data and the training data. In practical applications, we can divide the historical data into a training set and a validation set, and then use the validation set loss to decide whether TimeGPT is the best choice for a specific dataset.