Rethinking the Role of LLMs in Time Series Forecasting

TL;DR

This study demonstrates that large language models significantly enhance time series forecasting performance across diverse scenarios, especially in cross-domain settings, challenging prior negative assessments and offering practical insights for model design.

Contribution

The paper provides the first large-scale evaluation of LLMs in time series forecasting, showing their benefits and clarifying conditions for effective use.

Findings

LLMs improve forecasting accuracy across multiple scenarios.

Pre-alignment strategies outperform post-alignment in most tasks.

Pretraining and architecture both contribute critically to performance.

Abstract

Large language models (LLMs) have been introduced to time series forecasting (TSF) to incorporate contextual knowledge beyond numerical signals. However, existing studies question whether LLMs provide genuine benefits, often reporting comparable performance without LLMs. We show that such conclusions stem from limited evaluation settings and do not hold at scale. We conduct a large-scale study of LLM-based TSF (LLM4TSF) across 8 billion observations, 17 forecasting scenarios, 4 horizons, multiple alignment strategies, and both in-domain and out-of-domain settings. Our results demonstrate that \emph{LLM4TS indeed improves forecasting performance}, with especially large gains in cross-domain generalization. Pre-alignment outperforming post-alignment in over 90\% of tasks. Both pretrained knowledge and model architecture of LLMs contribute and play complementary roles: pretraining is critical under distribution shifts, while architecture excels at modeling complex temporal dynamics. Moreover, under large-scale mixed distributions, a fully intact LLM becomes indispensable, as confirmed by token-level routing analysis and prompt-based improvements. Overall, Our findings overturn prior negative assessments, establish clear conditions under which LLMs are not only useful, and provide practical guidance for effective model design. We release our code at https://github.com/EIT-NLP/LLM4TSF.