Context information can be more important than reasoning for time series forecasting with a large language model

TL;DR

This paper shows that providing relevant context information is often more effective than complex reasoning prompts for time series forecasting with large language models, highlighting the importance of context over reasoning.

Contribution

It demonstrates that simple context provision can match advanced prompting techniques in time series forecasting, revealing new insights into prompt design and LLM limitations.

Findings

Context information can outperform complex prompts.

LLMs struggle with procedural following and calculations.

Prompt semantics are often misunderstood by LLMs.

Abstract

With the evolution of large language models (LLMs), there is growing interest in leveraging LLMs for time series tasks. In this paper, we explore the characteristics of LLMs for time series forecasting by considering various existing and proposed prompting techniques. Forecasting for both short and long time series was evaluated. Our findings indicate that no single prompting method is universally applicable. It was also observed that simply providing proper context information related to the time series, without additional reasoning prompts, can achieve performance comparable to the best-performing prompt for each case. From this observation, it is expected that providing proper context information can be more crucial than a prompt for specific reasoning in time series forecasting. Several weaknesses in prompting for time series forecasting were also identified. First, LLMs often fail to follow the procedures described by the prompt. Second, when reasoning steps involve simple algebraic calculations with several operands, LLMs often fail to calculate accurately. Third, LLMs sometimes misunderstand the semantics of prompts, resulting in incomplete responses.