Revisiting LLMs as Zero-Shot Time-Series Forecasters: Small Noise Can Break Large Models

TL;DR

This paper critically evaluates the use of Large Language Models for zero-shot time-series forecasting, revealing their sensitivity to noise and suggesting fine-tuning as a more effective approach than zero-shot prompting.

Contribution

The study provides a rigorous comparison of LLMs with domain-specific models for zero-shot forecasting and explores noise robustness solutions, highlighting the limitations of LLMs in this task.

Findings

LLMs often underperform compared to domain-specific models in zero-shot forecasting.

Noise sensitivity significantly hampers LLM accuracy in time-series prediction.

Fine-tuning LLMs may offer a more promising path than zero-shot methods.

Abstract

Large Language Models (LLMs) have shown remarkable performance across diverse tasks without domain-specific training, fueling interest in their potential for time-series forecasting. While LLMs have shown potential in zero-shot forecasting through prompting alone, recent studies suggest that LLMs lack inherent effectiveness in forecasting. Given these conflicting findings, a rigorous validation is essential for drawing reliable conclusions. In this paper, we evaluate the effectiveness of LLMs as zero-shot forecasters compared to state-of-the-art domain-specific models. Our experiments show that LLM-based zero-shot forecasters often struggle to achieve high accuracy due to their sensitivity to noise, underperforming even simple domain-specific models. We have explored solutions to reduce LLMs' sensitivity to noise in the zero-shot setting, but improving their robustness remains a significant challenge. Our findings suggest that rather than emphasizing zero-shot forecasting, a more promising direction would be to focus on fine-tuning LLMs to better process numerical sequences. Our experimental code is available at https://github.com/junwoopark92/revisiting-LLMs-zeroshot-forecaster.