Enhancing Zero-Shot Time Series Forecasting in Off-the-Shelf LLMs via Noise Injection

TL;DR

This paper proposes a noise injection technique into raw time series data to enhance the zero-shot forecasting capabilities of off-the-shelf large language models without any fine-tuning, improving robustness and performance.

Contribution

Introducing a simple noise injection method during inference to improve zero-shot time series forecasting with frozen LLMs, supported by theoretical analysis and new datasets.

Findings

Noise injection improves forecasting accuracy across benchmarks.

The method enhances robustness against distribution shifts.

New datasets prevent pre-training data contamination.

Abstract

Large Language Models (LLMs) have demonstrated effectiveness as zero-shot time series (TS) forecasters. The key challenge lies in tokenizing TS data into textual representations that align with LLMs' pre-trained knowledge. While existing work often relies on fine-tuning specialized modules to bridge this gap, a distinct, yet challenging, paradigm aims to leverage truly off-the-shelf LLMs without any fine-tuning whatsoever, relying solely on strategic tokenization of numerical sequences. The performance of these fully frozen models is acutely sensitive to the textual representation of the input data, as their parameters cannot adapt to distribution shifts. In this paper, we introduce a simple yet highly effective strategy to overcome this brittleness: injecting noise into the raw time series before tokenization. This non-invasive intervention acts as a form of inference-time augmentation, compelling the frozen LLM to extrapolate based on robust underlying temporal patterns rather than superficial numerical artifacts. We theoretically analyze this phenomenon and empirically validate its effectiveness across diverse benchmarks. Notably, to fully eliminate potential biases from data contamination during LLM pre-training, we introduce two novel TS datasets that fall outside all utilized LLMs' pre-training scopes, and consistently observe improved performance. This study provides a further step in directly leveraging off-the-shelf LLMs for time series forecasting.