Rethinking Time Series Forecasting with LLMs via Nearest Neighbor Contrastive Learning

TL;DR

This paper introduces NNCL-TLLM, a novel approach that leverages nearest neighbor contrastive learning and token embeddings to improve time series forecasting with large language models, especially in few-shot scenarios.

Contribution

It proposes a new prompt formulation method using text prototypes and contrastive learning, fine-tunes specific LLM layers, and demonstrates superior forecasting performance.

Findings

Outperforms existing methods in few-shot forecasting.

Achieves competitive results in long-term and short-term tasks.

Reduces computational cost by fine-tuning limited LLM layers.

Abstract

Adapting Large Language Models (LLMs) that are extensively trained on abundant text data, and customizing the input prompt to enable time series forecasting has received considerable attention. While recent work has shown great potential for adapting the learned prior of LLMs, the formulation of the prompt to finetune LLMs remains challenging as prompt should be aligned with time series data. Additionally, current approaches do not effectively leverage word token embeddings which embody the rich representation space learned by LLMs. This emphasizes the need for a robust approach to formulate the prompt which utilizes the word token embeddings while effectively representing the characteristics of the time series. To address these challenges, we propose NNCL-TLLM: Nearest Neighbor Contrastive Learning for Time series forecasting via LLMs. First, we generate time series compatible text prototypes such that each text prototype represents both word token embeddings in its neighborhood and time series characteristics via end-to-end finetuning. Next, we draw inspiration from Nearest Neighbor Contrastive Learning to formulate the prompt while obtaining the top-$k$ nearest neighbor time series compatible text prototypes. We then fine-tune the layer normalization and positional embeddings of the LLM, keeping the other layers intact, reducing the trainable parameters and decreasing the computational cost. Our comprehensive experiments demonstrate that NNCL-TLLM outperforms in few-shot forecasting while achieving competitive or superior performance over the state-of-the-art methods in long-term and short-term forecasting tasks.