Cross-RAG: Zero-Shot Retrieval-Augmented Time Series Forecasting via Cross-Attention

TL;DR

Cross-RAG introduces a novel zero-shot time series forecasting framework that uses cross-attention to selectively incorporate relevant external data, significantly enhancing forecasting accuracy across diverse models and scenarios.

Contribution

It presents a new retrieval-augmented framework with query-relevance modeling for improved zero-shot time series forecasting.

Findings

Consistently improves forecasting performance across models.

Effective across various retrieval scenarios.

Enhances zero-shot generalization in time series forecasting.

Abstract

Recent advances in time series foundation models (TSFMs) demonstrate strong expressive capacity through large-scale pretraining across diverse time series domains. Zero-shot time series forecasting with TSFMs, however, exhibits limited generalization to unseen datasets, which retrieval-augmented forecasting addresses by leveraging an external knowledge base. Existing approaches rely on a fixed number of retrieved samples that may introduce irrelevant information. To this end, we propose Cross-RAG, a zero-shot retrieval-augmented forecasting framework that selectively attends to query-relevant retrieved samples. Cross-RAG models input-level relevance between the query and retrieved samples via query-retrieval cross-attention, while jointly incorporating information from the query and retrieved samples. Extensive experiments demonstrate that Cross-RAG consistently improves zero-shot forecasting performance across various TSFMs and RAG methods, and additional analyses confirm its effectiveness across diverse retrieval scenarios. Code is available at https://github.com/seunghan96/cross-rag/.