In-context Pre-trained Time-Series Foundation Models adapt to Unseen Tasks

TL;DR

This paper introduces a novel in-context learning framework for time-series foundation models, allowing them to adapt to unseen tasks at inference time without fine-tuning, significantly improving their generalization capabilities.

Contribution

The paper proposes In-Context Time-series Pre-training (ICTP), a method that equips TSFMs with in-context learning abilities by restructuring pre-training data, enabling adaptation to new tasks.

Findings

ICTP improves TSFM performance by 11.4% on unseen tasks.

The framework enables test-time adaptation without fine-tuning.

Experiments validate the effectiveness of ICTP across diverse datasets.

Abstract

Time-series foundation models (TSFMs) have demonstrated strong generalization capabilities across diverse datasets and tasks. However, existing foundation models are typically pre-trained to enhance performance on specific tasks and often struggle to generalize to unseen tasks without fine-tuning. To address this limitation, we propose augmenting TSFMs with In-Context Learning (ICL) capabilities, enabling them to perform test-time inference by dynamically adapting to input-output relationships provided within the context. Our framework, In-Context Time-series Pre-training (ICTP), restructures the original pre-training data to equip the backbone TSFM with ICL capabilities, enabling adaptation to unseen tasks. Experiments demonstrate that ICT improves the performance of state-of-the-art TSFMs by approximately 11.4% on unseen tasks without requiring fine-tuning.