Large Pre-trained time series models for cross-domain Time series analysis tasks

TL;DR

This paper introduces Large Pre-trained Time-series Models (LPTM), which utilize adaptive segmentation to effectively pre-train on multi-domain datasets, enabling superior zero-shot and fine-tuned performance with less data and training time.

Contribution

The paper presents a novel adaptive segmentation method for pre-training large time-series models across multiple domains, improving efficiency and performance over existing models.

Findings

LPTM matches or exceeds domain-specific models in downstream tasks.

LPTM requires up to 40% less data for training.

LPTM reduces training time by up to 50%.

Abstract

Large pre-trained models have been vital in recent advancements in domains like language and vision, making model training for individual downstream tasks more efficient and provide superior performance. However, tackling time-series analysis tasks usually involves designing and training a separate model from scratch leveraging training data and domain expertise specific to the task. We tackle a significant challenge for pre-training a foundational time-series model from multi-domain time-series datasets: extracting semantically useful tokenized inputs to the model across heterogenous time-series from different domains. We propose Large Pre-trained Time-series Models (LPTM) that introduces a novel method of adaptive segmentation that automatically identifies optimal dataset-specific segmentation strategy during pre-training. This enables LPTM to perform similar to or better than domain-specific state-of-art model when fine-tuned to different downstream time-series analysis tasks and under zero-shot settings. LPTM achieves superior forecasting and time-series classification results taking up to 40% less data and 50% less training time compared to state-of-art baselines. Code: www.github.com/AdityaLab/Samay