Pre-trained Forecasting Models: Strong Zero-Shot Feature Extractors for Time Series Classification

TL;DR

This paper demonstrates that pre-trained forecasting models can serve as effective zero-shot feature extractors for time series classification, often outperforming models trained specifically for classification tasks.

Contribution

It introduces a novel approach of using frozen forecasting models for classification and compares various embedding strategies, showing their strong generalization capabilities.

Findings

Forecasting models achieve classification accuracy comparable or superior to specialized models.

Positive correlation observed between forecasting and classification performance.

Model-agnostic embedding augmentations enhance classification results.

Abstract

Recent research on time series foundation models has primarily focused on forecasting, leaving it unclear how generalizable their learned representations are. In this study, we examine whether frozen pre-trained forecasting models can provide effective representations for classification. To this end, we compare different representation extraction strategies and introduce two model-agnostic embedding augmentations. Our experiments show that the best forecasting models achieve classification accuracy that matches or even surpasses that of state-of-the-art models pre-trained specifically for classification. Moreover, we observe a positive correlation between forecasting and classification performance. These findings challenge the assumption that task-specific pre-training is necessary, and suggest that learning to forecast may provide a powerful route toward constructing general-purpose time series foundation models.