TiCT: A Synthetically Pre-Trained Foundation Model for Time Series Classification

TL;DR

TiCT is a transformer-based foundation model pre-trained on synthetic data for versatile, in-context time series classification, reducing the need for labeled data and retraining.

Contribution

Introduces TiCT, a novel synthetic pre-training framework with a scalable architecture for in-context classification of time series data.

Findings

Achieves competitive performance on UCR Archive datasets.

Operates without updating model weights during inference.

Handles arbitrary class numbers with a new label encoding mechanism.

Abstract

The ubiquity of time series data creates a strong demand for general-purpose foundation models, yet developing them for classification remains a significant challenge, largely due to the high cost of labeled data. Foundation models capable of in-context learning (ICL) offer a powerful solution, adapting to new tasks with minimal examples and reducing the need for extensive retraining. However, prior work on large-scale time series models has predominantly focused on forecasting, leaving a critical gap for versatile, fine-tuning-free classification. To address this, we introduce TiCT (Time-series in-Context Transformer), a transformer-based model pre-trained exclusively on synthetic data to perform in-context classification. We make two primary technical contributions: 1) a novel architecture featuring a scalable bit-based label encoding and a special output attention mechanism to handle an arbitrary number of classes; and 2) a synthetic pre-training framework that combines a Mixup-inspired process with data augmentation to foster generalization and noise invariance. Extensive evaluations on the UCR Archive show that TiCT achieves competitive performance against state-of-the-art supervised methods. Crucially, this is accomplished using only in-context examples at inference time, without updating a single model weight.