UTICA: Multi-Objective Self-Distllation Foundation Model Pretraining for Time Series Classification

TL;DR

This paper introduces UTICA, a non-contrastive self-distillation approach for pretraining time series foundation models, achieving state-of-the-art classification results and demonstrating the effectiveness of this paradigm in time series analysis.

Contribution

UTICA adapts DINOv2-style self-distillation for time series, combining temporal invariance and local structure learning, a novel application in this domain.

Findings

Achieves state-of-the-art results on UCR and UEA benchmarks.

Demonstrates the effectiveness of non-contrastive pretraining for time series.

Highlights potential of self-distillation in time series foundation models.

Abstract

Self-supervised foundation models have achieved remarkable success across domains, including time series. However, the potential of non-contrastive methods, a paradigm that has driven significant advances in computer vision, remains underexplored for time series. In this work, we adapt DINOv2-style self-distillation to pretrain a time series foundation model, building on the Mantis tokenizer and transformer encoder architecture as our backbone. Through a student-teacher framework, our method Utica learns representations that capture both temporal invariance via augmented crops and fine-grained local structure via patch masking. Our approach achieves state-of-the-art classification performance on both UCR and UEA benchmarks. These results suggest that non-contrastive methods are a promising and complementary pretraining strategy for time series foundation models.