Time to Embed: Unlocking Foundation Models for Time Series with Channel Descriptions

TL;DR

This paper introduces CHARM, a foundation model for multivariate time series that leverages channel descriptions and innovative training techniques to achieve state-of-the-art results across various tasks.

Contribution

The paper presents CHARM, a novel foundation embedding model for time series that incorporates channel descriptions and a new training architecture, advancing beyond traditional task-specific models.

Findings

Achieves state-of-the-art performance on multiple downstream tasks.

Effectively incorporates channel descriptions for improved representations.

Demonstrates robustness and interpretability in time series modeling.

Abstract

Traditional time series models are task-specific and often depend on dataset-specific training and extensive feature engineering. While Transformer-based architectures have improved scalability, foundation models, commonplace in text, vision, and audio, remain under-explored for time series and are largely restricted to forecasting. We introduce $\textbf{CHARM}$, a foundation embedding model for multivariate time series that learns shared, transferable, and domain-aware representations. To address the unique difficulties of time series foundation learning, $\textbf{CHARM}$ incorporates architectural innovations that integrate channel-level textual descriptions while remaining invariant to channel order. The model is trained using a Joint Embedding Predictive Architecture (JEPA), with novel augmentation schemes and a loss function designed to improve interpretability and training stability. Our $7$M-parameter model achieves state-of-the-art performance across diverse downstream tasks, setting a new benchmark for time series representation learning.