UniTS: A Unified Multi-Task Time Series Model

TL;DR

UniTS is a versatile multi-task time series model that unifies predictive and generative tasks, leveraging a modified transformer architecture to transfer knowledge across diverse domains and outperform specialized models.

Contribution

The paper introduces UniTS, a novel unified multi-task time series model that integrates various tasks using task tokenization and a modified transformer, enabling transferability across heterogeneous datasets.

Findings

Outperforms 12 forecasting, 20 classification, 18 anomaly detection, and 16 imputation models.

Demonstrates strong few-shot and prompt capabilities in new domains.

Excels in single-task settings, surpassing task-specific models.

Abstract

Although pre-trained transformers and reprogrammed text-based LLMs have shown strong performance on time series tasks, the best-performing architectures vary widely across tasks, with most models narrowly focused on specific areas, such as time series forecasting. Unifying predictive and generative time series tasks within a single model remains challenging. We introduce UniTS, a unified multi-task time series model that utilizes task tokenization to integrate predictive and generative tasks into a single framework. UniTS employs a modified transformer block to capture universal time series representations, enabling transferability from a heterogeneous, multi-domain pre-training dataset-characterized by diverse dynamic patterns, sampling rates, and temporal scales-to a wide range of downstream datasets with varied task specifications and data domains. Tested on 38 datasets across human activity sensors, healthcare, engineering, and finance, UniTS achieves superior performance compared to 12 forecasting models, 20 classification models, 18 anomaly detection models, and 16 imputation models, including adapted text-based LLMs. UniTS also demonstrates strong few-shot and prompt capabilities when applied to new domains and tasks. In single-task settings, UniTS outperforms competitive task-specialized time series models. Code and datasets are available at https://github.com/mims-harvard/UniTS.