NuwaTS: a Foundation Model Mending Every Incomplete Time Series

TL;DR

NuwaTS is a versatile framework that adapts pre-trained language models for general time series imputation, capable of handling diverse domains and missing patterns with minimal fine-tuning, outperforming domain-specific models.

Contribution

The paper introduces NuwaTS, a novel PLM-based framework for time series imputation that generalizes across domains and variables, with a new benchmarking protocol for cross-domain evaluation.

Findings

NuwaTS outperforms state-of-the-art models on diverse datasets.

It generalizes well to unseen variables and domains.

The framework also benefits other tasks like forecasting.

Abstract

Time series imputation is critical for many real-world applications and has been widely studied. However, existing models often require specialized designs tailored to specific missing patterns, variables, or domains which limits their generalizability. In addition, current evaluation frameworks primarily focus on domain-specific tasks and often rely on time-wise train/validation/test data splits, which fail to rigorously assess a model's ability to generalize across unseen variables or domains. In this paper, we present \textbf{NuwaTS}, a novel framework that repurposes Pre-trained Language Models (PLMs) for general time series imputation. Once trained, NuwaTS can be applied to impute missing data across any domain. We introduce specialized embeddings for each sub-series patch, capturing information about the patch, its missing data patterns, and its statistical characteristics. By combining contrastive learning with the imputation task, we train PLMs to create a versatile, one-for-all imputation model. Additionally, we employ a plug-and-play fine-tuning approach, enabling efficient adaptation to domain-specific tasks with minimal adjustments. To evaluate cross-variable and cross-domain generalization, we propose a new benchmarking protocol that partitions the datasets along the variable dimension. Experimental results on over seventeen million time series samples from diverse domains demonstrate that NuwaTS outperforms state-of-the-art domain-specific models across various datasets under the proposed benchmarking protocol. Furthermore, we show that NuwaTS generalizes to other time series tasks, such as forecasting. Our codes are available at https://github.com/Chengyui/NuwaTS.