Measuring Time-Series Dataset Similarity using Wasserstein Distance

TL;DR

This paper introduces a novel distribution-based method using Wasserstein distance to measure similarity between time-series datasets, aiding model selection, transfer learning, and dataset analysis.

Contribution

It proposes a new approach that models time-series datasets as multivariate normal distributions and computes their similarity via Wasserstein distance, demonstrating effectiveness through experiments.

Findings

High correlation (>0.60) between Wasserstein distance and inference loss.

Effective identification of similar datasets for model transfer.

Facilitates dataset comparison and inference performance estimation.

Abstract

The emergence of time-series foundation model research elevates the growing need to measure the (dis)similarity of time-series datasets. A time-series dataset similarity measure aids research in multiple ways, including model selection, finetuning, and visualization. In this paper, we propose a distribution-based method to measure time-series dataset similarity by leveraging the Wasserstein distance. We consider a time-series dataset an empirical instantiation of an underlying multivariate normal distribution (MVN). The similarity between two time-series datasets is thus computed as the Wasserstein distance between their corresponding MVNs. Comprehensive experiments and visualization show the effectiveness of our approach. Specifically, we show how the Wasserstein distance helps identify similar time-series datasets and facilitates inference performance estimation of foundation models in both out-of-distribution and transfer learning evaluation, with high correlations between our proposed measure and the inference loss (>0.60).