Self-Supervised Learning of Disentangled Representations for Multivariate Time-Series

TL;DR

This paper introduces TimeDRL, a self-supervised framework for learning disentangled representations of multivariate time-series data, improving downstream tasks like forecasting and classification without relying on data augmentation.

Contribution

The paper proposes a novel dual-level disentangled embedding approach for multivariate time-series, addressing limitations of existing self-supervised methods in representation learning.

Findings

TimeDRL outperforms existing methods on forecasting and classification tasks.

It effectively learns disentangled timestamp and instance embeddings.

Validation shows robustness in semi-supervised scenarios with limited labels.

Abstract

Multivariate time-series data in fields like healthcare and industry are informative but challenging due to high dimensionality and lack of labels. Recent self-supervised learning methods excel in learning rich representations without labels but struggle with disentangled embeddings and inductive bias issues like transformation-invariance. To address these challenges, we introduce TimeDRL, a framework for multivariate time-series representation learning with dual-level disentangled embeddings. TimeDRL features: (i) disentangled timestamp-level and instance-level embeddings using a [CLS] token strategy; (ii) timestamp-predictive and instance-contrastive tasks for representation learning; and (iii) avoidance of augmentation methods to eliminate inductive biases. Experiments on forecasting and classification datasets show TimeDRL outperforms existing methods, with further validation in semi-supervised settings with limited labeled data.