Mixing Up Contrastive Learning: Self-Supervised Representation Learning for Time Series

TL;DR

This paper introduces a novel contrastive learning method for time series that uses data mixing and label smoothing to improve unsupervised representation quality, especially for transfer learning in medical applications.

Contribution

It proposes a new contrastive loss with data mixing and soft targets, enhancing unsupervised time series representation learning.

Findings

Outperforms existing methods on univariate and multivariate time series

Improves transfer learning performance in clinical time series

Demonstrates effectiveness through extensive experiments

Abstract

The lack of labeled data is a key challenge for learning useful representation from time series data. However, an unsupervised representation framework that is capable of producing high quality representations could be of great value. It is key to enabling transfer learning, which is especially beneficial for medical applications, where there is an abundance of data but labeling is costly and time consuming. We propose an unsupervised contrastive learning framework that is motivated from the perspective of label smoothing. The proposed approach uses a novel contrastive loss that naturally exploits a data augmentation scheme in which new samples are generated by mixing two data samples with a mixing component. The task in the proposed framework is to predict the mixing component, which is utilized as soft targets in the loss function. Experiments demonstrate the framework's superior performance compared to other representation learning approaches on both univariate and multivariate time series and illustrate its benefits for transfer learning for clinical time series.