Contrastive Learning Is Not Optimal for Quasiperiodic Time Series

TL;DR

This paper introduces DEAPS, a non-contrastive learning method for quasiperiodic time series like ECGs, which improves representation learning and classification accuracy with limited labeled data.

Contribution

The paper proposes DEAPS, a novel non-contrastive approach that captures dynamic patterns in quasiperiodic time series, outperforming contrastive methods especially with few labels.

Findings

DEAPS achieves +10% accuracy over SOTA with limited labeled data.

Avoiding negative pairs helps the model focus on temporal changes.

The Gradual Loss enhances dynamic pattern capturing.

Abstract

Despite recent advancements in Self-Supervised Learning (SSL) for time series analysis, a noticeable gap persists between the anticipated achievements and actual performance. While these methods have demonstrated formidable generalization capabilities with minimal labels in various domains, their effectiveness in distinguishing between different classes based on a limited number of annotated records is notably lacking. Our hypothesis attributes this bottleneck to the prevalent use of Contrastive Learning, a shared training objective in previous state-of-the-art (SOTA) methods. By mandating distinctiveness between representations for negative pairs drawn from separate records, this approach compels the model to encode unique record-based patterns but simultaneously neglects changes occurring across the entire record. To overcome this challenge, we introduce Distilled Embedding for Almost-Periodic Time Series (DEAPS) in this paper, offering a non-contrastive method tailored for quasiperiodic time series, such as electrocardiogram (ECG) data. By avoiding the use of negative pairs, we not only mitigate the model's blindness to temporal changes but also enable the integration of a "Gradual Loss (Lgra)" function. This function guides the model to effectively capture dynamic patterns evolving throughout the record. The outcomes are promising, as DEAPS demonstrates a notable improvement of +10% over existing SOTA methods when just a few annotated records are presented to fit a Machine Learning (ML) model based on the learned representation.