Learning Without Augmenting: Unsupervised Time Series Representation Learning via Frame Projections

TL;DR

This paper introduces a novel self-supervised learning method for time series that replaces traditional data augmentations with geometric transformations using orthonormal bases and overcomplete frames, leading to improved representations.

Contribution

It proposes a new unsupervised learning approach that leverages geometric properties of different representation spaces instead of data augmentations, enhancing generalization for temporal data.

Findings

Achieves up to 20% performance improvement over existing methods.

Effective across nine datasets and five temporal sequence tasks.

Does not rely on handcrafted data augmentations.

Abstract

Self-supervised learning (SSL) has emerged as a powerful paradigm for learning representations without labeled data. Most SSL approaches rely on strong, well-established, handcrafted data augmentations to generate diverse views for representation learning. However, designing such augmentations requires domain-specific knowledge and implicitly imposes representational invariances on the model, which can limit generalization. In this work, we propose an unsupervised representation learning method that replaces augmentations by generating views using orthonormal bases and overcomplete frames. We show that embeddings learned from orthonormal and overcomplete spaces reside on distinct manifolds, shaped by the geometric biases introduced by representing samples in different spaces. By jointly leveraging the complementary geometry of these distinct manifolds, our approach achieves superior performance without artificially increasing data diversity through strong augmentations. We demonstrate the effectiveness of our method on nine datasets across five temporal sequence tasks, where signal-specific characteristics make data augmentations particularly challenging. Without relying on augmentation-induced diversity, our method achieves performance gains of up to 15--20\% over existing self-supervised approaches. Source code: https://github.com/eth-siplab/Learning-with-FrameProjections