Contrastive random lead coding for channel-agnostic self-supervision of biosignals

TL;DR

This paper introduces contrastive random lead coding (CRLC), a novel self-supervised learning method for biosignals that improves model generalization across varying channel configurations by using random channel subsets for positive pair creation.

Contribution

The paper proposes CRLC, a new contrastive learning strategy that enhances channel-agnostic self-supervision for biosignals, outperforming existing methods in EEG and ECG tasks.

Findings

CRLC outperforms competing strategies in channel-agnostic settings.

CRLC surpasses the state-of-the-art in EEG tasks.

CRLC achieves comparable results to the state-of-the-art in ECG tasks.

Abstract

Contrastive learning yields impressive results for self-supervision in computer vision. The approach relies on the creation of positive pairs, something which is often achieved through augmentations. However, for multivariate time series effective augmentations can be difficult to design. Additionally, the number of input channels for biosignal datasets often varies from application to application, limiting the usefulness of large self-supervised models trained with specific channel configurations. Motivated by these challenges, we set out to investigate strategies for creation of positive pairs for channel-agnostic self-supervision of biosignals. We introduce contrastive random lead coding (CRLC), where random subsets of the input channels are used to create positive pairs and compare with using augmentations and neighboring segments in time as positive pairs. We validate our approach by pre-training models on EEG and ECG data, and then fine-tuning them for downstream tasks. CRLC outperforms competing strategies in both scenarios in the channel-agnostic setting. For EEG, the approach additionally outperforms the state-of-the-art reference model. Notably, for EEG tasks CRLC surpasses the current state-of-the-art reference model. While, the state-of-the-art reference model is superior in the ECG task, incorporating CRLC allows us to obtain comparable results. In conclusion, CRLC helps generalization across variable channel setups when training our channel-agnostic model.