HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis

TL;DR

HeartBERT is a self-supervised ECG embedding model inspired by BERT, designed to improve medical signal analysis by reducing labeled data needs, computational resources, and enhancing performance.

Contribution

The paper introduces HeartBERT, a novel self-supervised ECG embedding model based on RoBERTa, optimized for medical signal analysis with improved efficiency and effectiveness.

Findings

HeartBERT performs well with smaller datasets.

It reduces training parameters compared to rivals.

It achieves superior results in sleep stage detection and heartbeat classification.

Abstract

The HeartBert model is introduced with three primary objectives: reducing the need for labeled data, minimizing computational resources, and simultaneously improving performance in machine learning systems that analyze Electrocardiogram (ECG) signals. Inspired by Bidirectional Encoder Representations from Transformers (BERT) in natural language processing and enhanced with a self-supervised learning approach, the HeartBert model-built on the RoBERTa architecture-generates sophisticated embeddings tailored for ECG-based projects in the medical domain. To demonstrate the versatility, generalizability, and efficiency of the proposed model, two key downstream tasks have been selected: sleep stage detection and heartbeat classification. HeartBERT-based systems, utilizing bidirectional LSTM heads, are designed to address complex challenges. A series of practical experiments have been conducted to demonstrate the superiority and advancements of HeartBERT, particularly in terms of its ability to perform well with smaller training datasets, reduced learning parameters, and effective performance compared to rival models. The code and data are publicly available at https://github.com/ecgResearch/HeartBert.