From Token to Rhythm: A Multi-Scale Approach for ECG-Language Pretraining

TL;DR

This paper introduces MELP, a multi-scale ECG-language pretraining model that captures hierarchical ECG features at token, beat, and rhythm levels to improve ECG analysis and diagnosis.

Contribution

MELP is the first to leverage hierarchical supervision across multiple time scales for ECG-language pretraining, enhancing representation learning for clinical tasks.

Findings

Outperforms existing SSL methods on multiple ECG tasks

Effective in zero-shot ECG classification and transfer learning

Captures multi-scale ECG features for better generalization

Abstract

Electrocardiograms (ECGs) play a vital role in monitoring cardiac health and diagnosing heart diseases. However, traditional deep learning approaches for ECG analysis rely heavily on large-scale manual annotations, which are both time-consuming and resource-intensive to obtain. To overcome this limitation, self-supervised learning (SSL) has emerged as a promising alternative, enabling the extraction of robust ECG representations that can be efficiently transferred to various downstream tasks. While previous studies have explored SSL for ECG pretraining and multi-modal ECG-language alignment, they often fail to capture the multi-scale nature of ECG signals. As a result, these methods struggle to learn generalized representations due to their inability to model the hierarchical structure of ECG data. To address this gap, we introduce MELP, a novel Multi-scale ECG-Language Pretraining (MELP) model that fully leverages hierarchical supervision from ECG-text pairs. MELP first pretrains a cardiology-specific language model to enhance its understanding of clinical text. It then applies three levels of cross-modal supervision-at the token, beat, and rhythm levels-to align ECG signals with textual reports, capturing structured information across different time scales. We evaluate MELP on three public ECG datasets across multiple tasks, including zero-shot ECG classification, linear probing, and transfer learning. Experimental results demonstrate that MELP outperforms existing SSL methods, underscoring its effectiveness and adaptability across diverse clinical applications. Our code is available at https://github.com/HKU-MedAI/MELP.