SuPreME: A Supervised Pre-training Framework for Multimodal ECG Representation Learning

TL;DR

SuPreME is a supervised pre-training framework that enhances multimodal ECG representations by integrating structured clinical labels and textual queries, enabling zero-shot classification and outperforming existing self-supervised methods.

Contribution

The paper introduces SuPreME, a novel supervised pre-training approach that leverages structured diagnostic labels and textual cardiac queries for improved ECG representation learning.

Findings

Achieves 77.20% zero-shot AUC on six datasets.

Surpasses state-of-the-art eSSL methods by 4.98%.

Effectively leverages clinical knowledge for ECG analysis.

Abstract

Cardiovascular diseases are a leading cause of death and disability worldwide. Electrocardiogram (ECG) is critical for diagnosing and monitoring cardiac health, but obtaining large-scale annotated ECG datasets is labor-intensive and time-consuming. Recent ECG Self-Supervised Learning (eSSL) methods mitigate this by learning features without extensive labels but fail to capture fine-grained clinical semantics and require extensive task-specific fine-tuning. To address these challenges, we propose $\textbf{SuPreME}$, a $\textbf{Su}$pervised $\textbf{Pre}$-training framework for $\textbf{M}$ultimodal $\textbf{E}$CG representation learning. SuPreME is pre-trained using structured diagnostic labels derived from ECG report entities through a one-time offline extraction with Large Language Models (LLMs), which help denoise, standardize cardiac concepts, and improve clinical representation learning. By fusing ECG signals with textual cardiac queries instead of fixed labels, SuPreME enables zero-shot classification of unseen conditions without further fine-tuning. We evaluate SuPreME on six downstream datasets covering 106 cardiac conditions, achieving superior zero-shot AUC performance of $77.20\%$, surpassing state-of-the-art eSSLs by $4.98\%$. Results demonstrate SuPreME's effectiveness in leveraging structured, clinically relevant knowledge for high-quality ECG representations.