Encoder-Free ECG-Language Models

TL;DR

ELF is an encoder-free ECG-language model that simplifies architecture by replacing ECG encoders with a single projection layer, achieving competitive performance and revealing reliance on artifacts over true ECG information.

Contribution

This work introduces ELF, a novel encoder-free ECG-language model that simplifies design and maintains state-of-the-art performance, challenging the necessity of complex encoders.

Findings

ELF matches or exceeds state-of-the-art models across five datasets.

Adding architectural biases does not significantly improve ELF.

Current evaluation may overestimate ECG information due to reliance on artifacts.

Abstract

ECG-Language Models (ELMs) extend recent progress in Multimodal Large Language Models (MLLMs) to automated ECG interpretation. However, most ELMs follow Vision-Language Model (VLM) designs and depend on pretrained ECG encoders, adding architectural and training complexity. Inspired by encoder-free VLMs, we introduce ELF, an encoder-free ELM that replaces the ECG encoder with a single projection layer trained jointly with the LLM. Across five datasets, ELF matches or exceeds state-of-the-art ELMs that use far more complex encoders and training pipelines. We also test whether adding architectural biases to ELF improves performance and find that the single linear projection remains competitive. Finally, we show that ELF, and potentially other ELMs, often rely more on benchmark artifacts and language priors than ECG-derived information, highlighting limitations in current evaluation practices and ELM design. All data and code is available at https://github.com/willxxy/ECG-Bench.