Generative Pre-Trained Transformer for Cardiac Abnormality Detection

TL;DR

This paper introduces a transformer-based model for classifying cardiac abnormalities from ECG signals, leveraging attention mechanisms to interpret periodic time series data for multi-lead heartbeat diagnosis.

Contribution

It adapts the Transformer Encoder architecture to ECG heartbeat classification by treating periodic signals as sequences of words, achieving competitive results in a cardiac diagnosis challenge.

Findings

Achieved low error scores across multiple ECG lead configurations.

Demonstrated the effectiveness of transformer attention in time series classification.

Provided a novel approach linking NLP models to biomedical signal analysis.

Abstract

ECG heartbeat classification plays a vital role in diagnosis of cardiac arrhythmia. The goal of the Physionet/CinC 2021 challenge was to accurately classify clinical diagnosis based on 12, 6, 4, 3 or 2-lead ECG recordings in order to aid doctors in the diagnoses of different heart conditions. Transformers have had great success in the field of natural language processing in the past years. Our team, CinCSEM, proposes to draw the parallel between text and periodic time series signals by viewing the repeated period as words and the whole signal as a sequence of such words. In this way, the attention mechanisms of the transformers can be applied to periodic time series signals. In our implementation, we follow the Transformer Encoder architecture, which combines several encoder layers followed by a dense layer with linear or sigmoid activation for generative pre-training or classification, respectively. The use case presented here is multi-label classification of heartbeat abnormalities of ECG recordings shared by the challenge. Our best entry, not exceeding the challenge's hardware limitations, achieved a score of 0.12, 0.07, 0.10, 0.10 and 0.07 on 12-lead, 6-lead, 4-lead, 3-lead and 2-lead test set respectively. Unfortunately, our team was unable to be ranked because of a missing pre-print.