MELEP: A Novel Predictive Measure of Transferability in Multi-Label ECG Diagnosis

TL;DR

This paper introduces MELEP, a new transferability measure for multi-label ECG diagnosis that efficiently predicts model performance, aiding in selecting suitable pre-trained models for ECG classification tasks.

Contribution

The paper presents MELEP, the first transferability metric tailored for multi-label ECG classification, capable of predicting model performance with high correlation using only a single forward pass.

Findings

MELEP correlates strongly with actual model performance (F1 scores).

It is computationally efficient and works with different label sets.

Effective for small, imbalanced ECG datasets.

Abstract

In practical electrocardiography (ECG) interpretation, the scarcity of well-annotated data is a common challenge. Transfer learning techniques are valuable in such situations, yet the assessment of transferability has received limited attention. To tackle this issue, we introduce MELEP, which stands for Muti-label Expected Log of Empirical Predictions, a measure designed to estimate the effectiveness of knowledge transfer from a pre-trained model to a downstream multi-label ECG diagnosis task. MELEP is generic, working with new target data with different label sets, and computationally efficient, requiring only a single forward pass through the pre-trained model. To the best of our knowledge, MELEP is the first transferability metric specifically designed for multi-label ECG classification problems. Our experiments show that MELEP can predict the performance of pre-trained convolutional and recurrent deep neural networks, on small and imbalanced ECG data. Specifically, we observed strong correlation coefficients (with absolute values exceeding 0.6 in most cases) between MELEP and the actual average F1 scores of the fine-tuned models. Our work highlights the potential of MELEP to expedite the selection of suitable pre-trained models for ECG diagnosis tasks, saving time and effort that would otherwise be spent on fine-tuning these models.