Fine-tuning Pre-trained Audio Models for COVID-19 Detection: A Technical Report

TL;DR

This study evaluates pre-trained audio models for COVID-19 detection, revealing limited generalization across datasets and emphasizing the importance of demographic controls and larger datasets for reliable performance.

Contribution

It demonstrates the impact of demographic balancing on model evaluation and highlights the challenges of developing generalizable audio-based COVID-19 detection models.

Findings

Moderate intra-dataset performance with Audio-MAE on Coswara (AUC 0.82)

Limited cross-dataset generalization (AUC 0.43-0.68)

Demographic balancing reduces performance metrics but offers more realistic assessment

Abstract

This technical report investigates the performance of pre-trained audio models on COVID-19 detection tasks using established benchmark datasets. We fine-tuned Audio-MAE and three PANN architectures (CNN6, CNN10, CNN14) on the Coswara and COUGHVID datasets, evaluating both intra-dataset and cross-dataset generalization. We implemented a strict demographic stratification by age and gender to prevent models from exploiting spurious correlations between demographic characteristics and COVID-19 status. Intra-dataset results showed moderate performance, with Audio-MAE achieving the strongest result on Coswara (0.82 AUC, 0.76 F1-score), while all models demonstrated limited performance on Coughvid (AUC 0.58-0.63). Cross-dataset evaluation revealed severe generalization failure across all models (AUC 0.43-0.68), with Audio-MAE showing strong performance degradation (F1-score 0.00-0.08). Our experiments demonstrate that demographic balancing, while reducing apparent model performance, provides more realistic assessment of COVID-19 detection capabilities by eliminating demographic leakage - a confounding factor that inflate performance metrics. Additionally, the limited dataset sizes after balancing (1,219-2,160 samples) proved insufficient for deep learning models that typically require substantially larger training sets. These findings highlight fundamental challenges in developing generalizable audio-based COVID-19 detection systems and underscore the importance of rigorous demographic controls for clinically robust model evaluation.