Comparison of Missing Data Imputation Methods using the Framingham Heart study dataset

TL;DR

This study compares advanced GAN and Autoencoder-based missing data imputation methods on the Framingham Heart Study dataset, demonstrating significant improvements over traditional techniques in both imputation accuracy and predictive performance.

Contribution

It introduces modified GAN and Autoencoder methods for missing data imputation and evaluates their effectiveness on medical datasets, showing notable performance gains.

Findings

Improvements of 0.20 in normalized RMSE

7.00% increase in AUROC

2.50% higher F1-score in post-imputation prediction

Abstract

Cardiovascular disease (CVD) is a class of diseases that involve the heart or blood vessels and according to World Health Organization is the leading cause of death worldwide. EHR data regarding this case, as well as medical cases in general, contain missing values very frequently. The percentage of missingness may vary and is linked with instrument errors, manual data entry procedures, etc. Even though the missing rate is usually significant, in many cases the missing value imputation part is handled poorly either with case-deletion or with simple statistical approaches such as mode and median imputation. These methods are known to introduce significant bias, since they do not account for the relationships between the dataset's variables. Within the medical framework, many datasets consist of lab tests or patient medical tests, where these relationships are present and strong. To address these limitations, in this paper we test and modify state-of-the-art missing value imputation methods based on Generative Adversarial Networks (GANs) and Autoencoders. The evaluation is accomplished for both the tasks of data imputation and post-imputation prediction. Regarding the imputation task, we achieve improvements of 0.20, 7.00% in normalised Root Mean Squared Error (RMSE) and Area Under the Receiver Operating Characteristic Curve (AUROC) respectively. In terms of the post-imputation prediction task, our models outperform the standard approaches by 2.50% in F1-score.