A Multidatabase ExTRaction PipEline (METRE) for Facile Cross Validation in Critical Care Research

TL;DR

This paper introduces METRE, an open-source extraction pipeline that simplifies cross-institutional EHR data analysis from MIMIC-IV and eICU, enabling robust model validation for critical care research.

Contribution

The paper presents a novel, versatile extraction pipeline that supports cross-validation across multiple ICU databases, addressing limitations of prior tools and facilitating clinical model deployment.

Findings

Extracted over 38,000 MIMIC-IV ICU records and 126,000 eICU records.

Achieved comparable AUC performance (0.723-0.888) on mortality prediction tasks.

Demonstrated minimal AUC variation when applying models across datasets.

Abstract

Transforming raw EHR data into machine learning model-ready inputs requires considerable effort. One widely used EHR database is Medical Information Mart for Intensive Care (MIMIC). Prior work on MIMIC-III cannot query the updated and improved MIMIC-IV version. Besides, the need to use multicenter datasets further highlights the challenge of EHR data extraction. Therefore, we developed an extraction pipeline that works on both MIMIC-IV and eICU Collaborative Research Database and allows for model cross validation using these 2 databases. Under the default choices, the pipeline extracted 38766 and 126448 ICU records for MIMIC-IV and eICU, respectively. Using the extracted time-dependent variables, we compared the Area Under the Curve (AUC) performance with prior works on clinically relevant tasks such as in-hospital mortality prediction. METRE achieved comparable performance with AUC 0.723- 0.888 across all tasks. Additionally, when we evaluated the model directly on MIMIC-IV data using a model trained on eICU, we observed that the AUC change can be as small as +0.019 or -0.015. Our open-source pipeline transforms MIMIC-IV and eICU into structured data frames and allows researchers to perform model training and testing using data collected from different institutions, which is of critical importance for model deployment under clinical contexts.