Benchmark of Deep Learning Models on Large Healthcare MIMIC Datasets

TL;DR

This paper benchmarks deep learning models against traditional machine learning and scoring systems on the MIMIC-III ICU dataset, demonstrating superior performance of deep models on clinical prediction tasks using raw time series data.

Contribution

It provides a comprehensive benchmarking of deep learning models versus existing methods on multiple clinical prediction tasks using a large public healthcare dataset.

Findings

Deep learning models outperform traditional methods on clinical prediction tasks.

Raw time series data enhances deep learning model performance.

Benchmark results establish new standards for ICU prediction models.

Abstract

Deep learning models (aka Deep Neural Networks) have revolutionized many fields including computer vision, natural language processing, speech recognition, and is being increasingly used in clinical healthcare applications. However, few works exist which have benchmarked the performance of the deep learning models with respect to the state-of-the-art machine learning models and prognostic scoring systems on publicly available healthcare datasets. In this paper, we present the benchmarking results for several clinical prediction tasks such as mortality prediction, length of stay prediction, and ICD-9 code group prediction using Deep Learning models, ensemble of machine learning models (Super Learner algorithm), SAPS II and SOFA scores. We used the Medical Information Mart for Intensive Care III (MIMIC-III) (v1.4) publicly available dataset, which includes all patients admitted to an ICU at the Beth Israel Deaconess Medical Center from 2001 to 2012, for the benchmarking tasks. Our results show that deep learning models consistently outperform all the other approaches especially when the `raw' clinical time series data is used as input features to the models.