Perfectly predicting ICU length of stay: too good to be true

TL;DR

This paper critically examines a study claiming perfect ICU length of stay prediction, revealing methodological flaws that inflate results and providing a more realistic AUROC of 88.91%, emphasizing the importance of rigorous methodology.

Contribution

The paper identifies and corrects methodological flaws in a previous ICU LOS prediction study, offering a more credible performance estimate and highlighting issues in reporting and reproducibility.

Findings

Original study claimed 100% AUROC, which is overly optimistic.

Corrected analysis shows an AUROC of approximately 88.91%.

Methodological flaws significantly impacted the reported results.

Abstract

A paper of Alsinglawi et al was recently accepted and published in Scientific Reports. In this paper, the authors aim to predict length of stay (LOS), discretized into either long (> 7 days) or short stays (< 7 days), of lung cancer patients in an ICU department using various machine learning techniques. The authors claim to achieve perfect results with an Area Under the Receiver Operating Characteristic curve (AUROC) of 100% with a Random Forest (RF) classifier with ADASYN class balancing over sampling technique, which if accurate could have significant implications for hospital management. However, we have identified several methodological flaws within the manuscript which cause the results to be overly optimistic and would have serious consequences if used in a clinical practice. Moreover, the reporting of the methodology is unclear and many important details are missing from the manuscript, which makes reproduction extremely difficult. We highlight the effect these oversights have had on the result and provide a more believable result of 88.91% AUROC when these oversights are corrected.