U-PASS: an Uncertainty-guided deep learning Pipeline for Automated Sleep Staging

TL;DR

U-PASS is a comprehensive deep learning pipeline that incorporates uncertainty estimation at all stages to improve sleep staging accuracy and reliability in clinical applications, especially for challenging datasets.

Contribution

The paper introduces U-PASS, a novel uncertainty-guided pipeline for sleep staging that enhances performance through active sample selection and uncertainty-aware training.

Findings

Achieved 85% accuracy on elderly sleep apnea dataset.

Systematically improved performance at all pipeline stages.

Demonstrated the effectiveness of uncertainty-guided training in clinical sleep staging.

Abstract

As machine learning becomes increasingly prevalent in critical fields such as healthcare, ensuring the safety and reliability of machine learning systems becomes paramount. A key component of reliability is the ability to estimate uncertainty, which enables the identification of areas of high and low confidence and helps to minimize the risk of error. In this study, we propose a machine learning pipeline called U-PASS tailored for clinical applications that incorporates uncertainty estimation at every stage of the process, including data acquisition, training, and model deployment. The training process is divided into a supervised pre-training step and a semi-supervised finetuning step. We apply our uncertainty-guided deep learning pipeline to the challenging problem of sleep staging and demonstrate that it systematically improves performance at every stage. By optimizing the training dataset, actively seeking informative samples, and deferring the most uncertain samples to an expert, we achieve an expert-level accuracy of 85% on a challenging clinical dataset of elderly sleep apnea patients, representing a significant improvement over the baseline accuracy of 75%. U-PASS represents a promising approach to incorporating uncertainty estimation into machine learning pipelines, thereby improving their reliability and unlocking their potential in clinical settings.