Pediatric Automatic Sleep Staging: A comparative study of state-of-the-art deep learning methods

TL;DR

This study compares state-of-the-art deep learning methods for pediatric sleep staging, demonstrating that ensemble models achieve expert-level accuracy and robustness, with potential clinical relevance and limited room for further improvement.

Contribution

It provides a large-scale comparative analysis of multiple deep learning models for pediatric sleep staging, highlighting ensemble methods' effectiveness and robustness across different conditions.

Findings

Ensemble models reach 88.8% accuracy and Cohen's kappa of 0.852.

Models are robust to data variations over time and clinical interventions.

Automatic staging shows near-perfect agreement with expert annotations.

Abstract

Background: Despite the tremendous progress recently made towards automatic sleep staging in adults, it is currently unknown if the most advanced algorithms generalize to the pediatric population, which displays distinctive characteristics in overnight polysomnography (PSG). Methods: To answer the question, in this work, we conduct a large-scale comparative study on the state-of-the-art deep learning methods for pediatric automatic sleep staging. Six different deep neural networks with diverging features are adopted to evaluate a sample of more than 1,200 children across a wide spectrum of obstructive sleep apnea (OSA) severity. Results: Our experimental results show that the individual performance of automated pediatric sleep stagers when evaluated on new subjects is equivalent to the expert-level one reported on adults. Combining the six stagers into ensemble models further boosts the staging accuracy, reaching an overall accuracy of 88.8%, a Cohen's kappa of 0.852, and a macro F1-score of 85.8%. At the same time, the ensemble models lead to reduced predictive uncertainty. The results also show that the studied algorithms and their ensembles are robust to concept drift when the training and test data were recorded seven months apart and after clinical intervention. Conclusion: However, we show that the improvements in the staging performance are not necessarily clinically significant although the ensemble models lead to more favorable clinical measures than the six standalone models. Significance: Detailed analyses further demonstrate "almost perfect" agreement between the automatic stagers to one another and their similar patterns on the staging errors, suggesting little room for improvement.