ALPEC: A Comprehensive Evaluation Framework and Dataset for Machine Learning-Based Arousal Detection in Clinical Practice

TL;DR

This paper introduces ALPEC, a new evaluation framework and dataset for sleep arousal detection using machine learning, emphasizing clinical relevance by focusing on arousal onsets and addressing annotation mismatches.

Contribution

It presents a novel post-processing and evaluation framework tailored for clinical needs and releases a comprehensive multimodal dataset for arousal onset detection.

Findings

Focusing on arousal onsets improves clinical applicability.

The new dataset reflects clinical annotation constraints and includes diverse modalities.

The framework enhances the integration of ML-based arousal detection in clinical practice.

Abstract

Detecting arousals in sleep is essential for diagnosing sleep disorders. However, using Machine Learning (ML) in clinical practice is impeded by fundamental issues, primarily due to mismatches between clinical protocols and ML methods. Clinicians typically annotate only the onset of arousals, while ML methods rely on annotations for both the beginning and end. Additionally, there is no standardized evaluation methodology tailored to clinical needs for arousal detection models. This work addresses these issues by introducing a novel post-processing and evaluation framework emphasizing approximate localization and precise event count (ALPEC) of arousals. We recommend that ML practitioners focus on detecting arousal onsets, aligning with clinical practice. We examine the impact of this shift on current training and evaluation schemes, addressing simplifications and challenges. We utilize a novel comprehensive polysomnographic dataset (CPS) that reflects the aforementioned clinical annotation constraints and includes modalities not present in existing polysomnographic datasets. We release the dataset alongside this paper, demonstrating the benefits of leveraging multimodal data for arousal onset detection. Our findings significantly contribute to integrating ML-based arousal detection in clinical settings, reducing the gap between technological advancements and clinical needs.