SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification

TL;DR

SleepTransformer is an interpretable transformer-based model for automatic sleep staging that includes uncertainty quantification, making it suitable for clinical use by providing insights and confidence measures for its decisions.

Contribution

This work introduces SleepTransformer, a novel sequence-to-sequence transformer model with interpretability at both epoch and sequence levels, and a simple entropy-based uncertainty quantification method.

Findings

Performs comparably to existing methods on two sleep datasets.

Provides interpretable attention heat maps highlighting sleep features.

Quantifies decision confidence to identify low-confidence epochs.

Abstract

Background: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. Methods: Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. Results: Making sense of the transformer's self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. Conclusion: Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. Significance: Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.