MSA-CNN: A Lightweight Multi-Scale CNN with Attention for Sleep Stage Classification

TL;DR

This paper introduces MSA-CNN, a lightweight multi-scale CNN with attention mechanisms designed for sleep stage classification, achieving high accuracy with significantly fewer parameters than existing models.

Contribution

The paper presents a novel, highly efficient multi-scale CNN architecture that reduces model complexity while maintaining superior performance in sleep stage classification.

Findings

MSA-CNN outperforms nine baseline models in accuracy and Cohen's kappa.

The large MSA-CNN achieves state-of-the-art results with only ~10,000 parameters.

Model analysis reveals key modules contributing to performance improvements.

Abstract

Recent advancements in machine learning-based signal analysis, coupled with open data initiatives, have fuelled efforts in automatic sleep stage classification. Despite the proliferation of classification models, few have prioritised reducing model complexity, which is a crucial factor for practical applications. In this work, we introduce Multi-Scale and Attention Convolutional Neural Network (MSA-CNN), a lightweight architecture featuring as few as ~10,000 parameters. MSA-CNN leverages a novel multi-scale module employing complementary pooling to eliminate redundant filter parameters and dense convolutions. Model complexity is further reduced by separating temporal and spatial feature extraction and using cost-effective global spatial convolutions. This separation of tasks not only reduces model complexity but also mirrors the approach used by human experts in sleep stage scoring. We evaluated both small and large configurations of MSA-CNN against nine state-of-the-art baseline models across three public datasets, treating univariate and multivariate models separately. Our evaluation, based on repeated cross-validation and re-evaluation of all baseline models, demonstrated that the large MSA-CNN outperformed all baseline models on all three datasets in terms of accuracy and Cohen's kappa, despite its significantly reduced parameter count. Lastly, we explored various model variants and conducted an in-depth analysis of the key modules and techniques, providing deeper insights into the underlying mechanisms. The code for our models, baselines, and evaluation procedures is available at https://github.com/sgoerttler/MSA-CNN.