SleepLM: Natural-Language Intelligence for Human Sleep

TL;DR

SleepLM introduces a novel multimodal foundation model that bridges natural language and sleep physiology, enabling advanced sleep analysis, interpretation, and interaction beyond traditional label-based systems.

Contribution

The paper presents SleepLM, the first large-scale sleep-text dataset and a unified pretraining approach for language-grounded sleep analysis.

Findings

Outperforms state-of-the-art in zero-shot and few-shot sleep understanding tasks.

Enables language-guided sleep event localization and insight generation.

Demonstrates zero-shot generalization to unseen sleep analysis tasks.

Abstract

We present SleepLM, a family of sleep-language foundation models that enable human sleep alignment, interpretation, and interaction with natural language. Despite the critical role of sleep, learning-based sleep analysis systems operate in closed label spaces (e.g., predefined stages or events) and fail to describe, query, or generalize to novel sleep phenomena. SleepLM bridges natural language and multimodal polysomnography, enabling language-grounded representations of sleep physiology. To support this alignment, we introduce a multilevel sleep caption generation pipeline that enables the curation of the first large-scale sleep-text dataset, comprising over 100K hours of data from more than 10,000 individuals. Furthermore, we present a unified pretraining objective that combines contrastive alignment, caption generation, and signal reconstruction to better capture physiological fidelity and cross-modal interactions. Extensive experiments on real-world sleep understanding tasks verify that SleepLM outperforms state-of-the-art in zero-shot and few-shot learning, cross-modal retrieval, and sleep captioning. Importantly, SleepLM also exhibits intriguing capabilities including language-guided event localization, targeted insight generation, and zero-shot generalization to unseen tasks. All code and data will be open-sourced.