Naturalistic sleep tracking in a longitudinal cohort: Uncertainty and bias in short duration sampling

TL;DR

Short-term sleep tracking introduces significant uncertainty and bias in sleep duration estimates, which can be reduced by longer observation periods.

Contribution

The study quantifies the bias and uncertainty in sleep metrics when using short observation periods and shows how these depend on sample size, statistics, and distribution shape.

Findings

Short observation periods (e.g., 7 nights) lead to underestimation of sleep duration standard deviation and large uncertainty ranges.

Single-night observations are more likely to misclassify typical sleepers as short sleepers due to natural variability.

Longer tracking (up to 365 nights) reduces bias and uncertainty in sleep health metrics.

Abstract

Despite broad interest in the health implications of sleep duration, traditional measurements via polysomnography or actigraphy are often limited to one or a few nights per person. Inferential uncertainty remains an important issue for interpreting descriptive statistics in this common research setting.

This retrospective analysis of observational data used a combined approach of simulated data and real-world data (30–365 nights) analysis from over 35,000 participants who provided informed consent to participate in the Apple Heart and Movement Study and elected to contribute sleep data.

Simulations demonstrate that the degree of uncertainty and bias, compared to truth defined by 1000 simulated nights, depended on several factors: sub-sample size, the simulated distribution (normal versus skewed), and the computed metrics of central tendency (mean, median) and dispersion (standard deviation (SD), interquartile range (IQR)). For example, the SD computed from n = 7 observations from a simulated normal distribution (7 ± 1 hours) showed a median 6.7% under-estimation bias, and an uncertainty range with IQR from 24% under- to 14.7% over-estimation. Defining ground truth with a small sample (7–14 nights) yielded overly optimistic estimates of bias and uncertainty when sub-sampled. Real-world sleep duration data, when randomly sub-sampled and compared to longer observations within-participant, showed similar SD bias and rates of convergence as the normal distribution simulations. Sub-sampled sleep stage durations also varied substantially from “true” values computed from longer observations. Finally, simulated cohorts with sleep durations of 7 ± 1 hours mixed with a subset of 6 ± 1 hours sleepers showed that a random single-night observation of “short sleep” (6 hours) is more likely from random variation of a 7-hour sleeper, than from an actual 6-hour sleeper. Extending the mean duration calculation to n = 7 nights mitigates this mis-classification risk.

The simulation and empiric data approaches both suggest that bias and uncertainty due to sub-sampling depend on: a) the sample size of observations within each participant, b) the descriptive statistic used to capture centrality or dispersion, and c) the distribution shape of the data (normal or skewed). Longer duration tracking provides important and tangible benefits to reduce bias and uncertainty in sleep health research that historically relies on small observation windows.

Figures