COMPRESSIVE DATA STORAGE FOR LONG-TERM EEG: VALIDATION BY VISUAL ANALYSIS

TL;DR

This paper shows that long-term EEG data can be compressed 20 times without losing important visual diagnostic features, enabling efficient storage and new scientific insights.

Contribution

A novel data-analytic pipeline using SVD and DCT achieves 20-fold compression of long-term EEG data while preserving diagnostic quality.

Findings

A 20-fold compression of long-term EEG data was achieved without compromising visual diagnostic features.

Reconstructed data using the second compression regime showed no significant difference in diagnostic scores compared to original data.

The latent space from compressed data may offer new scientific insights into acute neurological illness.

Abstract

•Long-term EEG monitoring (LTM) accrues massive data volumes that are challenging to permanently archive in their entirety.•Analytic techniques can achieve a 20-fold compression of LTM data size without compromising visually diagnostic features.•The latent space may suggest new scientific questions in the EEG of acute neurological illness.

Long-term EEG monitoring (LTM) accrues massive data volumes that are challenging to permanently archive in their entirety.

Analytic techniques can achieve a 20-fold compression of LTM data size without compromising visually diagnostic features.

The latent space may suggest new scientific questions in the EEG of acute neurological illness.

Long-term EEG monitoring (LTM) in acute neurology generates massive data volumes. We investigated whether data-analytic techniques could reduce LTM data size yet conserve their visual diagnostic features.

LTM exemplars from 50 patients underwent singular value decomposition (SVD). High-variance SVD components were transformed using discrete cosine transform (DCT), and significant elements run-length encoded. Two regimes were tested: (I) SVD and DCT compression ratio (CR) of 1.7 and 12, and (II) CR of 3.7 and 5.7; each achieved an overall CR of ≈20. Compressed data were reconstructed alongside uncompressed originals, to create a total of 200 recordings that were scored by two blinded reviewers. Scores of original and reconstructed data were statistically analyzed.

Score differences between original recordings were smaller than comparisons involving reconstructions using the first regime but did not differ significantly from reconstructions using the second regime.

Raw LTM EEG has sufficient redundancy to undergo extreme (20-fold) data compression without compromising visual diagnostic information. A balanced mix of SVD and DCT appears to be a suitable data-analytic pipeline for achieving such compression.

Dimension reduction is a significant goal in managing big biomedical data. Our results suggest a pathway for archival of meaningful representations of entire LTM datasets. The latent space suggests new lines of data-scientific inquiry of the EEG in acute neurological illness.

Figures