Using growth transform dynamical systems for spatio-temporal data sonification

TL;DR

This paper introduces a novel sonification framework that uses growth transform dynamical systems to encode high-dimensional data and learning processes into binaural audio, aiding decision-making and anomaly detection.

Contribution

It presents an integrated approach combining data-driven optimization with psychoacoustic parameters to produce meaningful audio signatures for complex datasets.

Findings

Successfully encodes statistical properties of data into audio

Reveals underlying learning or optimization processes through sound

Demonstrates potential for epileptic seizure detection in EEG data

Abstract

Sonification, or encoding information in meaningful audio signatures, has several advantages in augmenting or replacing traditional visualization methods for human-in-the-loop decision-making. Standard sonification methods reported in the literature involve either (i) using only a subset of the variables, or (ii) first solving a learning task on the data and then mapping the output to an audio waveform, which is utilized by the end-user to make a decision. This paper presents a novel framework for sonifying high-dimensional data using a complex growth transform dynamical system model where both the learning (or, more generally, optimization) and the sonification processes are integrated together. Our algorithm takes as input the data and optimization parameters underlying the learning or prediction task and combines it with the psychoacoustic parameters defined by the user. As a result, the proposed framework outputs binaural audio signatures that not only encode some statistical properties of the high-dimensional data but also reveal the underlying complexity of the optimization/learning process. Along with extensive experiments using synthetic datasets, we demonstrate the framework on sonifying Electro-encephalogram (EEG) data with the potential for detecting epileptic seizures in pediatric patients.