A novel audio representation using space filling curves

TL;DR

This paper introduces a new method for converting audio waveforms into 2D images using space filling curves, enhancing CNN-based audio processing by preserving local structure and leveraging vision models.

Contribution

Proposes a novel audio-to-image mapping using space filling curves, demonstrating improved CNN performance and shift equivariance in keyword spotting tasks.

Findings

Z curve performs best among tested SFCs.

Comparable results to mel frequency cepstral coefficients.

Preserves local structure without signal compression.

Abstract

Since convolutional neural networks (CNNs) have revolutionized the image processing field, they have been widely applied in the audio context. A common approach is to convert the one-dimensional audio signal time series to two-dimensional images using a time-frequency decomposition method. Also it is common to discard the phase information. In this paper, we propose to map one-dimensional audio waveforms to two-dimensional images using space filling curves (SFCs). These mappings do not compress the input signal, while preserving its local structure. Moreover, the mappings benefit from progress made in deep learning and the large collection of existing computer vision networks. We test eight SFCs on two keyword spotting problems. We show that the Z curve yields the best results due to its shift equivariance under convolution operations. Additionally, the Z curve produces comparable results to the widely used mel frequency cepstral coefficients across multiple CNNs.