Audio Spectral Enhancement: Leveraging Autoencoders for Low Latency Reconstruction of Long, Lossy Audio Sequences

TL;DR

This paper introduces a novel autoencoder-based method for spectral enhancement of low-quality, long audio sequences, improving reconstruction quality and efficiency for real-world applications.

Contribution

It presents a new autoencoder architecture with skip-connections for spectral inpainting, along with techniques for model size reduction and optimized loss selection.

Findings

Effective reconstruction of high-frequency audio from low-quality sequences

Model size reduced by over 50% with maintained performance

Faster inference suitable for real-world deployment

Abstract

With active research in audio compression techniques yielding substantial breakthroughs, spectral reconstruction of low-quality audio waves remains a less indulged topic. In this paper, we propose a novel approach for reconstructing higher frequencies from considerably longer sequences of low-quality MP3 audio waves. Our technique involves inpainting audio spectrograms with residually stacked autoencoder blocks by manipulating individual amplitude and phase values in relation to perceptual differences. Our architecture presents several bottlenecks while preserving the spectral structure of the audio wave via skip-connections. We also compare several task metrics and demonstrate our visual guide to loss selection. Moreover, we show how to leverage differential quantization techniques to reduce the initial model size by more than half while simultaneously reducing inference time, which is crucial in real-world applications.